How to do information extraction from a fragmented text in python?

We can solve this with an algorithm/state machine. The following code opens your file called "datafile.txt" in the same directory as the python script, parses it, and prints the results. The keys to the algorithm are assuming that there are empty lines only between every two fields and that any line containing the beginning of a description field we want to record has its code attribute separated from its description attribute by three or more spaces. These assumptions are always going to be true as far as I can tell from your file snippet.

index = -1
record = False
description_block = False
codes = []
descriptions = []
with open("datafile.txt", "r") as file:
  for line in file:
    line = [portion.strip() for portion in line.split("   ") if portion != ""]
    if record:
      if len(line) == 2:
        index += 1
        codes.append(line[0])
        descriptions.append(line[1])
      else:
        if line[0]:
          description_block = True
        if description_block:
          if not line[0]:
            description_block = False
            record = False
            continue
          else:
            descriptions[index] += " "+line[0]
    if line[0] == "Code":
      record = True
print("codes:", codes)
print("descriptions:", descriptions)

Result:

codes: ['Blank', '1', 'Blank1', '11']
descriptions: ['Not reviewed, or reviewed and corrected', 'Reviewed and confirmed as reported: A patient had behavior code of in situ and laterality is not stated as right: origin of primary; left: origin of primary; or only one side involved, right or left origin not specified', 'Not reviewed', 'A patient had laterality coded non-specifically and extension coded specifically']

Tested in python 3.8.2

EDIT: Updating the code to reflect the whole data file as provided in the comments.

import re
column_separator = "     "
index = -1
record = False
block_exit = False
break_on_newline = False
codes = []
descriptions = []
templine = ""
def add(line):
  global index
  index += 1
  block_exit = False
  codes.append(line[0])
  descriptions.append(line[1])
with open("test", "r", encoding="utf-8") as file:
  while True:
    line = file.readline()
    if not line:
      break
    if record:
      line = [portion.strip() for portion in line.split(column_separator) if portion != ""]
      if len(line) > 1:
        add(line)
      else:
        if block_exit:
            record = False
            block_exit = False
        else:
          if line[0]:
            descriptions[index] += " "+line[0]
          else:
            while True:
              line = [portion.strip() for portion in file.readline().split(column_separator) if portion != ""]
              if not line:
                break
              if len(line) > 1:
                if templine:
                  descriptions[index] += templine
                  templine = ""
                add(line)
                break
              else:
                print(line)
                if line[0] and "Instructions" not in line[0]:
                  templine += " "+line[0]
                else:
                  if break_on_newline:
                    break_on_newline = False
                    record = False
                    templine = ""
                    break
                  else:
                    templine += " "+line[0]
                    break_on_newline = True
    else:
      if line == "Code           Description\n":
        record = True

print("codes:", codes)
print("\n")
print("descriptions:", descriptions)

# for i in range(len(codes)):
#   print(codes[i]+"\t\t", descriptions[i])

Try the following regex approach:

blanks = re.findall(r'\bCode\b.*?\bDescription\s*?(\S+)\s+.*?\r?\n(\d+)\s+.*?(?=\r?\n\r?\n)', inp, flags=re.DOTALL)
print(blanks)

reviews = re.findall(r'\bCode\b.*?\bDescription\s*?\S+\s+(.*?)\r?\n\d+\s+(.*?)(?=\r?\n\r?\n)', inp, flags=re.DOTALL)

This prints:

[('Blank', '1'), ('Blank1', '11')]

[('Not reviewed, or reviewed and corrected\n', 'Reviewed and confirmed as reported: A patient had behavior \ncode of in situ and laterality is not\nstated as right: origin of primary; left: origin of primary; or only one side \ninvolved, right or left\norigin not specified'), ('Not reviewed\n', 'A patient had laterality \ncoded non-specifically and\nextension coded specifically')]

The idea here is to just match and capture the various desired portions of the Code ... Description ... Blank portions of your input text. Note that this answer assumes that you have already read the text into a Python string variable.