Pickle. EOFError: Ran out of input. Can I recover any information from a corrupt pickle file?

Viewed 53

Similar questions have been asked numerous times here, but this is slightly different. I have a pickle file that got somehow corrupted. I assume the program crashed during saving. Either way, the file is not empty, and judging by its size has all/most of the information in it. Trying to load is with pickle gets me a standard EOFError: Ran out of input error.

I understand that the file is corrupted and understand how most likely it happened. My question is if there is a way to somehow recover any of it?

1 Answers

Maybe try this code. First you will have to pad your file (with zeros) to be at least the original size, can be more (or maybe you can change some code to make this not necessary).

I modified the function _Unpickler.load() to not terminate at exceptions and to return its stack if the unpickling fails. The stack seems to be a list with the names and values of the attributes, so using setattr() you can assign them to a created object.

My example creates an object to pickle if read=False and unpickles the potentially corrupted file if read=True.

import pickle

read = False # run once with False, then corrupt file and change to True 

class AClass:
    def __init__(unpickler, a, b, c):
        unpickler.a = a
        unpickler.b = b
        unpickler.c = c


def load(unpickler):
    """Read a pickled object representation from the open file.

    Return the reconstituted object hierarchy specified in the file.
    """
    # Check whether Unpickler was initialized correctly. This is
    # only needed to mimic the behavior of _pickle.Unpickler.dump().
    if not hasattr(unpickler, "_file_read"):
        raise pickle.UnpicklingError("Unpickler.__init__() was not called by "
                              "%s.__init__()" % (unpickler.__class__.__name__,))
    unpickler._unframer = pickle._Unframer(unpickler._file_read, unpickler._file_readline)
    unpickler.read = unpickler._unframer.read
    unpickler.readline = unpickler._unframer.readline
    unpickler.metastack = []
    unpickler.stack = []
    unpickler.append = unpickler.stack.append
    unpickler.proto = 0
    read = unpickler.read
    dispatch = unpickler.dispatch
    try:
        while True:
            key = read(1)
            if not key:
                return unpickler.stack
                #raise EOFError
            assert isinstance(key, pickle.bytes_types)
            try:
                dispatch[key[0]](unpickler)
            except KeyError as e:
                print("KeyError")
                #dispatch[NONE[0]](unpickler)
            except ValueError as e:
                print("ValueError")
    except pickle._Stop as stopinst:
        print("Stop raised")
        print(stopinst.value)
        return stopinst.value


if not read:
    # create an object and save it
    obj = AClass(1,2,3) 

    with open("obj.pkl", 'wb') as outFile:
        pickle.dump(obj, outFile, pickle.HIGHEST_PROTOCOL)
else:
    # open the object 
     with (open("obj.pkl", "rb")) as inFile:
        try:
            unpickler = pickle._Unpickler(inFile)
            obj = load(unpickler)
        except Exception as e:
            print(e)
            
# a list will be returned, if the unpickling fails
if isinstance(obj, list):
    l = obj
    print(l)
    i = 0
    obj = AClass(0,0,0)

    while i < len(l)-1:
        setattr(obj, l[i], l[i+1])
        i += 2
            
print(obj.a, obj.b, obj.c)
Related