ValueError: I/O operation on closed file within context manager scope

user_7407324 0
Viewed 582

I've read a lot of similar questions, but the majority of them were solved by fixing the indentation so either i'm clueless or there's some easy way of fixing my problem but i dont think it is about identation. So i have this function that basically performs a couple of operations using two *.txt files and returns a generator object of namedtuples with some info i need to look up later.

def master_reader(file1, file2):
    with open(file1, "r", encoding="utf-8") as genomas:
        with open(file2, "r", encoding="utf-8") as listas:
            l = parser_listas(listas)
            f = parser_genomas(genomas)
            f = map(intifier, f)
            f = (people_maker(linea, l) for linea in f)
            f = map(genotipo_getter, f)
            f = map(fen_getter, f)
            return f

The thing is everything works fine when i call it and assign it to a variable.But i need to use it as a parameter so i can call it everytime i need it for some queries i need to perform over it:

print(valor_caracteristica("TCT", "Luna  Lovegood", master_reader("genomas.txt", "listas.txt")))

But i am getting this exception:

Original exception was:
Traceback (most recent call last):
  File "lib.py", line 204, in <module>
    print(valor_caracteristica("TCT", "Luna  Lovegood", master_reader("genomas.txt", "listas.txt")))
  File "lib.py", line 194, in valor_caracteristica
    a = next(filter(lambda x: x.nombre == nombre, file))
  File "lib.py", line 185, in <genexpr>
    f = (people_maker(linea, l) for linea in f)
ValueError: I/O operation on closed file.
python-3.6 contextmanager with-statement python-3.x python
2 Answers

map() returns an iterator. Only when you loop over a map() object does it actually apply the function to the next elements of the input iterable.

As such, no data was read from your file until you started using the map object and underlying generator expression. You do so outside of the function, and the file is already closed by that time, because the return f statement exited the function and by extension the context.

The work-around is to either not use lazy objects like map(), or to make your function a generator function. The latter won't exit (and signal the with block to exit the context) until you are done with the file.

This can be done very simply by using yield from:

def master_reader(file1, file2):
    with open(file1, "r", encoding="utf-8") as genomas:
        with open(file2, "r", encoding="utf-8") as listas:
            l = parser_listas(listas)
            f = parser_genomas(genomas)
            f = map(intifier, f)
            f = (people_maker(linea, l) for linea in f)
            f = map(genotipo_getter, f)
            yield from map(fen_getter, f)

yield from keeps the generator open until the underlying map() object raises StopIteration.

A quick demo to illustrate the difference:

>>> from contextlib import contextmanager
>>> @contextmanager
... def democtx():
...     print('Entering the context')
...     yield
...     print('Exiting the context')
...
>>> def return_map():
...     with democtx():
...         return map(lambda x: x**2, range(3))
...
>>> def yield_from_map():
...     with democtx():
...         yield from map(lambda x: x**2, range(3))
...
>>> example1 = return_map()
Entering the context
Exiting the context
>>> example2 = yield_from_map()
>>> next(example2)
Entering the context
0
>>> next(example2)
1
>>> next(example2)
4
>>> next(example2)
Exiting the context
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

Note how for example1 the context was exited immediately upon return, while example2 did not open the context until iteration started, and did not close the context until we iterated in full over the map() object.

user_1002970

Since you're using Python 3, map constructs returns generators, which are lazily evaluated. Hence when the generator f is evaluated, file handler would already have been closed (context manager that opens the file makes sure of that).

The solution is to either evaluate the map therein, or not to use them at all, and use the list comprehension.

return list(f) # evaluate the map statement therein.

# or you should just return a generator as @MartinPieters suggested.
user_47257560
Related