Can I reset an iterator / generator in Python? I am using DictReader and would like to reset it to the beginning of the file.
Can I reset an iterator / generator in Python? I am using DictReader and would like to reset it to the beginning of the file.
One possible option is to use itertools.cycle(), which will allow you to iterate indefinitely without any trick like .seek(0).
iterDic = itertools.cycle(csv.DictReader(open('file.csv')))
I've had the same issue before. After analyzing my code, I realized that attempting to reset the iterator inside of loops slightly increases the time complexity and it also makes the code a bit ugly.
Open the file and save the rows to a variable in memory.
# initialize list of rows
rows = []
# open the file and temporarily name it as 'my_file'
with open('myfile.csv', 'rb') as my_file:
# set up the reader using the opened file
myfilereader = csv.DictReader(my_file)
# loop through each row of the reader
for row in myfilereader:
# add the row to the list of rows
rows.append(row)
Now you can loop through rows anywhere in your scope without dealing with an iterator.
I'm arriving at this same issue - while I like the tee() solution, I don't know how big my files are going to be and the memory warnings about consuming one first before the other are putting me off adopting that method.
Instead, I'm creating a pair of iterators using iter() statements, and using the first for my initial run-through, before switching to the second one for the final run.
So, in the case of a dict-reader, if the reader is defined using:
d = csv.DictReader(f, delimiter=",")
I can create a pair of iterators from this "specification" - using:
d1, d2 = iter(d), iter(d)
I can then run my 1st-pass code against d1, safe in the knowledge that the second iterator d2 has been defined from the same root specification.
I've not tested this exhaustively, but it appears to work with dummy data.
Return a newly created iterator at the last iteration during the 'iter()' call
class ResetIter:
def __init__(self, num):
self.num = num
self.i = -1
def __iter__(self):
if self.i == self.num-1: # here, return the new object
return self.__class__(self.num)
return self
def __next__(self):
if self.i == self.num-1:
raise StopIteration
if self.i <= self.num-1:
self.i += 1
return self.i
reset_iter = ResetRange(10)
for i in reset_iter:
print(i, end=' ')
print()
for i in reset_iter:
print(i, end=' ')
print()
for i in reset_iter:
print(i, end=' ')
Output:
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
0 1 2 3 4 5 6 7 8 9
The simplest solution possible: use deepcopy
from copy import deepcopy
iterator = your_iterator
# Start iteration
iterator_altered = deepcopy(iterator)
for _ in range(2):
a = next(iter(iterator_altered))
# Your iterator is still unaltered.
I think this is the simples approach.