Assume I have Python stream-processing code that looks like this:
def F1(stream):
for x in stream:
yield f1(x)
def F2(stream):
for x in stream:
yield f2(x)
def F3(stream):
for x in stream:
yield f3(x)
def F4(stream):
for x in stream:
yield f4(x)
for x in F4(F3(F2(F1(range(1000000))))):
print(x)
This is roughly equivalent to range 1000000 | F1 | F2 | F3 | F4 in Unix (assuming a range command), but in Unix each process in the pipe runs in parallel.
Is there a simple way to parallelize the Python code?