GCP dataflow has issue with pyorc library

Viewed 101

Working on a GCP project, I am trying to use pyorc library to save data in ORC format. I have noticed the dataflow pipeline works with DirectRunner (local machine) however DataFlow Runner does not work (there is no error but the job never ends). I have found the issue after some investigation and I noticed once I have pyorc==0.3.0 in requirements.txt file then dataflow does not work. As I mentioned there is no error but I believe it cant not install pyorc. Below is my requirements.txt.

google-cloud-dlp==0.13.0
google-cloud-language==1.3.0
google-cloud-translate==2.0.1
ratelimiter==1.2.0
xmltodict==0.12.0
pyyaml==5.3.1
google-api-python-client==1.12.5
google-cloud-logging==1.15.1
pyorc==0.3.0
0 Answers
Related