TensorFlow Transform Python using AWS S3 as data source

Viewed 199

I am trying to run TensorFlow Transform, using Python, Apache Flink as the Beam Runner. I noticed that Beam does not have AWS S3 as the io connector, and would like to know any work around for this.

Here is the list of supported io connectors, but Python+S3 is not even in the roadmap.

I can think of two work around:

  1. mount the S3 bucket as a local drive to a EC2 instance
  2. Write my own Python S3 connector using their guide.

I want to know if there are other creative (easy) way out.

Thanks!

0 Answers
Related