I want to use data stored in a BigQuery table as input to Google's Natural Language API, perform entity extraction and sentiment analysis, and persist the result back to BigQuery. What tools/services could I use to handle this in GCP? Performance is not a concern, and running this in an overnight batch would be acceptable for this use-case.
1 Answers
This should be fairly straightforward to do with Dataflow--you could write a pipeline that reads from BigQuery followed by a DoFn that uses Google's NLP Libraries, and then writes the results to BigQuery.
Some wrappers are already provided for you in https://github.com/apache/beam/blob/master/sdks/python/apache_beam/ml/gcp/naturallanguageml.py
Related