I'm a bit confused by the AWS EMR ecosystem and was wondering what tool I can use to set up a data sink/export to s3 where each "row" is a separate json file.
It seems like there are a bunch of tools to ingest and process the data, e.g. pig, hive, Athena, spark, etc... but once it's been processed, if I want to export out not to one big file, but 1 million files on s3, one per row - is there a good tool in the AWS toolset to use? Would AWS glue be able to handle this scenario? Could I just use a lambda, or would that possibly time out?