I read the book ‘Data Pipelines Pocket Reference’ from James Densmore. Like many other pipelines, this sample pipeline saves the data in the extract phase to csv on the local drive. Is this also how it would go in production? Saving the file first on the local machine and then upload it to a data lake or whatsoever?
1 Answers
If by "local drive" you mean a user's personal PC then no, that's not what you would do in Production. You would land the data on a server or in a cloud storage location.
Related