I have different types of data sources, relational data from MySQL and unstructured data from Elasticsearch and more data sources will be added. I want to create a data lake to further processing and analyzing (with quick queries or by training machine learning models.)
The point is, that I want to keep my current data storage but at the same time, I want to keep my data lake updated. It doesn't need to be perfectly synced with the live data, it can be updated hourly or daily (in the worst case.)
My question is, as in the title, what is the best way to create a data lake, which retrieves data from different data sources that store different types of data and keep the lake updated?
What I found out from my research is Google Cloud with BigQuery and Amazon S3 with Redshift look promising for this purpose. Once the data lake is created in the warehouse, the rest looks pretty straightforward, at least not tricky as merging these data and keeping them updated. They offer to use pipelines for migration and update operations but they are not really supporting external data storage, for example, Amazon wants you to move your MySQL to RDS, then it allows you to migrate data from RDS MySQL to S3 Bucket, which makes things much more complicated and harder. Please let me know if they have easier ways for this purpose.