I'm using Firehose to put a data stream into Redshift via S3. The Redshift cost is eye watering for the small amount of data I have at the moment, so I'm looking to pause Redshift for up to 23 hours a day.
Redshift has the ability to schedule periodic pause and resume, but the Firehose retry policy means once a Redshift retry fails, it simply records an error manifest in S3 and continues as usual, but only processing new data.
Is there any way to tell Firehose to re-try the records that failed to load?
Or alternatively, a way to tell Firehose to avoid running the copy command when Redshift is paused so that the data can be safely loaded when Redshift is running?
Appreciate any suggestions.