I was checking the storage usage for our account and saw more than half is used by Fail Safe
What does it mean if more than %50 of the storage usage is used by Fail Safe?
I was checking the storage usage for our account and saw more than half is used by Fail Safe
What does it mean if more than %50 of the storage usage is used by Fail Safe?
Failsafe can be collated data after the time travel is over. For example if you deleted a number of tables, the Time Travel = 1 day. Immediately after this, the data is moved to Fail-Safe which can be 0 - 7 days. In case there are lot of data which is been modified, you will see Fail-safe storage to be high. In order to get a more perspective view of fail-safe, you need to monitor the data over a period of 7 days and see if this is deleting forever or not. https://docs.snowflake.com/en/user-guide/data-failsafe.html
At my list job 1 year ago (when i left) our total disk usage was 180TB 100TB was data (plus that days churn) thus 80Tb was time-travel/failsafe keeping of the churn. I recall we might have bill in the order 6K USD then was in the order of 10% of our monthly compute. So while we did things to improve it we spent more time focusing on compute savings as active tasks.
Now some of the costs where related to tables that had clustering turned on. We made most tables that where generated from primary sources as transient. As we could just rebuild those if “the bad things” happened.
But if you have 120TB of base data it seems you are churning 20TB a day. That seems rather high to me. As an ignore rant outsider. Now that might one days worth of change every thirty minutes. Or you might be rewriting one 20TB once a day. It is hard to judge. For us. Each of the spikes/churn was giving us something as a tradeoff and that saved more in compute cost (more or less). But we where happy. And our storage change was rather slow linear. That again is something that needs to be asked has it changed recently or is you system stable and you are looking to save some dollars.