I have an Athena table like this:
values_by_time:
- Columns:
idstring (contains UUIDs)valuestringcreatedtimestamp (this is a partition key)
- Partition:
[created]
But I want to query like this:
SELECT * FROM values_by_time WHERE id = 'ac09b809-471e-4296-b99b-e57408075609'
So my partition by [created] is inefficient.
I read that bucketing can be used for high cardinality columns, such as my id:
values_by_id:
- Columns:
idstring (bucket by this)createdtimestampvaluestring
- Bucket By:
[id]
This makes the look-up efficient:
SELECT * FROM values_by_id WHERE id = 'ac09b809-471e-4296-b99b-e57408075609'
However, I cannot do an INSERT INTO on a table with bucketing! This means that I cannot add new data to it after table creation. Creating the table fresh every day seems inefficient too.
How should I arrange my Athena table so that:
- I can efficiently query by
id - I can insert new data every day