I am new to dynamodb & was having some trouble in finding a way to randomly getting items without a full table scan ,most of the algorithms that i found consist of full table scans I am also taking the case where we don’t have additional information of the table(Like columns and column Type such info is unknown) Is there a way exist to do so
2 Answers
You can randomly sample by using a randomly generated exclusive start key for the scan or query operation. The exclusive start key does not have to match a record in the table. It just needs to follow the key structure of the table/index.
As with most questions about queries in DynamoDB, how you structure your data depends on how you want to query it.
For something like a random sampling, you have to make it confirm to the following core constraint of DynamoDB:
- You have to provide a partition key
- You can provide a sort key
So with a "single table" type design, you could structure your data something like this:
| PK | SK | myVal |
|---|---|---|
| my_dict | 6caaf1e3-eb8d-404a-a2ae-97d6682b0224 | foo |
| my_dict | 1c5496e8-c660-4b4e-980f-4abfb1942863 | bar |
| my_dict | 56551340-fff8-4824-a5be-70fcaece2e1a | baz |
| my_other_dict | 520a7b37-233c-49dd-87da-77d871d98c92 | test1 |
| my_other_dict | 65ccd54e-72c3-499d-a3a7-0cd989252607 | test2 |
The PK is the identifier for your collection of random things to look up. The SK is a random UUID. And myVal contains the value you want to be returned.
You can query this db the following way:
SELECT * FROM "my-table" WHERE PK = 'my_dict' AND SK < '06a04e20-b239-48f2-a205-552eb61fef35'
By querying with an UUID as the SK, you'll get the first item in the table with an UUID close to the one you query for. By using a random uuid each time you query, you'll get a random result back.
The particular query above actually returns nothing, so you need to retry until you get a result.
Also, I haven't done the math (who has?), but I'd imagine that periodic queries like this won't generate perfectly random distributions, especially for small data sets.
Related