DynamoDB Update Item condition on PK only

Question

I have some DynamoDB data that looks like:

PK:              SK:
subscriptionId | changeId       | status     | scheduled_for |
--------------------------------------------------------------
A              | 19695a80-e3... | DONE       | 2009-01-01    |
A              | 3327f283-a1... | DONE       | 2012-11-13    |
X              | aebb5fe4-78... | DONE       | 2019-06-24    |
X              | 8982f726-69... | PENDING    | 2022-01-01    |

If an new change request is made I currently cancel the old change and pend a new one as follows:

PK:              SK:
subscriptionId | changeId       | status     | scheduled_for |
--------------------------------------------------------------
A              | 19695a80-e3... | DONE       | 2009-01-01    |
A              | 3327f283-a1... | DONE       | 2012-11-13    |
X              | aebb5fe4-78... | DONE       | 2019-06-24    |
X              | 8982f726-69... | CANCELLED  | 2022-01-01    |
X              | 1f3380aa-f2... | PENDING    | 2022-02-01    |

There is a (small) possibility that two requests for the same subscription come in at the same time, so I figured the best way to guarantee only one 'PENDING' record is to use a ConditionCheck within a transaction and avoid any round trip race conditions - e.g.

// Only insert if there is no PENDING item for the subscriptionId
writeItems := []*dynamodb.TransactWriteItem{
    {
        ConditionCheck: &dynamodb.ConditionCheck{
            TableName:           aws.String("subscription_changes"),
            ConditionExpression: aws.String("attribute_not_exists(changeId)"),
            Key: map[string]*dynamodb.AttributeValue{
                "status":         {S: aws.String("PENDING")},
                "subscriptionId": {S: aws.String(subscriptionId)},
            },
        },
    },
    {
        Put: &dynamodb.Put{
            TableName: aws.String("subscription_changes"),
            Item:      attributesMap,
        },
    },
}

...however, DynamoDB wants me to include the Sort Key (changeId) in my condition, which I can't know without querying for the data (and hence opening up the possibility of a race condition where both threads 'win' and we have two PENDING items in the table).

Is there a way to do this?

In SQL this would be something like:

INSERT subscription_items 
      (subscriptionId, changeId, status, ...)

SELECT 'X', '1f3380aa-f2...', 'PENDING', ...

WHERE NOT EXISTS (SELECT 1 
                  FROM   subscription_items
                  WHERE subscriptionId = ‘X’ AND status = 'PENDING')

user_26051 · Answer

Indeed a limitation of consistency in Dynamo is that you need to tell Dynamo exactly which items (PK/SK) it should operate on. As you mention, this is relatively trivial to do in SQL, but as always with Dynamo, the flip-side is that it's also relatively impossible to make the SQL way of doing it web scale... :)

One way to solve this is to add an extra "tracking" item in your tx, with the PK X and the SK current_pending or something like that. In your TX, add this item, and verify that it does not already exist. The two tracking items themselves are independent of one another, but they cannot both succeed in being created, since just one of them will not already have the current_pending SK in place. Then, when an item is no longer pending, also delete the current_pending item in the same TX to release the "lock".

A lot of transaction and consistency issues in Dynamo are solved by adding more items!

DynamoDB Update Item condition on PK only

1 Answers