How to assign custom define unique number to entitiy in GAE?

user_5614956 0
Viewed 66

How to assign unique number to entity. The unique numbers range will be defined by user e.g. 1000000001 so the next number will be 1000000002, and so on.

Currently I'm maintaining a separate entity for managing the number range for my other entities.

What I'm doing right now while saving any entity. I read my number range entity and pick last number updated there and increment that number by 1 and assigned that number to my entity which to be saved. After that I'm saving my number range entity with updated number.

Problem: As it is a web application it is accessed by multiple users simultaneously. If multiple user save same entity at same time then same unique number is assigned to multiple records. How to overcome this issue?

gwt google-app-engine java
1 Answers

If by entity number you're referring to the entity's key name/identifier, the only way to prevent assigning the same number to different records is to reserve them beforehand. From Assigning identifiers:

Note: Instead of using key name strings or generating numeric IDs automatically, advanced applications may sometimes wish to assign their own numeric IDs manually to the entities they create. Be aware, however, that there is nothing to prevent Cloud Datastore from assigning one of your manual numeric IDs to another entity. The only way to avoid such conflicts is to have your application obtain a block of IDs with the methods DatastoreService.allocateIds() or AsyncDatastoreService.allocateIds(). Cloud Datastore's automatic ID generator will keep track of IDs that have been allocated with these methods and will avoid reusing them for another entity, so you can safely use such IDs without conflict.

Regardless of using that using number as a key name/identifier or just as a property value, to prevent multiple simultaneous requests from assigning the same number you need to perform inside a single transaction all these ops:

  • reading the last assigned entity in which you stored the last assigned value
  • incrementing the value to obtain the current number
  • saving the current number in the last assigned entity
  • allocating the new identifier as mentioned above, if applicable
  • creating the new entity with the current number

Your code should be prepared to retry this entire transaction as it will fail when multiple requests attempt to do it at the same time - only one (max) will succeed in updating the last assigned entity.

It's worth nothing that using such incrementing values for key names/identifiers or just properties can cause the "hot tablets" datastore scalability issue, affecting performance of the read/write/indexing operations. From High read/write rates to a narrow key range:

If you are using Cloud Datastore, you can get slow writes due to a hot tablet if you have a sudden increase in the write rate to a small range of keys that exceeds the capacity of a single tablet server. Bigtable will eventually split the key space to support high load.

...

By default, Cloud Datastore allocates keys using a scattered algorithm. Thus you will not normally encounter hotspotting on Cloud Datastore writes if you create new entities at a high write rate using the default ID allocation policy. There are some corner cases where you can hit this problem:

  • If you create new entities at a very high rate and you are allocating your own IDs which are monotonically increasing.

...

  • You will also see this problem if you create new entities at a high rate with a monotonically increasing indexed property like a timestamp, because these properties are the keys for rows in the index tables in Bigtable.
user_44950810
Related