Sorry for the lengthy question. I know this will sound like I want you guys to do the work for me. But I am new to NoSQL, and I need some help on getting my mindset straight when it comes to thinking the NoSQL way :).
Let's say, I have 3 different entities, Person, Group, and Tags. A person can be in multiple groups. A group can consist of many persons. Both Group and Person can have many tags assigned to them.
public class Person
{
public string Name { get; set; }
public List<Group> Groups { get; set; }
public List<Tag> Tags { get; set; }
}
public class Group
{
public string Name { get; set; }
public List<Person> { get; set; }
public List<Tag> Tags { get; set; }
}
public class Tag
{
public string Name { get; set; }
}
These data can be retrieved and modified by some API. Here are some of the scenarios of what would happen:
- If I modify the name of a person then retrieve a group that contains that person, the change should be reflected in the response, and vice versa.
- If I change the name of a tag, the tag's name should be updated in the corresponding persons, and groups.
- If I delete a tag, They should be deleted from persons and groups.
Questions 1: How should I model these entities for Azure Table Storage?
My idea is to model each entity as is. Each entity will have their own table. Each properties will have their own column (plus the partition key, and the row key). The collections will be serialized and stored as strings.
Let's say if I need to change the name of a group, I will need to:
- Update the Group
- Read the group to get the list of Persons affected.
- Read all the persons in the list
- Update the group in the group collections.
I know NoSQL models are suppose to be designed for fast read, but are there other ways to model the data so it will take less operations when updating? Or is this fairly common with NoSQL?
This approach will be a bigger problem when it comes to tags. If I delete a tag, I will need to:
- Delete the tag.
- Scan the whole Person table to find persons that contain the tag.
- Delete the tag from the Persons' tag collection
- Write the affect persons back to the table
- Scan the whole Groups table to find groups that contain the tag.
- Delete the tag from the Groups' tag collection.
- Write the affected groups back to the table.
Feels like it has way too many full table scans with this approach. (but hey, at least the read will be fast)
Question 2: How do I maintain data consistency across tables?
If my approach above is correct, the update operations will not be atomic. How should I handle scenarios if one of the write failed in the middle?
I did some research and found the compensation transaction pattern, and seems like some kind of message broker is required. But I am implementing this in a single Azure Function App and have no access to a service bus, or some kind of event system. Should I implement it like this?
public void DeleteTag(Tag tag)
{
try
{
// Delete Tag
}
catch(Exception)
{
// throw
}
try
{
// Read Person
// Update Person
}
catch(Exception)
{
// Revert the changes to tag
// throw
}
try
{
// Read Group
// Update Group
}
catch(Exception)
{
// Revert the changes to Tag
// Revert the changes to Person
// throw
}
}
But this also rise of the questions of what happen if the reverting operations fail?
I have a colleague who suggested that I should just put everything into the same table and same partition to avoid this problem (the data set we are dealing with start off pretty small). Doesn't that mean I will need to read all the data when I need to delete a tag?