Confusion regarding number of tablets in YugabyteDB cluster

dh YB 0
Viewed 7

[Question posted by a user on YugabyteDB Community Slack]

I have one question regarding the number of tablets for a table. I am using YSQL API, my cluster is having 3 nodes with an RF of 3 and each node has 16 cores.

I haven't specified the number of shards per table using SPLIT INTO N TABLETS syntax so I guess the number of tablets will be decided by the cores a node has, based on documentation it will be 8 shards per table per node.

In this case, the total shards for a table should be 24(8 x 3)

We have RF=3 as well, so will that mean the total shards after replication will be 72? (24 x 3)

I am confused here, as I have seen only 24 shards in the tserver tablets UI where it's mentioned that 8 shards are the leaders out of 24. Seeing this it seems the 24 shards contain the replicated ones as well.

Please correct my understanding here.

I am using 2.12, latest stable.

More questions related to the same topic, if ysql_num_shards_per_tserver=8, then:

If we create a cluster with 4 nodes with RF 3 , then the total tablets/shards will be 8 x 4 = 32 (without peers)? and 32 x 3 = 96 (including peers) ? Also, suppose if we add one more node in an existing cluster with 3 nodes, then after node addition, a new 8 tablets/shards will be created for the new node? and then tablets/shards will be rebalanced ? or new tablets/shards are not created and just the rebalancing of existing ones will happen?

num_shards_per_tserver is 8

enter image description here

This is for one table, at the left it shows 24 shards, 8 are leaders and rest are followers

enter image description here

yugabytedb
1 Answers

In this case, the total shards for a table should be 24(8 x 3),

No, what is called shard in ysql_num_shards_per_tserver is the number of tablets. Each tablet/shard has 3 tablet peers (one leader and two followers). So, with ysql_num_shards_per_tserver=8 it is expected that you see 8 leaders on a server, and 16 followers which are the tablet peers of the tablets having their followers in the two other servers. And your screenshots are from one tserver endpoint (:9000/tables) The master endpoint (localhost:7000/table?id=000030af000030008000000000004200) will show you all tablets and their peers

The master shows the tablets (you have 24) and where are their leader and followers (1 leader 2 followers) so 24*4=the number of tablet peers in total. The tserver shows the tablet peers within this server

If we create a cluster with 4 nodes with RF 3 , then the total tablets/shards will be 8 x 4 = 32 (without peers)? and 32 x 3 = 96 (including peers) ?

Yes, it will be 96 total.

Also, suppose if we add one more node in an existing cluster with 3 nodes, then after node addition, a new 8 tablets/shards will be created for the new node?

No new tablets will be created.

or new tablets/shards are not created and just the rebalancing of existing ones will happen?

Yes, only rebalancing. Note that you can use auto-splitting to get new tablets created on a few threshold (> size and < number per node)

dh YB0
Related