I thought I understood what databases indexes are sharding/horizontal-partitioning in databases is. I was reading this article on System design, where the author builds a "search" system for Twitter. There is a database that stores tweets, and then author has an index built for search where key is a word in the search and value is tweet-ids. So far so good, we can find all the tweetIDs and hence the tweets that contain a word.
But then the author talks about sharding the index, and this got me confused! Aren't indexes present on a DB itself to facilitate finding data from the DB? The author talks about keeping the index in memory, and then sharding this index. In the image that follows, the database shows a separate index-server and a database (the index-builder server is just an helper to rebuild the index):
So I am confused if the indexes can be outside a DB, or if a separate DB can be used as an index to another DB (which seems like it would be slow!). I tried searching for "index server" but I don't see anything relevant. Please correct me if I am wrong!
This seems like something that could be achieved in the original tweets database by using a secondary index which is words to tweetIds, right? The primary index would be the tweetIds which will be unique.