A map-style dataset in Pytorch has the __getitem__() and __len__() and iterable-style datasets has __iter__() protocol. If we use map-style, we can access the data with dataset[idx] which is great, however with the iterable dataset we can't.
My question is why this distinction was necessary? What makes the data random read so expensive or even improbable?
I wrote a short post on how to use PyTorch datasets, and the difference between map-style and iterable-style dataset.
In essence, you should use map-style datasets when possible. Map-style datasets give you their size ahead of time, are easier to shuffle, and allow for easy parallel loading.
It’s a common misconception that if your data doesn’t fit in memory, you have to use iterable-style dataset. That is not true. You can implement a map-style dataset such that it retrives data as needed.
Check out the full post here.
It's quite possible that the full dataset doesn't fit in memory (could be on a disk, or only accessible over a network). A stream of information doesn't have to be retained if you're not going to access arbitrary offsets. If you're going to request data[0], then data[1], then data[2] over a network, you're sending a lot of requests which introduces latency.
Iterable-like (ResultSet) objects are typical when incrementally reading rows in the results of a database query. It's also conceivable that a dataset could inherently be a stream of information, like logging data, or transactions, or incrementally discovered pages found by a web crawler.
