Moving Dataloader data to GPU PyTorch

I am just getting started with PyTorch...

Currently I am following this method of getting training data to the GPU. Source: https://stanford.edu/~shervine/blog/pytorch-how-to-generate-data-parallel

# Loop over epochs
for epoch in range(max_epochs):
    # Training
    for local_batch, local_labels in training_generator:
        # Transfer to GPU
        local_batch, local_labels = local_batch.to(device), local_labels.to(device)

        # Model computations
        [...]

Question: This seems less efficient than just loading the entire data set onto the GPU. Does loading data on each batch not cause latency? Am I missing something?