I built up a convolutional Autoencoder and trained it on a large set of synthetic streets, that do not contain any other vehicles or objects within the roads. The quality of my reconstructions (image 1.) is quite sharp, even on unseen data.
Spawning an object on the road (image 2.) slightly causes some reconstruction errors, but the overall reconstruction remains illustrative. The AE seems to ignore the spawned object, since streets do not contain any objects in the training data.
However, when I encounter a nearby object (image 3.), the reconstruction stops and I get a noisy image.
Why is this the case? Why does the reconstruction of the (same) background not work anymore?.