What does "needed" mean in this context? Vulkan defines an abstraction of what's going on in actual hardware. Nothing is strictly "needed"; there are merely different consequences of different abstractions.
One consequence of the descriptor set abstraction is that descriptor set layouts tell both the pipeline building and the descriptor set binding code what to expect in so far as the mapping between any particular Vulkan resource and the underlying hardware resources. The pipeline layout defines a direct mapping.
The pipeline layout represents a mapping from Vulkan set/binding indices to internal resource indices. The internal hardware has different kinds of resources. So which internal resources a particular binding takes depends on the kind of binding. Sampled images take up a set of resources that is separate from SSBOs, for example. The hardware potentially has X sampled image indices and Y indices for storage buffers.
However, some hardware doesn't have different internal resource types for certain Vulkan constructs. For example, some hardware doesn't have a "uniform buffer" as a distinct construct. The implementation implements UBOs as a read-only storage buffer. But this means that any Vulkan UBO also takes up the same resource indices from an SSBO.
As such, we need a way to map from Vulkan resources to the internal resource lists, one which allows the implementation to hide details like this. This is what the pipeline layout is for: the layout defines a mappings from each descriptor in the layout to a particular resource.
If set 0 assigns a storage image to internal index 0, the system knows that if set 1 uses a UBO, that UBO must use internal index 1, since internal index 0 was already taken by set 0. Because in this hardware, UBOs and SSBOs use the same list of resources.
This can't be done without pipeline layouts or some similar system. You need something which tells the system what all of the resources are, so that it can build a mapping table.
Pipeline layouts are grouped into sets to make it easier to change a large number of resources at once. If you want to switch to a different set of 8 textures, your way might require 8 different function calls. Furthermore, because you can have different kinds of descriptors bundled into the same set, you can change 3 textures, 2 UBOs, and one SSBO all in a single bind call.
Set groupings also allow you to have pipeline layouts that are partially compatible. Two pipelines can have the same set 0 but different set 1s. This means when you switch pipelines, you can bind a different descriptor for set 1 without changing set 0's binding. This is useful for descriptors whose update frequency is different.
For example, every object in a scene may use the same perspective and camera matrices, but different world matrices and textures. You can put the former into set 0 and the latter into set 1.
