While going through the documentation provided by Microsoft, it doesn't mention specifically how many self hosted agents can be in a single agent pool and how many agent pools can be in a single organization. So basically my questions are,
Is there a limit or it's not mentioned because there's no such limit?
For these two issues, yes, Microsoft currently does not provide such documents regarding the limitation of the number of agent pools and self-hosted agents.
In the documentation, there is a description of the "Azure virtual machine scale set" type agent pool:
You specify a virtual machine scale set, a number of agents to keep on standby, a maximum number of virtual machines in the scale set, and Azure Pipelines manages the scaling of your agents for you.
I tested this type of agent pool, you can see from the figure below that we can specify a very large number. Taking this as a reference, under normal circumstances, the number limit of self-hosted agents can definitely meet our needs.
And for count of build agents, there's no count limit for microsoft-hosted agents. If you're meaning self-hosted agents, you can own many agents in your agent pool.(The limit of count is something we won't meet in normal situation.) We can also install more than one agents in same local machine, see Can I install multiple self-hosted agents on the same machine?.
