The first thing that is important to remember is that there is no absolute rule regarding how you will divide your application into components, what is needed is a pragmatic view of the matter.
That said, if we look at it on the one hand, nothing prevents us from making an application in a single component, but why don't we?
1- Knowing which pieces of state and event handlers went with what parts of JSX would be very hard...
2- Code sharing/reusability would be difficult
3- All yours tests will be integration tests, which would not allow a good unitary approach to the "parts" of the app.
And as you said, Performance would probably suffer, because every state change results in a re-render of the entire application - which breaks the whole idea of single page applications. So, writing custom components allows us to solve these problems.
However, you need to remember that breaking a single component into multiple components is what's called "abstraction" and every abstraction comes with a cost(you can read this for more https://kentcdodds.com/blog/aha-programming#aha-), and you have to be aware of that cost and the benefits before you take the plunge.
obs: I wrote this answer based in these:
https://kentcdodds.com/blog/when-to-break-up-a-component-into-multiple-components,
https://sandimetz.com/blog/2016/1/20/the-wrong-abstraction