I visited the documentation of material UI, here's what I found while inspecting these:

Container

When you use <Container> in your code, following styles will be applied to your <Container> component.

As you can see in above screenshot, there are two important properties present,

• display: block
• max-width: 600px (In the media queries, so it varies depending on the viewport width)

Grid

When you use in your code with attribute container, following styles will be applied,

As you can see above, it utilizes the flexbox layout for holding the columns, which can be useful for holding the flex-items (i.e. columns). Also it has its max-width set depending on the type of spacing you are specifying.

After inspecting these differences, in my opinion, <Container> are used for outer wrapper which holds the all the components in it, whereas the <Grid container> can be used to create column layout (12-columns), which is not really possible in case of <Container>.