how can I customize mui pagination bar?

You can override these components like this. There's a section in their documentation here and the PaginationItem's API here.

<Pagination
  count={10}
  renderItem={(item) => (
    <PaginationItem
      components={{
        last: (props) => <button {...props}>Last</button>,
        next: (props) => <button {...props}>Next</button>,
        first: (props) => <button {...props}>First</button>,
        previous: (props) => <button {...props}>Previous</button>
      }}
      {...item}
    />
  )}
  showFirstButton
  showLastButton
/>

I solved it with the following CSS, but it looks like a complex one.

prevButtonStyle: {
    '& nav > ul > li:nth-of-type(1)': {
      '& button': {
        width: '37px',
        '& svg': {
          visibility: 'hidden'
        },
        '& span': {
          '&:before': {
            content: `'Prev' !important`,
            color: 'rgba(0, 0, 0, 0.87)',
            verticalAlign: 'sub'
          }
        }
      }
    }
  },
  nextButtonStyle: {
    '& nav > ul > li:last-child': {
      '& button': {
        width: '37px',
        '& svg': {
          visibility: 'hidden'
        },
        '& span': {
          '&:before': {
            content: `'Next' !important`,
            color: 'rgba(0, 0, 0, 0.87)',
            verticalAlign: 'sub'
          }
        }
      }
    }
  }