Can't change outline border color of MUI textfield

I'm using MUI together with styled-components for my react project but I can't change the outline border color of a textfield... I've read the documentation on MUI, there it says it is possible to overwrite the class names of the component, but that does not seem to work..

My styled-component for the textfield:

export const LoginField = styled(TextField)`
  margin-top: 10px !important;
  width: 100% !important;
  border-color: ${(props) => props.theme.text.primary}!important;

  & .MuiInputLabel-root {
    color: ${(props) => props.theme.text.primary};
  }

  & .MuiOutlinedInput-root {
    border-color: ${(props) => props.theme.text.primary};
    
  }

  & .MuiOutlinedInput-root {
    & fieldset {
      border-color: red,
    },

  & .MuiSvgIcon-root {
    color: ${(props) => props.theme.text.primary};
  }
  & .MuiFormControl-root {
    color: ${(props) => props.theme.text.primary};
  }

  // Mui bug where it changes the background of the input on autofill
  input:-webkit-autofill,
  input:-webkit-autofill:hover,
  input:-webkit-autofill:focus,
  input:-webkit-autofill:active {
    transition: background-color 5000s ease-in-out 0s,
      color 5000s ease-in-out 0s;
    transition-delay: background-color 5000s, color 5000s;
  }
`;

The funny thing is it does work on all the other classes I overwrite but .MuiOutlinedInput-root is the only one where it does not work. I've checked several posts where it said it is the right class.

The textfield in my react component:

<LoginField
  variant="outlined"
  label="E-mail"
  type="email"
  value={email}
  onChange={handleEmail}
  InputProps={{
    startAdornment: (
      <InputAdornment position="start">
        <Person />
      </InputAdornment>
    )
  }}
/>

I'm using a custom theme for light and dark theme, thats is why I use the theme props to set the color dynamically.

I've also tried using the solution below:

export const LoginField = styled(TextField)({
  "& .MuiOutlinedInput-root": {
    "& fieldset": {
      borderColor: "white",
    }
  },
});

It does change the border color to white, but I cant seem to use the custom theme props. Plus I want to keep consistency for all the styled-components, where the solution above does not conform.