Should we use BEM(Block, Element, Modifier) in react styled component

That's on your own, you could only use styled-components to define every node that needs style, or you could mix the two: use styled-components + inline style, styled-components for component container + classname for inner nodes (with style defined in the styled container), or even mix these 3 approaches depending on the case.

In every case, if you stick with styled-component, you don't need BEM because everything is, at least, scoped to only 1 component

EDIT: response to your edit

What's not working ? style on SiteHeaderStyled (.SiteHeader) or on the nav child ?

The &__nav can't work because there is no parent (in the scss code part), you have to wrap it like this:

export const SiteHeaderStyled = styled(SiteHeader).attrs({
 className: 'SiteHeader',
 })`
   .SiteHeader {
     &__nav {
       color: red;
     }
   }
 `;

By using above is coming like screenshot in any case, if it's a project where you use styled-component everywhere, something like this is should be sufficient:

import { SiteHeader } from './site-header';

export const SiteHeaderStyled = styled(SiteHeader)`
   .nav {
     color: red;
   }
 `;