Avoid re-rendering of open layers map when context changes

I am using an Open Layers map inside React. I am attaching it to the DOM using setTarget like so:

const mapObject = useMap(maplayers);

useEffect(() => {
    mapObject.setTarget(mapRef.current);
    return () => mapObject && mapObject.setTarget(undefined);
},[mapObject]); 

Inside the useMap.js hook I am writing changes to the center and zoom of the map back to the context:

const {
    zoom,
    setZoom,
    center,
    setCenter,
} = useContext(MapContext);

let mapObject = null;

var projection = new olProj.Projection({
    code: 'EPSG:25832',
    units: 'm',
});

let options = {
    target: 'map',
    controls: olControl.defaults({
        rotate: false,
        zoom: false,
        attributionOptions: {
            collapsible: false,
        },
    }),
    layers: maplayers,
    view: new ol.View({
        projection: projection,
        center: center,
        zoom: zoom
    }),
};
mapObject = new ol.Map(options);

mapObject.on('moveend', (e) => {
    console.log('moveend');
    setCenter(() => e.target.getView().getCenter());
    setZoom(() => e.target.getView().getZoom());
});

The problem now is that the whole map gets rerendered every time the center and zoom change. How can I avoid this? The only idea I have is to use useEffect with a dependency array that misses center and zoom but I know this is bad practise and should not be done.