How to properly cleanup in React useEffect?

I have a react function that receives data from an Apollo subscription.

For each subscription, a network connection is created. I need to destroy the network connections when the component unmounts but not when a new subscription arrives.

const { data } = useSubscription(...);
const {connections, setConnections} = useState([]);

const createNetworkConnection = (id) => {
    const connection = new CreatSomeConnection(id);

    connection.onMessage = () => doSomeOtherMutations();

    return connection;
};

useEffect(() => {
    setConnections((current) => [...current, createNetworkConnection(id)]);
}, [data])

If I add the cleanup to the hook above, then it runs every time a new subscription arrives, which is not what I want (I don't want to destroy the other network connections yet).

If I add another hook useEffect(() => () => doCleanUp(), []) then the scope of the cleanup function does not have the latest connections (or any, actually).

I think the only way to fix this is with a useRef, which I was trying to avoid?