How can I specify which Google Wallet pass is for a specific user?

I'm currently working on building some loyalty cards via Google wallet api. Creating and distributing passes is not a problem, but I'm not so sure on how to specify one card to one user.

I understand you are able to set the MultipleDevicesAndHoldersAllowedStatus property to one user. I also understand that you are supposed to set a unique userId which can be the user's email_address, but this does not necessary mean that only this user with this email_address is able to get the pass.

How can I make sure that a user with user1@mail.com is only able to install a specific pass?

I created a sample pass object with my personal email address as the userId; however, I was able to use the save link created by

const claims = {
    iss: credentials.client_email,
    aud: "google",
    origins: ["www.example.com"],
    typ: "savetowallet",
    payload: {
      loyaltyObjects: [
        {
          id: objectId,
        },
      ],
    },
  };

  const token = jwt.sign(claims, credentials.private_key, {
    algorithm: "RS256",
  });
  const saveUrl = `https://pay.google.com/gp/v/save/${token}`;

to save it to a device that was not logged in w/ my personal email.

Is there a way for the pass (or maybe the save url?) to check again for the proper gmail address before letting the user download the pass to their wallet?

(I had to use android-pay since i don't have enough reputation to tag it with google-wallet-api)