Issue with centering and scaling small emojis?

I'm trying to center an emoji and scale it when it's hovered. I'm running into an issue where, on certain displays, the emoji is displayed off-center. Then when scaling up, there is a jitter in the emoji's position.

The behavior is sometimes different depending on browser zoom and font size. I did try the solution from here (setting a larger font size and then scaling it down) but it didn't seem to have an effect.

Here's a codepen and videos demonstrating the issue:

https://codepen.io/josh-conley/pen/LYmPjbr

<div class="container">
  <div class="emoji-grid">
    <button class="emoji"><span></span></button>
    <button class="emoji"><span>❤️</span></button>
    <button class="emoji"><span></span></button>
    <button class="emoji"><span></span></button>
  </div>
</div>


.container {
  display: flex;
  padding: 20px;
}

.emoji-grid {
  display: grid;
  grid-template-columns: 1fr 1fr 1fr 1fr;
  gap: 6px;
}

.emoji {
  border: none;
  display: inline-flex;
  background-color: grey;
  justify-content: center;
  align-items: center;
  border-radius: 60px;
  width: 28px;
  height: 28px;
  transition: transform 150ms ease;
  font-size: 0;
  font-family: Segoe UI Emoji,Segoe UI Symbol,Segoe UI,Apple Color Emoji,Twemoji Mozilla,Noto Color Emoji,Android Emoji;
  
  > span {
    font-size: 36px;
    width:38px;
    line-height: 1;
    transform: translateY(0.05em) scale(0.5);
    color: white;
  }
  
  &:hover {
    cursor: pointer;
    transform: scale(1.2);
  }
}

The codepen looks fine on another mac with the same specs as mine, so it's possible that you might not be able to see it there.

Monitor: https://youtu.be/ZKgErMU6pTg

• on my external monitor, resolution 2048 x 1152

Mac: https://youtu.be/_ERKh88uCfI

• on the Mac screen, resolution 1312 x 848

I'm on an M1 v12.4 MacBook. I see this on Chrome, Firefox, and Safari.