Why is Docker producing different images when building my Rust program on macOS versus Ubuntu? Is this a bug in Docker?

Here's my simple Rust program:

use std::time::SystemTime;

fn main() {
    let now = SystemTime::now();
    println!("{:?}", now);
}

And here's my Dockerfile:

FROM rust:1.50 as builder
WORKDIR /usr/src
RUN cargo new --bin playground
WORKDIR /usr/src/playground
COPY Cargo.toml .
COPY Cargo.lock .
RUN cargo build --release
RUN rm -rf src
COPY src src
RUN cargo build --release

FROM ubuntu:18.04
COPY --from=builder /usr/src/playground/target/release/playground /usr/local/bin/playground
CMD [ "playground" ]

My builder image contains cargo build --release twice because the first time it builds my dependencies and the second time it builds my executable. I know the simple program above doesn't have external dependencies, but this is the setup I normally use and it greatly speeds up Docker builds when my src code has changed but my Cargo.toml hasn't changed (it builds in a few seconds versus a few minutes).

However, I get completely different results when I docker build and docker run this on macOS compared to Ubuntu.

On macOS, I get the system time, e.g.:

SystemTime { tv_sec: 1614652600, tv_nsec: 426791496 }

On Ubuntu, I get the default Rust program output:

Hello, world!

It appears that the executable generated by the first cargo build --release is cached when building on Ubuntu such that the second cargo build --release doesn't do anything (while it rebuilds the executable on macOS). I'm not sure what is the "right" behavior, but as far as I know, Docker should not produce different images when building the same thing on two different operating systems? Is this a bug in Docker or do I fundamentally misunderstand how Docker works?

Interestingly, this only happens when I use something. If I change my Rust program to:

fn main() {
    println!("CHANGED!");
}

Then I get Hello, world! on both macOS and Ubuntu. Note that I can get consistent behavior on both operating systems if I fix the issue by adding RUN cargo clean before the second RUN cargo build --release. However, I still think it's problematic that I get different images without this fix.

Note that I'm using macOS Catalina Version 10.15.7 with Docker Desktop Version 3.1.0, and I'm using Ubuntu 18.04.4 with Docker Version 19.03.6.