Fixing initial weights when training Keras model in R

I want to fix the weights of the model I'm training to get reproducible results for the report. The problem is that on different runs I get similar but slightly different results and training sometimes takes 600 epochs, sometimes takes 3500 using callback_early_stopping monitoring validation MSE and with min_delta of 0.00003.

Overall, I'm happy enough with results of all runs, but just need to find if there's a way to get reproducible results by fixing weights.

I tried setting seed at various parts of the process - before creating model, before compiling it and before training but nothing seems to work. Any way to do it?

BATCH <- nrow(x_train)
SHAPE <- ncol(x_train)

# Create a neural network model
set.seed(42)
model <- keras_model_sequential()
model %>% 
    layer_dense(units = 12, activation = "relu", input_shape = c(SHAPE)) %>%
    layer_dense(units = 24, activation = "relu") %>%
    layer_dense(units = 1, activation = "linear")

#print model summary
print(summary(model))

# initialise early stopping callback and optimiser
early_stoping <- callback_early_stopping(
    monitor = "val_loss",
    min_delta = 0.00003,
    patience = 50,
    restore_best_weights = TRUE
)
optim <- optimizer_adam(learning_rate = 0.00005)
set.seed(42)
model %>% compile(
    optimizer = optim,
    loss = "mse",
    metrics = c("mse", "mae")
)

# fit model
set.seed(42)
val_data <- list(x_val = x_val, y_val = y_val)
hist <- model %>% fit(
    x = x_train,
    y = y_train,
    batch_size = BATCH,
    epochs = 6000,
    validation_data = val_data,
    shuffle = FALSE,
    callbacks = early_stoping
)