Tensorflow: How to handle preprocessing.Normalization returning NaN

I am building a keras model and trying to use the tf.keras.layers.experimental.preprocessing.Normalization from https://www.tensorflow.org/api_docs/python/tf/keras/layers/experimental/preprocessing/Normalization to normalize the input training data as such:

normalize = preprocessing.Normalization()
normalize.adapt(trainX)
model = Sequential([
    normalize,
    Dense(dim + 1, input_dim=dim, activation="relu"),
    Dense(dim / 2, activation="relu"),
])

The goal is to save the normalization within the saved model. I am running into the issue where the normalize(trainX) is normalizing some of the inputs to nan due to most likely a division of zero in the variance. E.g. a row of the output: [ 0.00000000e+00, nan, 9.40457404e-01, 9.40672755e-01, 9.40672755e-01, 9.40672755e-01, 9.40672755e-01, nan, nan, nan, nan, nan]

Is there a way to handle division of zero in preprocessing.Normalization() or is this there another way of normalization I should consider that I can save within the model?