handling unlabelled pixels in semantic segmentation/unet model

I have a label data, consisting of 4 values [0,1,2,3]. It has 3 defined labels [1,2,3], where 0 refers to unlabelled pixel. The goal is predict each 0s (unlabelled pixels) into one of three classes [1,2,3].

Following is unet model run for an example data.

data = np.random.randint(low=1,high=29, size=(300, 160, 160, 10)) # (samples, width, height, channels)
labels = np.random.randint(low=0,high=3, size=(300, 160, 160)) # (samples, width, height, channels)

input_dim = (160,160,10) #(width, height, channels)

n_class = len(np.unique(labels))  

model = unet_model()

model.compile(optimizer = tf.keras.optimizers.Adam(0.0001),
              loss=tf.keras.losses.SparseCategoricalCrossentropy(),
              metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
              
           
model.fit(data,                 
          labels,                 
          epochs=10,
          verbose=2)

However, it is predicting 0s values as well.

How can I handle unlabelled pixels [0s] so that the model predicts only one of the [1,2,3]?