Tensorflow Keras model has loss of nan while fitting

I have a tensorflow keras model which I want to train with obtained data. However, when training, there is a loss of nan after the first epoch. The Input has a range [-1|1].

This is my model:

x = Input(shape=(8, 8, 7), name='input')

value_head = Conv2D(
        filters=1,
        kernel_size=(1, 1),
        padding='same',
        use_bias=False,
        activation='linear',
        kernel_regularizer=regularizers.l2(0.0001),
    )(x)

value_head = BatchNormalization()(value_head)

value_head = LeakyReLU()(value_head)

value_head = Flatten()(value_head)

value_head = Dense(
        20,
        use_bias=False,
        activation='linear',
        kernel_regularizer=regularizers.l2(0.0001),
    )(x)

value_head = LeakyReLU()(value_head)

value_head  = Dense(
        1,
        use_bias=False,
        activation='tanh',
        kernel_regularizer=regularizers.l2(0.0001),
        name='value_head',
    )(value_head)

policy_head = Conv2D(
        filters=2,
        kernel_size=(1, 1),
        padding='same',
        use_bias=False,
        activation='linear',
        kernel_regularizer=regularizers.l2(0.0001),
    )(x)

policy_head  = BatchNormalization()(policy_head)

policy_head  = LeakyReLU()(policy_head)

policy_head  = Flatten()(policy_head)

policy_head = Dense(
        self.mOutputDim,
        use_bias=False,
        activation='linear',
        kernel_regularizer=regularizers.l2(0.0001),
        name='policy_head',
    )(x)

model = Model(inputs=[nn_input], outputs=[value_head, policy_head])

model.compile(loss={'value_head': 'mean_squared_error', 'policy_head': softmax_cross_entropy_with_logits},
                  optimizer=SGD(lr=0.00001, momentum=0.9),
                  loss_weights={'value_head': 0.5, 'policy_head': 0.5}
                  )

Training looks as following:

training_states = np.array(
            [self.mAI.convert_training_state_node_to_model_input(current_node)
             .reshape((8, 8, 7))
             for current_node in training_data.mTrainingStateNodes])

training_targets = {
            'value_head': np.array([np.array([training_data.mWinner])]),
            'policy_head': np.array([current_node.convert_edges_visit_count_to_policy_map()
                                   for current_node in training_data.mTrainingStateNodes])}

fit = model.fit(training_states,
                training_targets,
                epochs=3,
                verbose=1,
                validation_split=0,
                batch_size=64)

I would be very grateful for a solution and constructive criticism to my question.