I am using a cGAN (ie. pix2pix) (tensorflow v 2.4, python 3.7) to predict images at different positions. I have an input image of one position, a simulated ground truth image of a target (displaced) position which the network should be learning, and am aiming to output a predicted position which looks like the ground truth image. I have not changed much in pix2pix for my project except the number of epochs. My issue is that the network is outputting predicted images which look much more like my input images rather than my ground truth, therefore indicating that it is not learning much or learning that there is no difference between input and ground truth (which is not the case). In tensorboard I can see that the discriminator is winning the dic/gen battle.
Which hyperparameters can I tune in order to help my network learn to better predict the ground truth images?
Current hyperparameters:
LRARRAY = iter([2e-4]) #learning rate - iterable for manual hyperparameter tuning #defauls = 2e-4
OUTPUT_CHANNELS = 8 #number of pTx channels
SHUFFLE_BUFFER = 50 #tf.data.Dataset parameters. Small shuffle buffer is ok since pseudo-shuffled when wr
BATCH_SIZE = 1 #OG default = 1
LAMBDA = 100 #hyperparam acting on L1 generator loss. Default = 100
use_amsgrad = 0 #use amsgrad on Adam optimizer (seems to cause memory issues)
leaky_relu_in_gen = 1
I have tried changing the number of epochs, learning rate, and switching from the original tf.random_normal initializer to a he_random initializer.