EDIT To make my question more general, how do you transfer weights if the structure of the network is different. Can I for example, transfer part subset of the network?
I'm trying to duplicate the paper SimCLR which do contrastive learning and generate weight. Then, I would like to feed those pretrained to Unet. However, as expected I got the below error when calling "model.load_weights("ckpt")"
WARNING:tensorflow:Inconsistent references when loading the checkpoint into this object graph. Either the Trackable object references in the Python program have changed in an incompatible way, or the checkpoint was generated in an incompatible program.
Two checkpoint references resolved to different objects (<keras.layers.convolutional.Conv2D object at 0x7f0a45a6fb10> and <keras.layers.pooling.MaxPooling2D object at 0x7f0a4402fc50>).
WARNING:tensorflow:Inconsistent references when loading the checkpoint into this object graph. Either the Trackable object references in the Python program have changed in an incompatible way, or the checkpoint was generated in an incompatible program.
Two checkpoint references resolved to different objects (<keras.layers.convolutional.Conv2D object at 0x7f0a44022110> and <keras.layers.convolutional.Conv2D object at 0x7f0a44166510>).
WARNING:tensorflow:Inconsistent references when loading the checkpoint into this object graph. Either the Trackable object references in the Python program have changed in an incompatible way, or the checkpoint was generated in an incompatible program.
Two checkpoint references resolved to different objects (<keras.layers.convolutional.Conv2D object at 0x7f0a44166510> and <keras.layers.convolutional.Conv2D object at 0x7f0a45a6f510>).
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-41-f45628c539be> in <module>()
2 model = UNet()
3 #------
----> 4 model.load_weights("ckpt")
This is expected since the SimCLR uses ResNet which have different architecture then Unet. But I wonder how do I utilize the weight from ResNet.
One idea (See below) is to consider the whole "down sampling" as ResNet and initialize it with the pretrained weight and consider the whole "up sampling" and another ResNet and initialize it randomly. But I'm not sure how to do that.
