Large training time for first epoch

I have been using unet for image segmentation but I face a unusual problem as my first training epochs takes about 2 hours of training time but subsequent epoch takes about 10 minutes. My problem about multi-class segmentation of a imbalance dataset and I have been using unet architecture from 2015 paper on biomedical image segmentation.