I am trying to train a DenseNet121 model on chest X-ray images using tensorflow.keras, and using ImageDataGenerator for augmentation. I have directories of files containing symlinks to the images that I believe is set up in the correct format for ImageDataGenerator:
Train
Normal
Abnormal
Val
Normal
Abnormal
However, when I call model.fit(), it throws FileNotFoundError: [Errno 2] No such file or directory: '.\\Train\\Normal\\00017275_014.png' which is a symlink file. .flow_from_directory(follow_links = True) did not solve the problem. Also, calling os.islink() with that path returns True.
In addition: calling imagedatagenerator returns:
Found 84090 images belonging to 2 classes. Found 28030 images belonging to 2 classes.
Any suggestions? Code below:
from tensorflow.keras.applications.densenet import preprocess_input
from tensorflow.keras import Model,layers
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.optimizers import Adam, SGD
from tensorflow.keras.metrics import binary_accuracy
from tensorflow.keras.losses import binary_crossentropy
batch_size = 64
train_datagen = ImageDataGenerator(
preprocessing_function = preprocess_input,
brightness_range = [0.75, 1.25],
horizontal_flip=True,
)
train_generator = train_datagen.flow_from_directory(
directory = '.\\Train',
color_mode = 'rgb',
classes = ['Normal', 'Abnormal'],
class_mode = 'binary',
batch_size = batch_size,
target_size = (224,224),
follow_links=True,
)
val_datagen = ImageDataGenerator(
preprocessing_function = preprocess_input,
)
val_generator = val_datagen.flow_from_directory(
directory = '.\\Val',
color_mode = 'rgb',
class_mode = 'binary',
classes = ['Normal', 'Abnormal'],
batch_size = batch_size,
target_size = (224,224),
follow_links = True,
)
from tensorflow.keras.callbacks import ModelCheckpoint, EarlyStopping
model_name = "Imagenet DenseNet121 on NIH full dataset 375 locked brightness flip.h5"
callback_checkpoint = [
EarlyStopping(monitor = 'val_loss', patience = 10, verbose = 1),
ModelCheckpoint(model_name,
verbose = 1,
monitor = 'val_loss',
save_best_only = True,
)
]
model.compile(
optimizer = Adam(),
#optimizer = SGD(learning_rate = 0.001, momentum = 0.9, decay = 0.0001),
loss = 'binary_crossentropy',
metrics = ['binary_accuracy'],
)
history = model.fit(
train_generator,
steps_per_epoch=1250,
epochs=50,
validation_data=val_generator,
validation_steps=437,
callbacks = [callback_checkpoint],
)
`os.path.islink((os.path.join(os.getcwd(), "Train", "Normal", "00017275_014.png")))
True`