Loading in your own Image data with tensorflow and tdfs.ImageFolder

user_13607895 0
Viewed 1835

I want to train a GAN and generate images of pokemon. I scraped around 10000 images from the internet which are locally saved. My folder is structured like so:

all_data:
    - train:
        -bulbasaur.png
        -45.png
        -....png
    - test:
        -bulbasaur.png
        -45.png
        -....png
     - validation:
        -bulbasaur.png
        -45.png
        -....png

I tried to load it via:

builder = tfds.ImageFolder(os.path.join(os.getcwd(), "all_data"))
print(builder.info)  # num examples, labels... are automatically calculated
ds = builder.as_dataset(split='train', shuffle_files=True)
tfds.show_examples(ds, builder.info)

but I get the error of: ValueError: Unrecognized split test. Subsplit API not yet supported for ImageFolder. Split name should be one of []. Is there a Problem with how I structured the dataset? As you can tell from the code snippet the different files all have completely varying names (either their English name or their Pokedex number) is that a problem? Since I do not want to classify anything I thought the labeling is not really important.

Also if it helps the splits from the output I get for the builder Info is empty.

tfds.core.DatasetInfo(
    ....
    supervised_keys=('image', 'label'),
    splits={
    },...
)

Thanks a lot in advance!

generative-adversarial-network tensorflow-datasets tensorflow2.0 tensorflow python
1 Answers

Your folder structure should be like;

/content/image_dir/
  train/  
    cat/  
      cat_1.png
      cat_2.png
      cat_3.png
    dog/
      dog_1.png
      dog_2.png
      dog_3.png
  test/
      cat.png
      dog.png

Below code works with this structured directory

import tensorflow as tf
import tensorflow_datasets as tfds

builder = tfds.ImageFolder('/content/image_dir/')
print(builder.info)  # num examples, labels... are automatically calculated
ds = builder.as_dataset(split='train', shuffle_files=True)
tfds.show_examples(ds, builder.info)
user_142902440
Related