tensorflow.load vs download URL

Viewed 598

I am a beginner with TensorFlow 2, I am using version 2.3.1 of tensorflow.

I want to build an Image Classifier based on Inception v3. Before I can use the data in the Inception network, I have to prepare the data at first. For this task I will use the 'oxford_flower102' dataset. I found two ways to get datasets, but I don't know which way should be used in which situation.

  1. by using tfds.load
import tensorflow as tf
dataset, dataset_info = tfds.load('oxford_flowers102, with_info=True, as_supervised=True)

  1. by downloading the dataset with URL
from six.moves import urllib
import os
import tarfile

FILE_NAME='102flowers.tgz'
FLOWERS_DIR = os.getcwd() + '/jpg'

def download_images():
   downloadedPath = os.getcwd()+ '/'+FILE_NAME
   if not os.path.exists(downloadedPath):
       download_ulr='http://www.robots.ox.ac.uk/~vgg/data/flowers/102/102flowers.tgz'
       urllib.request.urlretrieve(download_ulr,FILE_NAME)
   else:
       print('dataset already downloaded in : ', downloadedPath)

def extract_tgz_file():
   if not os.path.exists(FLOWERS_DIR):
       print('extracting files ...')
       tar=tarfile.open(FILE_NAME, "r:gz")
       tar.extractall()
       tar.close()
       print('files now extracted in : ', FLOWERS_DIR)
   else:
       print('extraced file already exists in : ', FLOWERS_DIR)

def make_dataset_ready():
   download_images()
   extract_tgz_file()

The first way is much shorter and the split into training and test datasets is done easily. The second way is much longer and the split into training and test dataset have to be done manually. Now let me come to my questions:

  1. what are the benefits of the second way, by downloading it through URL and split later the dataset manually into training and test datasets?
  2. My goal will be later to feed the datasets into inception v3, are there any disadvantages for this task by using tfds.load() function ?

Thank you for the support!

1 Answers
  • tfds.load is a utility method of tensorflow using which you download a predefined set of datasets. The advantage of using this method is that it returns the data in tf.data.Dataset which can be directly used for training the model. It also returns a second value of type tfds.core.DatasetInfo which contains the information about the dataset.
  • urllib.request.urlretrieve is the python module to download the data from a url. So you will have to download a dataset hosted on a url, understand its format and convert it into the format so that it can be used for training a model or doing inference.
  • If your intension is to train an inception model in tensorflow then its meaning full use tfds.load to download the data and use the tensorflow dataset to train it.
  • However, if your dataset is not available as part of tfds.load named datasets, then you will have to download the data and convert it into the required format and one way of doing it is using urllib
Related