Extracting Features for Image Caption Generator using Cnn & Lstm?

Question

1.This section i got the output as Created a "descriptions.txt" file

# Set these path according to project folder in you system
dataset_text = 'C:\Users\Srikanth Bhattu\Project\Flickr8k_text\Flickr8k.token.txt'
dataset_images = 'C:\Users\Srikanth Bhattu\Project\Flickr8k_Dataset\Flicker8k_Dataset'
#we prepare our text data
filename = dataset_text + "/" + 'C:\Users\Srikanth Bhattu\Project\Flickr8k_text\Flickr8k.token.txt'
#loading the file that contains all data
#mapping them into descriptions dictionary img to 5 captions
descriptions = all_img_captions()
print("Length of descriptions =" ,len(descriptions))
#cleaning the descriptions
clean_descriptions = cleaning_text(descriptions)
#building vocabulary 
vocabulary = text_vocabulary()
print("Length of vocabulary = ", len(vocabulary))
#saving each description to file 
save_descriptions()

2.Help me to load an image data set and generate as below given output

def extract_features(directory):
        model = Xception( include_top=False, pooling='avg' )
        features = {}
        for img in tqdm(os.listdir(directory)):
            filename = directory + "/" + img
            image = Image.open(filename)
            image = image.resize((299,299))
            image = np.expand_dims(image, axis=0)
            #image = preprocess_input(image)
            image = image/127.5
            image = image - 1.0
            feature = model.predict(image)
            features[img] = feature
        return features
#2048 feature vector
features = extract_features(dataset_images)
dump(features, open("features.p","wb"))

Error:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
 in 
      1 #2048 feature vector
----> 2 features = extract_features(dataset_images)
      3 dump(features, open('C:\Users\Srikanth Bhattu\Project\features.p','wb'))

 in extract_features(directory)
      1 def extract_features(directory):
----> 2         model = Xception( include_top=False, pooling='avg' )
      3         features = {}
      4         for img in tqdm(os.listdir(directory)):
      5             filename = directory + "/" + img

NameError: name 'Xception' is not defined.

I Need this Output Image: Click here to Open output Image

I am working with "IMAGE CAPTION GENERATOR using CNN & LSTM". I have all source codes and data set with me but iam not able to include my data set and paths in my code. If anyone Interested to Help me to complete my Project Please Comment I will pay some Charges to them. Thank You...!!!

user_726872 · Answer

You have no definition of the model called "Xception" in the code snippets you have shared. So it won't work unless you define the model or import it. I'm guessing you're following this tutorial https://data-flair.training/blogs/python-based-project-image-caption-generator-cnn/

Read it properly and you'll see that they have imported the model in the beginning of the article, do it and it should be fine. Follow the article sequentially and you'll not face issues.

import string
import numpy as np
from PIL import Image
import os
from pickle import dump, load
import numpy as np
from keras.applications.xception import Xception, preprocess_input
from keras.preprocessing.image import load_img, img_to_array
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.utils import to_categorical
from keras.layers.merge import add
from keras.models import Model, load_model
from keras.layers import Input, Dense, LSTM, Embedding, Dropout
# small library for seeing the progress of loops.
from tqdm import tqdm_notebook as tqdm
tqdm().pandas()

These have to be executed before you run the snippet. It imports the Xception model and the other libraries that you are using.

Extracting Features for Image Caption Generator using Cnn & Lstm?

1 Answers