I've one doubt regarding AI and ML model...While training a textual data like spam dataset or any other textual data, does the length of each row's text also matters.? Because while I'm training a model with various algorithms the accuracy for that specific dataset is getting good. But while trying with my own sample text it showing error...So could anyone please explain me this term?
from re import T
import tensorflow as tf
import tensorflow
import warnings
warnings.filterwarnings('ignore')
# import libtraries for reading,exploring and plotting the data
import numpy as np
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator
# %matplotlib inline
# Library for train_y_test_split
from sklearn.model_selection import train_test_split
# Deeplearning models for test-preprocessing
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
# Modeling
from tensorflow.keras.callbacks import EarlyStopping
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding,GlobalAveragePooling1D,Dense,Dropout,LSTM,Bidirectional
import warnings
warnings.filterwarnings('ignore')
import pickle
df=pd.read_csv("spams_DBs/emails.csv")
df
df.sample(7)
df2 = df.replace('Subject:','', regex=True)
df3=df2.replace('re :','', regex=True)
df3
df3.sample(7)
df3["text"][155]
df3["spam"][155]
file=df3
file.shape
len(file)
file.columns
file.rename(columns={"spam":"target"},inplace=True)
file.columns
file.duplicated().sum()
file.drop_duplicates
file.shape
file["target"]=file["target"].replace(1,"spam")
file
file["target"]=file["target"].replace(0,"ham")
file
file.isna().sum()
file.drop_duplicates(keep=False, inplace=True)
file
file.groupby('target').describe().T
spam_msgs=file[file.target=='spam']
ham_msgs=file[file.target=='ham']
# Get all the ham and spam emails
spam_msgs.head(5)
file.shape
len(spam_msgs)
ham_msgs.head(5)
len(ham_msgs)
# creating numpy list to visualize using wordcloud
spam_msgs_text="".join(spam_msgs.text.to_numpy().tolist())
ham_msgs_text="".join(ham_msgs.text.to_numpy().tolist())
spam_msgs_text
ham_msgs_text
print(len(spam_msgs_text))
print(len(ham_msgs_text))
#wordcloud of spam_msgs
spam_msg_cloud = WordCloud(width =520, height =260, stopwords=STOPWORDS,max_font_size=50, background_color ="black", colormap='Blues').generate(spam_msgs_text)
plt.figure(figsize=(16,10))
plt.imshow(spam_msg_cloud, interpolation='bilinear')
plt.axis('off') # turn off axis
plt.show()
# wordcloud of ham messages
ham_msg_cloud = WordCloud(width =520, height =260, stopwords=STOPWORDS,max_font_size=50, background_color ="black", colormap='Blues').generate(ham_msgs_text)
plt.figure(figsize=(16,10))
plt.imshow(ham_msg_cloud, interpolation='bilinear')
plt.axis('off') # turn off axis
plt.show()
# we can observe imbalance data here
plt.figure(figsize=(8,6))
sns.countplot(file.target)
# Percentage of spam messages
(len(spam_msgs)/len(ham_msgs))*100
# one way to fix it is to downsample the ham msg
ham_msg_df = ham_msgs.sample(n = len(spam_msgs), random_state = 44)
spam_msg_df = spam_msgs
print(ham_msg_df.shape, spam_msg_df.shape)
# create a dataframe with ham and spam msgs
balanced_df=ham_msg_df.append(spam_msg_df).reset_index(drop=True)
# now it's the balanced dataset
sns.countplot(balanced_df.target)
plt.title('Distribution of ham and spam email messages (after downsampling)')
plt.xlabel('Message types')
plt.show()
# Get length column for each text
balanced_df['text_length']=balanced_df['text'].apply(len)
#Calculate average length by label types
labels = balanced_df.groupby('target').mean()
labels
# Map ham label as 0 and spam as 1
balanced_df['msg_type']= balanced_df['target'].map({'ham': 0, 'spam': 1})
balanced_df_label = balanced_df['msg_type'].values
# Split data into train and test
train_msg, test_msg, train_labels, test_labels = train_test_split(balanced_df['text'], balanced_df_label, test_size=0.2, random_state=434)
print(len(train_msg),len(test_msg),len(train_labels),len(test_labels))
# Defining pre-processing hyperparameters
max_len = 250
trunc_type = "post"
padding_type = "post"
oov_tok = "<OOV>"
vocab_size = 10000
tokenizer = Tokenizer(num_words = vocab_size,char_level=False, oov_token = oov_tok)
tokenizer.fit_on_texts(train_msg)
# Get the word_index
word_index = tokenizer.word_index
word_index
# check how many words
tot_words = len(word_index)
print('There are %s unique tokens in training data. ' % tot_words)
# Sequencing and padding on training and testing
training_sequences = tokenizer.texts_to_sequences(train_msg)
training_padded = pad_sequences (training_sequences, maxlen = max_len, padding = padding_type, truncating = trunc_type )
testing_sequences = tokenizer.texts_to_sequences(test_msg)
testing_padded = pad_sequences(testing_sequences, maxlen = max_len,
padding = padding_type, truncating = trunc_type)
# Shape of train tensor
print('Shape of training tensor: ', training_padded.shape)
print('Shape of testing tensor: ', testing_padded.shape)
# Before padding
len(training_sequences[0]), len(training_sequences[1])
# After padding
len(training_padded[0]), len(training_padded[1])
print(training_padded[0])
# vocab_size = 100 # As defined earlier
embeding_dim = 16
drop_value = 0.2 # dropout
n_dense = 24
#Dense model architecture
model = Sequential()
model.add(Embedding(vocab_size, embeding_dim, input_length=max_len))
model.add(GlobalAveragePooling1D())
model.add(Dense(24, activation='relu'))
model.add(Dropout(drop_value))
model.add(Dense(1, activation='sigmoid'))
# #Dense model architecture
# model = Sequential()
# model.add(Embedding(embeding_dim))
# model.add(GlobalAveragePooling1D())
# model.add(Dense(24, activation='relu'))
# model.add(Dropout(drop_value))
# model.add(Dense(1, activation='sigmoid'))
model.summary()
model.compile(loss='binary_crossentropy',optimizer='adam' ,metrics=['accuracy'])
# fitting a dense spam detector model
num_epochs = 30
early_stop = EarlyStopping(monitor='val_loss', patience=3)
history = model.fit(training_padded, train_labels, epochs=num_epochs, validation_data=(testing_padded, test_labels),callbacks =[early_stop], verbose=2)
# Model performance on test data
model.evaluate(testing_padded, test_labels)
# Read as a dataframe
metrics = pd.DataFrame(history.history)
# Rename column
metrics.rename(columns = {'loss': 'Training_Loss', 'accuracy': 'Training_Accuracy', 'val_loss': 'Validation_Loss', 'val_accuracy': 'Validation_Accuracy'}, inplace = True)
def plot_graphs1(var1, var2, string):
metrics[[var1, var2]].plot()
plt.title('Training and Validation ' + string)
plt.xlabel ('Number of epochs')
plt.ylabel(string)
plt.legend([var1, var2])
plot_graphs1('Training_Loss', 'Validation_Loss', 'loss')
plot_graphs1('Training_Accuracy', 'Validation_Accuracy', 'accuracy')
predict_msg = ["Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...",
"Ok lar... Joking wif u oni...",
"Free entry in 2 a wkly comp to win FA Cup final tkts 21st May 2005. Text FA to 87121 to receive entry question(std txt rate)T&C's apply 08452810075over18's"]
# Defining prediction function
def predict_spam(predict_msg):
new_seq = tokenizer.texts_to_sequences(predict_msg)
padded = pad_sequences(new_seq, maxlen =max_len,
padding = padding_type,
truncating=trunc_type)
return (model.predict(padded))
predict_spam(predict_msg)
# # Defining prediction function
# def predict_spam(predict_msg):
# new_seq = tokenizer.texts_to_sequences(predict_msg)
# padded = pad_sequences(new_seq,
# padding = padding_type,
# truncating=trunc_type)
# return (model.predict(padded))
# predict_spam(predict_msg)
a=["Benefits worth Rs 10,000 on New JioFiber Entertainment Bonanza. Enjoy 550+ on-demand channels, 14+ OTT Apps, Video Calling on TV & High-Speed Internet. No Security Deposit for both boxes or any Installation Charge. Plans starting at Rs 399+100/month Click www.jio.com/r/uhNo1lqem to get your JioFiber NOW"]
predict_spam(a)
predict_spam(["freekjhjksdhklsklhsdsih"])
text="The definite article is the word the. It limits the meaning of a noun to one particular thing. For example, your friend might ask, “Are you going to the party this weekend?” The definite article tells you that your friend is referring to a specific party that both of you know about. The definite article can be used with singular, plural, or uncountable nouns. Below are some examples of the definite article the used in"
predict_spam(["Hi"])
predict_spam(["freekjhjksdhklsklhsdsih"])
predict_spam(["The AIDFI Ram Pump is a perfected model of a forgotten technology: the hydraulic ram pump. It is an automatic device which utilizes the energy contained in moving water (hydro power) to lift a portion of this water to high elevations 24/7. For the AIDFI model there are hardly any operating costs since it doesn’t use electricity or fuel and has very cheap, locally available spare parts (e.g. an ordinary door hinge and valve from an old car tire). The AIDFI model is a crossbreed (best of both) of expensive heavy duty cast models commercially available and inferior low-cost models developed by appropriate technology groups and universities. The AIDFI Ram Pump, compared to other ram pumps, uses a steel-to-steel impulse valve mechanism, making regular and costly replacement of gaskets unnecessary. This unique mechanism makes the model also very efficient. The highest elevation reached so far in an actual setting has been 240 meters and can most probably go beyond that. Another uniqueness is that the pumps are made one by one per order and are already available in ten different sizes with each pump having a specific flow range making it a perfect solution for either drinking or irrigation projects."])
After the model training I'm getting the good accuracy as 0.99,0.98...But when it came to test on my sample text simple text "Hi how are you" also getting high spam accuracy which is totally wrong..Before this I trained two more different models with different datset..But for one dataset's model of Navie's model it showing good result for the short text and bad result for long text...and after for mails dataset it giving better result but when testing on large text like 10 lines..it's giving bad result....So in my mind I'm not sure but thinking that vocab length and number of words matters lot...So any one please explain me this...