Predicting the square root of a number using Machine Learning

user_13825240 0
Viewed 464

I am trying to create a program in python that uses machine learning to predict the square root of a number. I am listing what all I have done in my program:-

  1. created a csv file with numbers and their squares
  2. extracted the data from csv into suitable variables (X stores squares, y stores numbers)
  3. scaled the data using sklearn's, StandardScaler
  4. built the ANN with two hidden layers each of 6 units (no activation functions)
  5. compiled the ANN using SGD as the optimizer and mean squared error as the loss function
  6. trained the model. Loss was around 0.063
  7. tried predicting but the result is something else.

My actual code:-

import numpy as np
import tensorflow as tf
import pandas as pd

df = pd.read_csv('CSV/SQUARE-ROOT.csv')

X = df.iloc[:, 1].values
X = X.reshape(-1, 1)
y = df.iloc[:, 0].values
y = y.reshape(-1, 1)

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0, test_size=0.2)

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_test_sc = sc.fit_transform(X_test)
X_train_sc = sc.fit_transform(X_train)
sc1 = StandardScaler()
y_test_sc1 = sc1.fit_transform(y_test)
y_train_sc1 = sc1.fit_transform(y_train)

ann = tf.keras.models.Sequential()
ann.add(tf.keras.layers.Dense(units=6))
ann.add(tf.keras.layers.Dense(units=6))
ann.add(tf.keras.layers.Dense(units=1))

ann.compile(optimizer='SGD', loss=tf.keras.losses.MeanSquaredError())

ann.fit(x = X_train_sc, y = y_train_sc1, batch_size=5, epochs = 100)

print(sc.inverse_transform(ann.predict(sc.fit_transform([[144]]))))

OUTPUT:- array([[143.99747]], dtype=float32)

Shouldn't the output be 12? Why is it giving me the wrong result?

I am attaching the csv file I used to train my model as well: SQUARE-ROOT.csv

keras tensorflow machine-learning neural-network python
2 Answers

TL;DR: You really need those nonlinearities.

The reason behind it not working could be one (or a combination) of several causes, like bad input data range, flaws in your data, over/underfitting, etc.

However, in this specific case the model you build literally can't learn the function you're trying to approximate, because not having nonlinearities makes this a purely linear model, which can't accurately approximate nonlinear functions.

A Dense layer is implemented as follows:

x_res = activ_func(w*x + b)

where x is the layer input, w the weights, b the bias vector and activ_func the activation function (if one is defined).

Your model, then, mathematically becomes (I'm using indices 1 to 3 for the three Dense layers):

pred = w3 * (w2 * ( w1 * x + b1 ) + b2 ) + b3
     = w3*w2*w1*x + w3*w2*b1 + w3*b2 + b3

As you see, the resulting model is still linear. Add activation functions and your mode becomes capable of learning nonlinear functions too. From there, experiment with the hyperparameters and see how the performance of your model changes.

user_32148720

The reason your code does not work is because you apply fit_transform to your test set, which is wrong. You can fix it by replacing fit_transform(test) to transform(test). Although I don't think StandardScaler is neccessary, please try this:

import numpy as np
import tensorflow as tf
import pandas as pd
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split

N = 10000
X = np.arange(1, N).reshape(-1, 1)
y = np.sqrt(X)

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=0, test_size=0.2)


sc = StandardScaler()
X_train_sc = sc.fit_transform(X_train)    
#X_test_sc = sc.fit_transform(X_test)      # wrong!!!
X_test_sc = sc.transform(X_test)

sc1 = StandardScaler()       
y_train_sc1 = sc1.fit_transform(y_train)    
#y_test_sc1 = sc1.fit_transform(y_test)   # wrong!!!
y_test_sc1 = sc1.transform(y_test)

ann = tf.keras.models.Sequential()
ann.add(tf.keras.layers.Dense(units=32, activation='relu'))    # you have 10000 data, maybe you need a little deeper network
ann.add(tf.keras.layers.Dense(units=32, activation='relu'))
ann.add(tf.keras.layers.Dense(units=32, activation='relu'))
ann.add(tf.keras.layers.Dense(units=1))

ann.compile(optimizer='SGD', loss='MSE')
ann.fit(x=X_train_sc, y=y_train_sc1, batch_size=32, epochs=100, validation_data=(X_test_sc, y_test_sc1))

#print(sc.inverse_transform(ann.predict(sc.fit_transform([[144]]))))  # wrong!!!
print(sc1.inverse_transform(ann.predict(sc.transform([[144]]))))
user_121253950
Related