Dynamic batch size in Keras LSTM (TensorFlow 2.x)

user_17964966 0
Viewed 298

I have a data frame that contains 27k records as a training set and another testing dataset with 4k records. Both datasets have 25 features each.

x_train shape: (27000, 25), 
x_test shape: (4000, 25)

Example of data in the training set:

|Subject ID|Feat_1|Feat_2|Feat_X|Hr_count|Label|
|s0001     |    89| 31   |  43  |   1    |  0  |
|s0001     |    94| 32   |  68  |   2    |  0  |
|s0001     |    38| 90   |  86  |   3    |  0  |
|s0001     |    79| 34   |  78  |   4    |  1  |
|s0001     |    85| 24   |  70  |   5    |  1  |
|s0002     |    7 | 9    |  32  |   1    |  0  |
|s0002     |    60| 56   |  72  |   2    |  0  |
|s0002     |    68| 72   |  23  |   3    |  0  |
|s0003     |    26| 88   |  1   |   1    |  0  |
|s0004     |    45| 27   |  22  |   1    |  0  |
|s0004     |    10| 80   |  67  |   2    |  0  |
|s0004     |    71| 48   |  21  |   3    |  0  |
|s0004     |    58| 9    |  60  |   4    |  1  |

Hr_count: Represents the hour each subject stayed in the experiment

Label: This is my target variable when building my classifier. It Represents the flag that the subject received after staying in the experiment

I trained the data on an LSTM RNN model which was defined as below:

model = Sequential()
model.add(LSTM(100, activation='tanh', return_sequences=True, input_shape=(1, 25)))
model.add(LSTM(49, activation='tanh'))
model.add(Dense(1, activation='sigmoid'))
 
model.fit(
    x_train, y_train,
    validation_data=(x_test, y_test),
    batch_size=32,
    epochs=200)

Question:

Due to the sequential nature of the data, I would like to define a dynamic batch_size parameter to be the max number of Hr_count per subject in the training when fitting the model so that the LSTM can pick up the relationships between the data for each subject separately (Each batch will contain the data for each subject only). This will mean each batch contains samples for 1 subject, ordered by Hr_count.

The flexibility of having a dynamic batch_size does not seem to be available in Keras or TensorFlow v2.x (contrary to TensorFlow v1.x)...

How can I define the batch size to be dynamic for the batch_size parameter?

lstm keras tensorflow python
2 Answers

You can make a loop that calls the model.fit() function for every subject and then set the batch size depending on the current Hr_count

for subject in list_of_subjects:
    hr_count,data = subject
    x_train,y_train = data
    model.fit(
    x_train, y_train,
    validation_data=(x_test, y_test),
    batch_size=hr_count,
    epochs=200)

list_subject for this code to run mus have the shape

[[Hr_count,[x_triain,y_train]]
user_148562410

I filtered on subject Id then fed the data segments to the model.fit(). It seems like the model learns quickly. try it on a bigger data set. the code is generalized to allow for more features.

import pandas as pd
from io import StringIO
import io
from keras.models import Sequential
from keras.layers import LSTM, Dense
import numpy as np
from sklearn.preprocessing import LabelEncoder

data="""SubjectID,Feat_1,Feat_2,Feat_X,Hr_count,Label
s0001,89,31,43,1,0
s0001,94,32,68,2,0
s0001,38,90,86,3,0
s0001,79,34,78,4,1
s0001,85,24,70,5,1
s0002,7 ,9 ,32,1,0
s0002,60,56,72,2,0
s0002,68,72,23,3,0
s0003,26,88,1 ,1,0
s0004,45,27,22,1,0
s0004,10,80,67,2,0
s0004,71,48,21,3,0
s0004,58,9 ,60,4,1
"""

df=pd.read_csv(io.StringIO(data),sep=",")
df.drop(columns='Hr_count',inplace=True)
encoder=LabelEncoder()
df['SubjectID']=encoder.fit_transform(df['SubjectID'])

print(df)

X_columns=[x for x in df.columns if x!='Label']
features=len(X_columns)
model = Sequential()
model.add(LSTM(100, activation='tanh', return_sequences=True, input_shape=(1, features)))
model.add(LSTM(49, activation='tanh'))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer="rmsprop", loss='binary_crossentropy', metrics=['accuracy'])


grouped=df.groupby('SubjectID')



for group in grouped:
    df_batch=pd.DataFrame(columns=['SubjectID','Feat_1','Feat_2','Feat_X','Hr_count','Label'])
    for subjectID in group:
        filter=df['SubjectID']==subjectID 
        for key,item in df[filter].dropna().iterrows():
            df_batch=df_batch.append({'SubjectID':item['SubjectID'],'Feat_1':item['Feat_1'],'Feat_2':item['Feat_2'],'Feat_X':item['Feat_X'],'Label':item['Label']},ignore_index=True)
    #print("\n",df_batch)
    X=df_batch[X_columns]
    X = np.resize(X,(X.shape[0],1,X.shape[1]))
    y=df_batch['Label']
    print("\n",X)
    model.fit(X,y,batch_size=len(X), 
          epochs=10)

output:

  Epoch 10/10
  1/1 [==============================] - 0s 13ms/step - loss: 0.0588 - accuracy: 1.0000
user_40011770
Related