How do I use cross categorical encoding with tensorflow Keras Functional API?

Question

I am trying to create a cross-categorical encoding for a lookup using the Keras Function API.

The simplified example (which has no real correlation between target and input data): I have 3 students from each class classes=['Freshman', 'Sophomore', 'Junior', 'Senior']. Each student has a grade grades=['A', 'B', 'C', 'D', 'E', 'F']. Based on this information I want to learn a final Target Grade for each sample.

I thought I could create a cross-categoical encoder using preprocessing.CategoryCrossing(classes,grades) and then feed in the full dataset to lookup its initial input from a hashtable.

This is my example data:

class_df: grades_df:

sample_df:

import pandas as pd
import numpy as np
import tensorflow.keras
import tensorflow as tf

grades_df = pd.DataFrame([['A'],['B'],['D'],['E'],['F']],columns=['Grade'])
class_df = pd.DataFrame([['Freshman'],['Sophomore'],['Junior'],['Senor']],columns=['Class'])
sample_df = pd.DataFrame([['A','D','E', 'F','C','B', 'B','E','A', 'B','D','B','C'],
                          ['B','E','F', 'A','D','A', 'C','F','B', 'A','B','A','A'],
                          ['C','F','A', 'B','E','B', 'D','A','C', 'C','A','C','B'],
                          ['D','A','B', 'C','F','C', 'E','B','D', 'E','D','B','C'],
                          ['F','B','C', 'D','C','D', 'F','C','E', 'A','B','D','D'],
                          ['E','C','D', 'F','D','E', 'A','D','F', 'F','E','F','E'],
                          ['A','D','A', 'F','E','F', 'B','E','A', 'D','F','A','F'],
                          ['B','E','B', 'A','F','A', 'C','F','B', 'A','D','B','A'],
                          ['C','F','C', 'B','A','B', 'D','A','C', 'D','A','C','B'],
                          ['D','F','D', 'C','B','C', 'E','B','D', 'A','F','F','C'],
                          ['E','A','E', 'D','D','D', 'F','C','E', 'F','A','D','D'],
                          ['F','B','A', 'F','E','E', 'A','D','F', 'A','C','E','F'],
                          ['C','C','F', 'E','F','F', 'B','E','A', 'B','D','B','E'],
                          ['B','A','C', 'B','A','A', 'C','F','B', 'B','C','F','C'],
                          ['E','D','D', 'A','B','B', 'C','A','C', 'A','D','B','A']
                         ], columns=['Freshman_1', 'Freshman_2', 'Freshman_3', 
                                     'Sophomore_1', 'Sophomore_2', 'Sophomore_3', 
                                     'Junior_1', 'Junior_2', 'Junior_3', 
                                     'Senior_1', 'Senior_2', 'Senior_3', 'TARGET'])

def create_ds(dataframe):
    df = dataframe.copy()
    target = df.pop('TARGET')
    ds = tf.data.Dataset.from_tensor_slices((dict(df), target))
    return ds

sample_ds = create_ds(sample_df)
grade_buckets = tf.feature_column.categorical_column_with_vocabulary_list(key="Grades", vocabulary_list=grades_df['Grade'])
class_buckets = tf.feature_column.categorical_column_with_vocabulary_list(key="Classes", vocabulary_list=class_df['Class'])

group = {}
group['Freshman'] = [column for column in sample_df if column.startswith("Freshman")]
group['Sophomore'] = [column for column in sample_df if column.startswith("Sophomore")]
group['Junior'] = [column for column in sample_df if column.startswith("Junior")]
group['Senior'] = [column for column in sample_df if column.startswith("Senior")]

inputs = []

encoder = preprocessing.CategoryCrossing(class_df['Class'].head().tolist(),
                                         grades_df['Grade'].head().tolist())
for c in group.keys():
    for feature_column in group[c]:
        categorical_col = tf.keras.Input(shape=(1,), name=feature_column, dtype='string')
'''
##--- HERE IS WHERE I'M STUCK ----
##    How do I lookup the input from the categorical column into the encoded table so that 
##    I can feed the full dataset in and generate the encoded keras layer?
##    - And then how do I encode the target data for training?  
##    -- Then is it possible to remap the output result back through the grade_buckets lookup table?
'''
        inputs.append(categorical_col)

I would like to ultimately pass the sample_ds into the model for training...

all_features = tf.keras.layers.concatenate(inputs)
x = tf.keras.layers.Dense(32, activation="relu")(all_features)
output = tf.keras.layers.Dense(5)(x)

model = Model(inputs, output)
model.compile(optimizer='adam',
              loss=tf.keras.losses.BinaryCrossentropy(from_logits=True),
              metrics=["accuracy"])
model.fit(sample_ds)

I may be going about this completely the wrong way and would love someone to show me the proper direction.

I have tried to use the tf.feature_columns to meet this end but that does not seem to process using the Keras Functional API...

inputs = []
encoded_features = []
for c in group.keys():
    for feature_column in group[c]:
        categorical_col = tf.keras.Input(shape=(1,), name=feature_column, dtype='string')
        studentGrade = tf.feature_column.categorical_column_with_vocabulary_list(key=feature_column, vocabulary_list=grades_df['Grade'])
        studentClass = tf.feature_column.categorical_column_with_vocabulary_list(key=c, vocabulary_list=class_df['Class'])
        student_Grade_Class = tf.feature_column.crossed_column([studentGrade, studentClass], len(grades_df)*len(class_df))
        student_Grade_Class_embedding = tf.feature_column.embedding_column(student_Grade_Class, len(grades_df)*len(class_df))
        inputs.append(categorical_col)
        encoded_features.append(student_Grade_Class_embedding)
targets = tf.feature_column.categorical_column_with_vocabulary_list(key='TARGET', vocabulary_list=grades_df['Grade'])
targets_embedding = tf.feature_column.embedding_column(student_Grade_Class, len(grades_df))

How do I use cross categorical encoding with tensorflow Keras Functional API?

0 Answers