I want to add one-to-one layer to deep learning with keras

user_14067192 0
Viewed 315

I want to add feature selection layer in deep learning with Keras. I wrote following code.

However, I need one more one-to-one linear layer after input layer. I know I can't use Dense layer, but how code should I use?

input_layer = Input(shape=(len(input_df.columns),), dtype='float')
modelx_1 = Dense(64)(input_layer)
modelx_1 = Activation("relu")(modelx_1)
modelx_1 = Dropout(0.2)(modelx_1)
modelx_2 = Dense(16)(modelx_1)
modelx_2 = Activation("linear")(modelx_2)
modelx_2 = Dropout(0.2)(modelx_2)
modelx_2 = Dense(1)(modelx_2)
modely_1 = Dense(64)(input_layer)
modely_1 = Activation("relu")(modely_1)
modely_1 = Dropout(0.2)(modely_1)
modely_2 = Dense(16)(modely_1)
modely_2 = Activation("linear")(modely_2)
modely_2 = Dropout(0.2)(modely_2)
modely_2 = Dense(1)(modely_2)

output_layer = Concatenate()([modelx_2, modely_2])
model = Model(inputs=input_layer, outputs=output_layer)

Here is my target deep learning architecture. enter image description here (https://www.researchgate.net/publication/271329170_Deep_feature_selection_Theory_and_application_to_identify_enhancers_and_promoters)

If you don't mind, please let me know.

keras layer deep-learning python
1 Answers

I am not sure the snippet you provided is matching the expected architecture displayed in the image since you have multiple outputs in your snippet but there is only 1 output with 3 units in the image displayed or maybe I am missing something.

My answer will be used to define the architecture displayed on the image.

You have several ways to do it, the first question is: do the weights of the Weighted Input layer are trainable or constant?

  1. If the weights are constant, you have 2 solutions:

1.1. Define a constant tensor with the weights and use it to multiply the input features:

import tensorflow as tf

n_features = 4
weights = tf.constant([1.1, 1.2, 1.3, 1.4])


input_ = tf.keras.layers.Input(shape=(n_features,))
input_weighted = tf.multiply(input_, weights)
dense_1 = tf.keras.layers.Dense(units=5, activation="relu")(input_weighted)
dense_2 = tf.keras.layers.Dense(units=4, activation="relu")(dense_1)
output = tf.keras.layers.Dense(units=3, activation="softmax")(dense_2)

model = tf.keras.Model(inputs=input_, outputs=output)

tf.keras.utils.plot_model(model, show_shapes=True)

This give you the following model: enter image description here

1.2. Use an input layer to provide the weights

import tensorflow as tf

n_features = 4

input_ = tf.keras.layers.Input(shape=(n_features,))
weights = tf.keras.layers.Input(shape=(n_features,))
input_weighted = tf.keras.layers.Multiply(name="weighted_input")([input_, weights])
dense_1 = tf.keras.layers.Dense(units=5, activation="relu")(input_weighted)
dense_2 = tf.keras.layers.Dense(units=4, activation="relu")(dense_1)
output = tf.keras.layers.Dense(units=3, activation="softmax")(dense_2)

model = tf.keras.Model(inputs=[input_, weights], outputs=output)

tf.keras.utils.plot_model(model, show_shapes=True)

This gives you the following model: enter image description here

  1. If the weights are trainable, you can create a custom layer and use it in your model
import tensorflow as tf

class WeightedLayer(tf.keras.layers.Layer):
    
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
    
    def build(self, input_shape):
        self.kernel = self.add_weight("kernel", shape=(int(input_shape[-1]),))
    
    def call(self, inputs):
        return tf.multiply(inputs, self.kernel)

n_features = 4

input_ = tf.keras.layers.Input(shape=(n_features,))
input_weighted = WeightedLayer(name="weighted_input")(input_)
dense_1 = tf.keras.layers.Dense(units=5, activation="relu")(input_weighted)
dense_2 = tf.keras.layers.Dense(units=4, activation="relu")(dense_1)
output = tf.keras.layers.Dense(units=3, activation="softmax")(dense_2)

model = tf.keras.Model(inputs=input_, outputs=output)

tf.keras.utils.plot_model(model, show_shapes=True)

This gives you the following model: enter image description here

By printing the model summary each time you can see that for the 2 first models you have 64 trainable parameters and for the last model you have 68 trainable parameters which corresponds to the 64 from the first two models + 4 weights added

user_80565720
Related