I am implementing a feed forward neural network model with random activation function at the out layer which has only one neuron. The random activation function works as if the output of the last hidden layer is greater than a threshold, the output of the model will be 1, otherwise, the model output is zero.
I used the following code using "get_custom_objects" to customize the activation function, but it gives me an error that "ValueError: initial_value must have a shape specified: Tensor("dense_4/MatMul:0", shape=(?, 1), dtype=float32)"
class Rand(Activation):
def __init__(self, activation, **kwargs):
super(Rand, self).__init__(activation, **kwargs)
self.__name__ = 'rand'
def rand(x):
result = tf.Variable(tf.cond(tf.random.uniform(shape=[1])[0] > tf.Variable(x), 1, 0))
return result
And the followings are the last hidden layer and the output layer:
model.add(Dense(1, activation='relu', use_bias=False))
model.add(Dense(1, activation='rand', use_bias=False))