The problem

I have a main task. To achieve it, I'm playing some synthetic data to understand the flow.

I have a data that has, let's say, 5 channels. I want to feed a model (let's say model_1) with these 5 channels, using 5 input layers. Then, I want to feed a concatenation and dense layers with output of model_1.

def create_model():
    input_x = Input(shape=(1000, 1), dtype='float32')
    reshape = Reshape((1000, 1))(input_x)

    conv_1 = Conv1D(16, kernel_size=(64), activation='relu')(reshape)
    maxpool_1 = MaxPool1D(pool_size=(4))(conv_1)
    bn_1 = BatchNormalization()(maxpool_1)

    conv_2 = Conv1D(32, kernel_size=(32), activation='relu')(bn_1)
    maxpool_2 = MaxPool1D(pool_size=(4))(conv_2)
    bn_2 = BatchNormalization()(maxpool_2)

    conv_3 = Conv1D(64, kernel_size=(16), activation='relu')(bn_2)
    bn_3 = BatchNormalization()(conv_3)
    conv_4 = Conv1D(128, kernel_size=(4), activation='relu')(bn_3)

    flatten = Flatten()(conv_4)
    dense1 = Dense(units=128, activation='relu')(flatten)
    out = Dense(units=64, activation='relu')(dense1)
    model = Model(input_x, out)

    inputA = Input(shape=(1000, 1), dtype='float32')
    inputB = Input(shape=(1000, 1), dtype='float32')
    inputC = Input(shape=(1000, 1), dtype='float32')
    inputD = Input(shape=(1000, 1), dtype='float32')
    inputE = Input(shape=(1000, 1), dtype='float32')

    cnn_out1 = model(inputA)
    cnn_out2 = model(inputB)
    cnn_out3 = model(inputC)
    cnn_out4 = model(inputD)
    cnn_out5 = model(inputE)

    combined = concatenate([cnn_out1, cnn_out2, cnn_out3, cnn_out4, cnn_out5], axis=-1)

    fully_connected = Dense(128, activation="relu")(combined)
    outputs_fc = Dense(21, activation="softmax")(fully_connected)

    model_encoded = Model(inputs=[inputA, inputB, inputC, inputD, inputE], outputs=outputs_fc)

    model_encoded.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

    print(model_encoded.summary())

    return model_encoded

The summary:

The model plot:

# generator that generates random data
def data_gen():
    x_buffer = []
    y_buffer = []
    xstack = []
    ystack = []
    while True:
        data = np.random.randint(1000, size=(1000, 1))
        label = np.random.randint(5)

        if len(xstack) == 5:
            x_buffer.append(np.array(xstack))
            y_buffer.append(np.array(ystack))
            xstack = []
            ystack = []
            if len(x_buffer) == 5:
                yield x_buffer, y_buffer
                x_buffer = []
                y_buffer = []
        else:
            xstack.append(data)
            ystack.append(label)

With this implementation, I get this error: ValueError: Error when checking input: expected input_27 to have 2 dimensions, but got array with shape (5, 1000, 1)

I can't really explain what that generator does because I really have tried lots of combinations that I got dizzy. I'm open to any code level or abstract advices. How can I achieve this?

Why?

In the main problem, I have 4 classes. And the data have some combinations. The data can be fed to model as one of these:

0 - [0,0,0,0,0]
1 - [1,1,1,1,1] 
2 - [2,2,2,2,2]
3 - [3,3,3,3,3]
4 - [0,1,1,1,1]
5 - [0,2,2,2,2]
6 - [0,3,3,3,3]
7 - [1,1,1,1,0]
8 - [2,2,2,2,0]
9 - [3,3,3,3,0]
10 - [0,1,1,1,0]
11 - [0,2,2,2,0]
12 - [0,3,3,3,0]
13 - [0,1,1,0,0]
14 - [0,2,2,0,0]
15 - [0,3,3,0,0]
16 - [0,0,1,1,0]
17 - [0,0,2,2,0]
18 - [0,0,3,3,0]
19 - [0,0,1,0,0]
20 - [0,0,2,0,0]
21 - [0,0,3,0,0]

The 4 classes and 5 channels have these 22 combinations. At the end, I want to the model learn and predict one of these combinations. By that I want to achieve that the model learns "Okay, if the first two channels are 0, it can't be anything like [0, 0, 1, 1, 1]" or so.

Sory if it's too abstract, but I really stuck. Any suggestiong or corrections would be appreciated.