I am using a Tensorflow Autoencoder to detect anomalies from production data.
I have a list with a total of 1927 elements. Each element represents a numpy array with a shape of 6000,20. (6000 count_timersiere from 19 features - representing one produced part)
all_data = []
for df in yp_clean:
batch_sample = np.asarray(df)
for sample in batch_sample:
all_data.append(sample)
all_data = np.asarray(all_data)
inp = tf.keras.layers.Input(shape = (20, ))
x = tf.keras.layers.Dense(128, activation='relu')(inp)
x = tf.keras.layers.Dense(64, activation='relu')(x)
enc = tf.keras.layers.Dense(2, activation='relu', name='enc')(x)
x = tf.keras.layers.Dense(64, activation='relu')(enc)
x = tf.keras.layers.Dense(128, activation='relu')(x)
out = tf.keras.layers.Dense(20, activation='relu')(x)
autoencoder = tf.keras.models.Model(inp, out)
autoencoder.summary()
the count_timeseries starts at 1 and goes up to 6000, which represents a time period of 30seconds
my approach was to stack all data into one hugh np.array. shape 1927*6000,20 At least i was able to run the autoencoder. but my fear that i lost the "time_series" information.
is my fear correct? has anybody done something similar before?
btw: so fare I only have data from ok parts, next step would be to add nok. but I need to know how to prepare the data in advanced.
thanks for your help!
