Character-based Text Classification with Triplet Loss

Im trying to implement a text-classifier using triplet loss to classify different job descriptions into categories based on this paper. But whatever i do, the classifier yields very bad results.

For the embedding i followed this tutorial and the NN architecture is based on this article.

I create my encodings using:

max_char_len = 20
group_numbers = range(0, len(job_groups))
char_vocabulary = {'PAD':0}
X_char = []
y_temp = []
i = 1
for group, number in zip(job_groups, group_numbers):
   for job in group:
      job_cleaned = some_cleaning_function(job)
      job_enc = []
      for c in job_cleaned:
         if c in char_vocabulary.keys():
            job_enc.append(char_vocabulary[c])
         else:
            char_vocabulary[c] = i
            job_enc.append(char_vocabulary[c])
            i+=1
      X_char.append(job_enc)
      y_temp.append(number)
X_char = pad_sequences(X_char, maxlen = max_char_length, truncating='post')

My Neural Network is set up the following way:

def create_base_model():
  char_in = Input(shape=(max_char_length,), name='Char_Input')
  char_enc = Embedding(input_dim=len(char_vocabulary)+1, output_dim=20, mask_zero=True,name='Char_Embedding')(char_in)
  x = Bidirectional(LSTM(64, return_sequences=True, recurrent_dropout=0.2, dropout=0.4))(char_enc)
  x = Bidirectional(LSTM(64, return_sequences=True, recurrent_dropout=0.2, dropout=0.4))(x)
  x = Bidirectional(LSTM(64, return_sequences=True, recurrent_dropout=0.2, dropout=0.4))(x)
  x = Bidirectional(LSTM(64, return_sequences=False, recurrent_dropout=0.2, dropout=0.4))(x)
  out = Dense(128, activation = "softmax")(x)
  return Model(char_in, out)

def get_siamese_triplet_char():
  anchor_input_c = Input(shape=(max_char_length,),name='Char_Input_Anchor')
  pos_input_c = Input(shape=(max_char_length,),name='Char_Input_Positive')
  neg_input_c = Input(shape=(max_char_length,),name='Char_Input_Negative')

  base_model = create_base_model(encoding_generator)
  encoded_anchor = base_model(anchor_input_c)
  encoded_positive = base_model(pos_input_c)
  encoded_negative = base_model(neg_input_c)

  inputs = [anchor_input_c, pos_input_c, neg_input_c]
  outputs = [encoded_anchor, encoded_positive, encoded_negative]

  siamese_triplet = Model(inputs, outputs)
  siamese_triplet.add_loss((triplet_loss(outputs)))

  siamese_triplet.compile(loss=None, optimizer='adam')
  return siamese_triplet, base_model

The triplet loss is defined as follows:

def triplet_loss(inputs):
   anchor, positive, negative = inputs
   positive_distance = K.square(anchor - positive)
   negative_distance = K.square(anchor - negative)
   positive_distance = K.sqrt(K.sum(positive_distance, axis=-1, keepdims = True))
   negative_distance = K.sqrt(K.sum(negative_distance, axis=-1, keepdims = True))
   loss = positive_distance - negative_distance
   loss = K.maximum(0.0, 1 + loss)
   return K.mean(loss)

The model is then trained with:

    siamese_triplet_char.fit(x=
              [Anchor_chars_train, 
               Positive_chars_train, 
               Negative_chars_train],
              shuffle=True, batch_size=8, epochs=22, verbose=1)

My goal is to: First, train the network with no label data in order to minimize the space of the different phrases and second, add a classification layer and create the final classifier.

My general problem is that even the first phase shows sinking cost-values it overfits and the validation results jump around and the second phase fails badly as I'm not able to train the model to actually classify.

My questions are the following:

1. Could someone explain the Embedding Architecture? What is the output dimension refering to? The individual characters? Would that even make sense? Or is there a better way to encode the input data?
2. How can i add validation_data to a network that does not contain labeled data? I could use validation_split, but i would rather prefer passing specific data to validate as my data is stratified.
3. Is there a reason why the classification does not work? Applying a simple K-Nearest Neighbor algorithm achieves at best 0.5 accuracy! Is it because of the data? Or is there a systematic error in my system?

All ideas and suggestions are really appreciated!