How to find Sentence Similarity using deep learning?

Have you tried the Universal Sentence Encoder (USE), or the Multilingual Universal Sentence Encoder?

There's a colab showing how to score sentence pairs for semantic textual similarity with USE on the Semantic Textual Similarity Benchmark (STS-B) and another for multilingual similarity.

Here's a heatmap of pairwise semantic similarity scores from USE on the Google AI blog post Advances in Semantic Textual Similarity. The model was trained on a large amount of web data so it should work well for a wide variety of input data.

Here is a very elaborate tutorial on how to perform sentence similarity analysis using any of the 50+ sentence Embeddings in NLU, like BERT, USE, Electra, and many more! NLU features over 50+ languages and includes multilingual embeddings!
It takes around 5 lines to generate a similarity matrix with NLU and you can use 3 or more Sentence Embeddings at the same time in just 1 line of code, all you need is :

nlu.load('embed_sentence.bert embed_sentence.electra use')

But let's keep it simple and let's say we want to calculate the similarity matrix for every sentence in our Dataframe

You need the following 3 steps :

1. Calculate embeddings

predictions = nlu.load('embed_sentence.bert').predict(your_dataframe)

2. Calculate the similarity matrix

def get_sim_df_total( predictions,e_col, string_to_embed,pipe=pipe):
  # This function calculates the distances between every sentence pair. Creates for ever sentence a new column with the name equal to the sentence it comparse to 
  # put embeddings in matrix
  embed_mat = np.array([x for x in predictions[e_col]])
  # calculate distance between every embedding pair
  sim_mat = cosine_similarity(embed_mat,embed_mat)
  # for i,v in enumerate(sim_mat): predictions[str(i)+'_sim'] = sim_mat[i]
  for i,v in enumerate(sim_mat): 
    s = predictions.iloc[i].document
    predictions[s] = sim_mat[i]

  return predictions 

sim_matrix_df = get_sim_df_total(predictions,'embed_sentence_bert_embeddings', 'How to get started with Machine Learning and Python' )
sim_matrix_df

3. Plot the heatmap of the similarity matrix

non_sim_columns  = ['text','document','Title','embed_sentence_bert_embeddings']

def viz_sim_matrix_first_n(num_sentences=20, sim_df = sim_matrix_df):
  # Plot heatmap for the first num_sentences
  fig, ax = plt.subplots(figsize=(20,14)) 
  sim_df.index = sim_df.document
  sim_columns = list(sim_df.columns)
  for b in non_sim_columns : sim_columns.remove(b)
  # sim_matrix_df[sim_columns]
  ax = sns.heatmap(sim_df.iloc[:num_sentences][sim_columns[:num_sentences]]) 

  ax.axes.set_title(f"Similarity matrix for the first {num_sentences} in the dataset",)

viz_sim_matrix_first_n()

To learn more checkout these links :)

Article : https://medium.com/spark-nlp/easy-sentence-similarity-with-bert-sentence-embeddings-using-john-snow-labs-nlu-ea078deb6ebf

Colab Notebook for sentence similarity Demo with NLU : https://colab.research.google.com/drive/1LtOdtXtRJ3_N8kYywPd5k2AJMCGcgAdN?usp=sharing

NLU Website : http://nlu.johnsnowlabs.com/