I'm trying to understand how LSTM can solve a Sentiment Analysis problem. Currently, I'm confused by this structure: 
So here is my list of questions:
After you fed LSTM layer with vectors which represented each words of each sentences of the dataset, what does LSTM layer do with them and what is the output?
The final output of the architecture you show (up to the point it is fed into the softmax) is a sentence embedding, i.e. a single (high-dimensional) floating-point vector that represents the sentence. (You showed the unrolled version, but I find it useful to be simultaneously thinking of its real rolled up form, too.)
As far as this question goes, it is doing the same thing as a simple RNN, so make sure you understand that first. There are a lot of articles and videos explaining it. E.g. https://towardsdatascience.com/animated-rnn-lstm-and-gru-ef124d06cf45
What does the forget gate do?
It tries to learn which are the more important words. If being trained for sentiment analysis it will hopefully learn to give more weight to words that are emotive, and less weight to the words carrying no sentiment. You might also see it called a keep gate. When the weight is high it keeps the word, when the weight is low it forgets the word.
Say you had a long sentence like: "This movie has some really funny characters and superb action scenes, and was directed by Tom Smith, produced by Dick Jones, and began production in 2019." The problem here for an RNN doing sentiment analysis is that it maintains state one word at a time. Each step it loses a bit of what went before. By the end it has all but forgotten the positive early words.
You really want it seen as: "really funny characters, superb action scenes.". That is a much shorter sentence to learn and understand. So this is what the forget gate in an LSTM is trying to do for you.
Spoiler alert: It does not do it that well, but it is distinctly better than not doing it at all.
Why there are two LSTM layers?
To allow more sophisticated understanding and better sentence embeddings. It is the same as adding another layer in a fully-connected neural network. LSTMs don't scale very well beyond two layers however.
how the model know that sentence's sentiment base on that vector exactly? Does that vector have some kind of features that make it positive or negative
The answer to your second sub-question is "maybe, but not always in a human-readable form". What the very last layer of your architecture is doing is trying to learn what kind of sentiment each element of the vector indicates, using a classic fully-connected neural net.
E.g. say you use vectors of dimension 100, and you want to classify into three classes: angry, happy, sad. Your training data then has sentences like "Everything went smoothly today" with a label of "happy". Maybe that sentence has a high number in element 77 of the vector the LSTM produces. Maybe most of the "happy" sentences have a high value in that element, but the angry and sad sentences show no pattern for it. So it gives a high positive weight connecting element 77 to the happy output, and a zero weight connecting 77 to each of angry and sad.
If your goal is improving your intuition, it can be interesting to build models with just 2 or 3 dimensions, and just a few training sentences, and then deliberately overfit the model on your training data. Sometimes you get noise, but sometimes you get lucky and can follow through how the vectors for each word interact, to give a different pattern in the output, and how the final layer combines them to give a prediction.
