AI Features
Home
Courses
Natural Language Processing with Machine Learning

Final States

Learn about the final state output of an LSTM and BiLSTM.

We'll cover the following...

Chapter Goals:

  • Learn about the final states for an LSTM and BiLSTM

A. The encoder

In an encoder-decoder model for seq2seq tasks, there are two components: the encoder and the decoder. The encoder is responsible for extracting useful information from the input sequence. For NLP applications, the encoder is normally an LSTM or BiLSTM.

B. LSTM final state

When using an LSTM or BiLSTM encoder, we need to pass the final state of the encoder into the decoder. The final state of an LSTM in TensorFlow is represented by the ...

Python 3.5
import tensorflow as tf
# Input sequences (embedded)
# Shape: (batch_size, max_seq_len, embed_dim)
input_embeddings = tf.compat.v1.placeholder(
    tf.float32, shape=(None, None, 4))
cell = tf.keras.layers.LSTMCell(5)
rnn = tf.keras.layers.RNN(
    cell,
    return_state=True)
output  = rnn(input_embeddings)
#With final_state = True , rnn will return 2 final state value that are stored in the output variable.
#get the final states in "final_state"
final_state = {output[1]} , {output[2]}
# final_state is the output of our LSTM encoder.
# it contains all the information about our input sequence,
# which in this case is just a tf.compat.v1.Placeholder object
print(final_state)

An ...