Understanding LSTMs – Part 6: How LSTM Produces Its Final Output
Rijul Rajesh
In the previous article we went through the input gate, in this article we will explore the next component.
Final Stage: Updating the Short-Term Memory
This final stage updates the short-term memory.
We begin with the new long-term memory and use it as the input to the tanh activation function.
After plugging in 2.96 into the tanh function, we obtain:
The value 0.99 represents a potential short-term memory.
Deciding How Much to Output
Now the LSTM must determine how much of this potential short-term memory to pass forward.
As in the previous stages, we use the sigmoid activation function to decide what percentage the LSTM keeps.
After performing the calculation, we obtain 0.99
Creating the New Short-Term Memory
We now multiply the two values:
This produces the new short-term memory: 0.98.
This value, 0.98, is also the final output of the entire LSTM unit.
Because the new short-term memory is also the output of the LSTM unit, this stage is called the output gate.
Now that we understand how the three stages of an LSTM work, we will see them in action with real data in the next article.
Looking for an easier way to install tools, libraries, or entire repositories?
Try Installerpedia: a community-driven, structured installation platform that lets you install almost anything with minimal hassle and clear, reliable guidance.
Just run:
ipm install repo-name
… and you’re done! 🚀
