Recurrent Neural Networks

  1. Recurrent Neural Networks are especially useful when past observations can help the current predictions.
  2. LSTMs with Peepholes allow the current memory (cell) state to influence the gates.
  3. GRUs tie the gates so every time something is forgotten, it is replaced by something new.
  4. LSTMs are very straightforward in PyTorch. It's just a loop.
Recurrent Nets and LSTMs