Understanding GRU
Gated recurrent units (GRU) was invented in 2014 and based on the ideas implemented in LSTM. GRU was made to simplify LSTM and provide a faster and more efficient way of achieving the same goals as LSTM to adaptively remember and forget based on past and present data. In terms of the learning capacity and metric performance achievable, there isn’t a clear silver-bullet winner among the two and often in the industry, the two RNN units are benchmarked against each other to figure out which method provides a better performance level. Figure 4.4 shows the structure of GRU.
Figure 4.4 – A low-level depiction of GRU
Figure 4.4 adopts the same weights and bias notations as the LSTM depicted in Figure 4.2. There are three different names here for the final small letter notation. R being the reset gate, z representing the update gate, and h representing weights used to obtain the next hidden states. This means a GRU cell has fewer...
