What is an autoregressive decoder? I saw that this was part of a deep belief network I was looking at.
I'm not sure what it means.
Is it a layer that transforms few inputs into many outputs and has a connection to itself?
What is an example of it?
 A: I think the confusion stems from the term autoregression.
Autoregressive is anything that uses previous information from time steps to predict (regress) the output at the current time step. Refer to the links given below for more information on autoregression [1,2].
Examples can be any time series models, for example, predicting share prices based on historical share prices.
An autoregressive decoder is a decoder model which uses information from previous time steps of the decoder to generate the value at the current time step. An example of such an autoregressive decoder is a English-French machine translation model. While translating from English to French, the decoder will condition its current time step's prediction on the previously generated words (past time steps) [3].

*

*https://en.wikipedia.org/wiki/Autoregressive_model

*https://www.investopedia.com/terms/a/autoregressive.asp

*https://ojs.aaai.org/index.php/AAAI/article/download/4476/4354 (Wang, Yiren, et al. "Non-autoregressive machine translation with auxiliary regularization." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019.)

A: decoder is a part that usually produces output from some hidden state. Autoregressive means that this is a recurrent structure that uses prediction from a previous state to generate next step, e.g. use previous predicted output word to generate next output word during translation.
