Why are autoregressive models neither directed or undirected, as described in the NADE paper?

Question

In the paper Neural Autoregressive Distribution Estimation (Uria et al., 2016), NADE (and other autoregressive models) seem to be described as neither directed or undirected models:

We’ve described the Neural Autoregressive Distribution Estimator, a tractable, flexible and competitive alternative to directed and undirected graphical models for unsupervised distribution estimation

...

NADE thus substantially differs from this literature focusing on directed and undirected models, benefiting from a few properties that these approaches lack. Mainly, NADE does not rely on latent stochastic hidden units, making it possible to tractably compute its associated data likelihood for some given ordering. This in turn makes it possible to efficiently produce exact samples from the model (unlike in undirected models) and get an unbiased gradient for maximum likelihood training (unlike in directed graphical models).

These slides by Russ Salakhutdinov seem to agree that autoregressive models (subset of fully observed) are separate from "undirected DGMs" and "directed DGMs":

How is this possible? Is it because autoregressive models lack a hierarchy?

However, my intuition is that they are directed, as the Flow-GAN blogpost by Aditya Grover and Manik Dhar suggests.

Similarly, according to a tutorial on generative models by Shakir Mohamed and Danilo Rezende from DeepMind at UAI 2017 (video recording and slides), autoregressive models such as NADE and PixelCNN are classified as directed models:

---

Answer 1

I agree the terminology is misleading -- perhaps a more precise naming would be (un)directed latent variable generative models.

So a "directed latent variable generative model" is any model in the form $P(x) = \int P(x|h)P(h) dh$, where $h$ is the latent variable. This includes VAEs, GANs, and flows. The slide you shows probably just splits GANs off into a separate category because it's such a large topic.

But in the strict sense yes, an autoregressive model is indeed a directed graphical model.