Can we actually get the probability of a text using recurrent neural network?

Question

I know that recurrent neural network is used to generate text and to model the probability of $P(x_0,x_1,x_2,x_3)=P(x_0)P(x_1|x_0)P(x_2|x_1,x_0)P(x_3|x_2,x_1,x_0)$ where $x_i$ is words/text.
If RNN can model probability, Can we calculate the probability of a text?
for example: It produces low probability for text with no meaning and high probability for normal text.
What about CNN, Can we generate probability for images(for example: low probability for fake images)?

Answer 1

Yes, in fact that's how most language models are trained -- you compute the (log) probability of given input text, and then backpropagate to maximize that probability.

There are some generative models with explicit densities: you can easily evaluate the probability/density the model assigns to any data point. There are some other generative models with only approximate densities (you can only get a lower bound on the probability of a point) -- or with implicit density (in which case finding or approximating the density is computationally intractable).

RNNs (and more broadly, autoregressive models) are in the explicit density family. There are many CNN based generative models of images (autoregressive, GAN, VAE, flows, etc), some of which are explicit, approximate, and implicit density each.