What does a subscript on a probability represent? I've been trying to look this up, but I must not be searching correctly because I think it's a pretty basic question.
What I want to know is what do subscripts mean when talking about the probability (or expectation) of something? For example $P_{\theta}(x|y^{(i)})$. What does the $\theta$ subscript mean here?
I saw someone ask the same question here, but I don't understand the answer.
 A: Maybe this helps:
$P$ is the distribution of random variable $x$ given the value of random variable $y$. And this distribution has parameters $\theta$. By varying the parameters, you get different distributions. For example, probability distribution over a random variable $x$ with  uniform distribution on support $[a,b]$ can be written as $$P_{(a,b)}(x) = \frac{1}{b-a}I_{[a,b]}(x).$$ You will get a different distribution by varying $a$ and $b$.
Edit: Frequently, you will see the probability density function of a random variable $X$ denoted as $f_X(x)$, here the subscript is just used to remember that $f_X$ is a probability density of $X$. Here $X$ is not a parameter. But if is written as $f_\theta(x)$, then $\theta$ is likely to be a parameter of the density function of the random variables $X$ which can also be written as $f_X(x|\theta).$ You will have to see what it is by the context which is very easy to see in all the cases I have encountered.
A: Abhinav Gupta gave a nice example (+1). The general answer is that you can use the subscript to carry descriptive information about the distribution. For example, the definition of independence can be written as
$$
P_{X,Y}(x,y) = P_X(x) \, P_Y(y)
$$
Or you could define mixture distribution as
$$
P_X(x) =\sum_k \pi_k P_k(x)
$$
where $\sum_k \pi_k = 1$ are the mixing weights and $P_k$ are the individual components, the distributions that are “mixed”.
In all the cases, the subscript carries information that makes more precise what does the distribution represent. As with mathematical notation in general, there is no single way how the subscripts are used, so you always need to verify it from the context.
