# What is the meaning of subscript in $p_{\theta}(x)$ and ${\mathbb E}_{\theta}\left[S(\theta)\right]$?

In the context of likelihood-based inference, I've seen some notation concerning the parameter(s) of interest which I've found a little confusing.

For example, notation such as $p_{\theta}(x)$ and ${\mathbb E}_{\theta}\left[S(\theta)\right]$.

What is the significance of the parameter ($\theta$) in subscript notation above? In other words, how should it be read?

My first assumption was that it simply meant "with parameter $\theta$"; for example, for $p_{\theta}(x)$, it would read:

"The probability density of $x$ with parameter $\theta$."

However, this probably isn't correct because $p_{\theta}(x) = L(\theta)$ and, in general, $L(\theta)$ is not a distribution (i.e. it does not integrate to unity); hence it can't be a density, can it?

In addition, in the case of ${\mathbb E}_{\theta}\left[S(\theta)\right]$, I'm not sure what it changes relative to ${\mathbb E}\left[(S(\theta)\right]$ (i.e. with the subscript $\theta$ omitted).

In the above $S(\theta)$ and $L(\theta)$ signify the score function and likelihood function respectively.

• $p_\theta$ is a probability (or density) for each $\theta$, that does not imply that the likelihood is a density function as a function of $\theta$. May 2, 2013 at 23:03
• Thanks for your reply!  So $p_{\theta}(x) \text{ is equivalent to } p(x;\theta)$?  From this, can I assume that: $p_{\theta}(x) = L(\theta) \text{ but } \int p_{\theta}(x) dx = 1 \neq \int L(\theta) d\theta$  And also that ${\mathbb E}_{\theta}(f(x))$ refers to the expectation of $x$ for each $\theta$ such that: ${\mathbb E}_{\theta}(f(x)) = \int f(x)p_{\theta}(x)dx$ 
– Hugo
May 2, 2013 at 23:56
• Usually the notation $\text{E}_X()$ represents an expectation with respect to the random variable $X$; if you're in a situation where it makes sense to regard $\theta$ as a random variable (such as a Bayesian context), that would be the intent. If you're not in a situation where $\theta$ could be regarded as a random variable, @Hugo's comment would the be meaning I'd look at next. May 3, 2013 at 0:30
• @Hugo Yes you understand. Rigorously we should always denote the expectation $\mathbb{E}_{P}$ where $P$ is the underlying probability but this is useless when there is only one $P$. Here $\mathbb{E}_\theta$ is a shortcut for $\mathbb{E}_{p_{\theta}}$. The notation $\mathbb{E}_X$ mentioned by Geln_b is appropriate for other contexts but usually I don't like this notation. May 3, 2013 at 6:03

$$p_\theta(x)$$ means the same as $$p(x; \theta)$$, its a shorthand. This is a density with respect to $$x$$, not with respect to $$\theta$$. So while by necessity $$\int p(x;\theta)\; dx = 1$$ it does not follow that $$\int p(x;\theta)\; d\theta=1$$, it could be anything, including $$\infty$$.
So, $${\mathbb E}_{\theta}\left[S(\theta)\right]$$ is the expectation of $$S(\theta)$$ with respect to the distribution $$p_\theta(x)$$. The subscript $$\theta$$ is there for clarity, not because it is necessary, so $${\mathbb E}\left[(S(\theta)\right]$$ has the same meaning. The distribution with respect to which we calculate the expectation should be clear from context, or indicated somehow (like by a subscript).