9
$\begingroup$

In the context of likelihood-based inference, I've seen some notation concerning the parameter(s) of interest which I've found a little confusing.

For example, notation such as $p_{\theta}(x)$ and ${\mathbb E}_{\theta}\left[S(\theta)\right]$.

What is the significance of the parameter ($\theta$) in subscript notation above? In other words, how should it be read?

My first assumption was that it simply meant "with parameter $\theta$"; for example, for $p_{\theta}(x)$, it would read:

"The probability density of $x$ with parameter $\theta$."

However, this probably isn't correct because $p_{\theta}(x) = L(\theta)$ and, in general, $L(\theta)$ is not a distribution (i.e. it does not integrate to unity); hence it can't be a density, can it?

In addition, in the case of ${\mathbb E}_{\theta}\left[S(\theta)\right]$, I'm not sure what it changes relative to ${\mathbb E}\left[(S(\theta)\right]$ (i.e. with the subscript $\theta$ omitted).

In the above $S(\theta)$ and $L(\theta)$ signify the score function and likelihood function respectively.

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ $p_\theta$ is a probability (or density) for each $\theta$, that does not imply that the likelihood is a density function as a function of $\theta$. $\endgroup$
    – Stéphane Laurent
    May 2, 2013 at 23:03
  • 1
    $\begingroup$ Thanks for your reply! $$$$ So $p_{\theta}(x) \text{ is equivalent to } p(x;\theta)$? $$$$ From this, can I assume that: $p_{\theta}(x) = L(\theta) \text{ but } \int p_{\theta}(x) dx = 1 \neq \int L(\theta) d\theta$ $$$$ And also that ${\mathbb E}_{\theta}(f(x))$ refers to the expectation of $x$ for each $\theta$ such that: ${\mathbb E}_{\theta}(f(x)) = \int f(x)p_{\theta}(x)dx$ $$$$ $\endgroup$
    – Hugo
    May 2, 2013 at 23:56
  • 1
    $\begingroup$ Usually the notation $\text{E}_X()$ represents an expectation with respect to the random variable $X$; if you're in a situation where it makes sense to regard $\theta$ as a random variable (such as a Bayesian context), that would be the intent. If you're not in a situation where $\theta$ could be regarded as a random variable, @Hugo's comment would the be meaning I'd look at next. $\endgroup$
    – Glen_b
    May 3, 2013 at 0:30
  • 1
    $\begingroup$ @Hugo Yes you understand. Rigorously we should always denote the expectation $\mathbb{E}_{P}$ where $P$ is the underlying probability but this is useless when there is only one $P$. Here $\mathbb{E}_\theta$ is a shortcut for $\mathbb{E}_{p_{\theta}}$. The notation $\mathbb{E}_X$ mentioned by Geln_b is appropriate for other contexts but usually I don't like this notation. $\endgroup$
    – Stéphane Laurent
    May 3, 2013 at 6:03

1 Answer 1

Reset to default
3
$\begingroup$

This is mostly answered in comments which I will summarize here.

$p_\theta(x)$ means the same as $p(x; \theta)$, its a shorthand. This is a density with respect to $x$, not with respect to $\theta$. So while by necessity $\int p(x;\theta)\; dx = 1$ it does not follow that $\int p(x;\theta)\; d\theta=1$, it could be anything, including $\infty$.

So, ${\mathbb E}_{\theta}\left[S(\theta)\right]$ is the expectation of $S(\theta)$ with respect to the distribution $p_\theta(x)$. The subscript $\theta$ is there for clarity, not because it is necessary, so ${\mathbb E}\left[(S(\theta)\right]$ has the same meaning. The distribution with respect to which we calculate the expectation should be clear from context, or indicated somehow (like by a subscript).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.