I'm studying the Fisher information, which leads me to the wiki page:


It says:

Formally, the partial derivative with respect to θ of the natural logarithm of the likelihood function is called the “score”. The variance of the score is defined to be the Fisher information. If log f(x; θ) is twice differentiable with respect to θ, and under certain regularity conditions, then the Fisher information may also be written as: Fisher information

However, in the very beginning, the notation was specified as follows:

Let f(X; θ) be the probability density function (or probability mass function) for X conditional on the value of θ.

Hence, it makes me comes to a question. Is the f in the picture a density function or a likelihood function?

I've checked the wiki page of the likelihood function and they are obviously different. https://en.wikipedia.org/wiki/Likelihood_function

and for a statistic like $X_1,\ldots,X_n$, the likelihood function should be something like: $f(x_1,\ldots,x_n|\theta)$ right?

Which definition should I take in? or have I taken it totally wrong at some point? lol

Great thanks!

  • 3
    $\begingroup$ A likelihood function is a pdf (or a pmf). The reason why people often use ";" instead of "|" in a likelihood is that it is often taken as a function on $\theta$. Note it is a function on $\theta$, but not a pdf on $\theta$. $\endgroup$
    – Tim Mak
    May 5 '20 at 6:05
  • $\begingroup$ @TimMak Ok, so do you mean that $f(X;\theta)=f(x_1,\ldots,x_n|\theta)$? $\endgroup$
    – 李智修
    May 5 '20 at 6:14
  • 1
    $\begingroup$ Yes. The whole problem is that of notation. $f()$ is often used to denote a pdf. However, it is also a general notation for a function. The $f(x;\theta)$ notation allows it to be both a pdf on $x$ and a function on $\theta$. $\endgroup$
    – Tim Mak
    May 5 '20 at 8:26
  • $\begingroup$ Ok, I now understand it. Great thanks! $\endgroup$
    – 李智修
    May 10 '20 at 11:05

I would think about your goal: when you use a probability density function (pdf) you are computing information on $X$ based on some known "fixed" parameters $\theta$. On the other side, you use the likelihood to get information about some parameters $\theta$ based on a sample that you observed $x = (x_1, x_2, \dots x_n)$, so it is a function of $\theta$ for an observed "fixed" sample $x$.

So in this sense you can use the notation $f(X|\theta)$ or $f(\theta|x)$ for $f(X;\theta)$.

In any case remember that the pdf is not a probability.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.