# $L(\theta;x)=f(x;\theta)$ vs. $L(\theta;x)\propto f(x|\theta)$

My second-year notes in Statistical Inference and Modelling (unpublished) have a definition,

The likelihood function of $$X$$, given the data $$x$$, is $$L∶\Theta\rightarrow\mathbb{R}$$ defined by $$L(\theta;x)=f(x;\theta)$$.

My third-year notes in Bayesian Statistics (unpublished) have a statement,

the likelihood is proportional to the joint distribution, i.e. $$L(\theta;x)\propto f(x|\theta)$$.

This makes me wonder several things.

1. What's the significance of the order of parameters separated by a semicolon, e.g. $$(\theta;x)$$ vs. $$(x;\theta)$$?
2. Do $$f(x;\theta)$$ and $$f(x|\theta)$$ refer to the same function?
3. $$f(x|\theta)$$ has a 'given' bar like a conditional but the third-year notes appear to refer to it as a 'joint'. Which is it? and why the apparent mis-match?
4. Is the likelihood $$L$$ equal to the other function $$f$$ or proportional? Is it proportional in general and sometimes equal as a special case, with a proportionality constant of $$1$$?

Edit: The paragraph from the third-year notes.

• The mismatch in (3) likely is a typographical error in your notes or interpretation. The Bayes approach views $\theta$ as a random variable, but in general it is a parameter. The other questions are answered in other threads., most notably stats.stackexchange.com/questions/2641.
– whuber
Sep 12, 2022 at 23:31
• Sep 13, 2022 at 2:04

1. What's the significance of the order of parameters separated by a semicolon, e.g. $$(\theta;x)$$ vs. $$(x;\theta)$$?
2. Do $$f(x;\theta)$$ and $$f(x|\theta)$$ refer to the same function?

The function $$f(\theta;x)$$ and $$f(x;\theta)$$ are the same. It converts two inputs $$x$$ and $$\theta$$ into a number.

Below you see a surface plot that makes this visible. (from the question The basic logic of constructing a confidence interval)

The significance of the order in $$(\theta;x)$$ vs $$(x;\theta)$$ occurs when you take a 'slice' out of this 3d picture. This can be done in different directions

• When you have the slices in direction left to right, along the $$x$$ axis, the interpretation is as a probability density function (pdf)
• When you slice in the direction back to front, along the $$\theta$$ axis, then the interpretation is the likelihood function.

Note that in the image below the function is not the same as a joint probability density. Only the slices in one direction relate to a probability density. What is done in the image, is that we stack multiple probability density functions, by varying $$\theta$$, which is creating a 3d image out of multiple slices.

This likelihood function, should not be tried to be overinterpreted. It is not really a goal by itselve and more like a tool that is used for analysis. If you like then you could interpret likelihood function as a measure for how strong evidence a particular observation $$x$$ is for different values of the parameter $$\theta$$. It is a function of both $$\theta$$ and $$x$$ but in the application we often have $$x$$ fixed. We know the observation and want to use $$x$$ to infer something about $$\theta$$, which may be an unknown parameter. The Bayes factor relates to this interpretation.

1. $$f(x|\theta)$$ has a 'given' bar like a conditional but the third-year notes appear to refer to it as a 'joint'. Which is it? and why the apparent mis-match?

I can not make sense of the lecture notes $$f(x|\theta) = a(x)g(\theta,x)$$

1. Is the likelihood $$L$$ equal to the other function $$f$$ or proportional? Is it proportional in general and sometimes equal as a special case, with a proportionality constant of $$1$$?

In your class they introduced the likelihood as being equal to the conditional probability $$\mathcal{L}(\theta;x) = f(x;\theta)$$, but this was just a simplification. The likelihood does not need to be equal and it is proportional. The reason that it does not need to be equal, is because in the practical application the ratio's of likelihood are used and any constant of proportionality will be eliminated. And also the reason that we want to define it this way (with the proportional I stead of equal) is because often we do not know the exact probability distribution but we do know it relatively (we might know the shape and proportions, but not the exact values).

1. & 2. $$;$$ means $$|$$, i.e., conditional dependence: $$p(x|\theta) = p(x;\theta)$$ and $$f(x|\theta) = f(x;\theta)$$.

2. Also, because $$\theta$$ is not a random variable per se, it's the model's $$f$$ parameters, it is possible that they refer to $$f(x|\theta)=f(x;\theta)=f_\theta(x)$$ as joint, if $$x\in R^d$$ and $$d>1$$.

3. The likelihood function is defined as $$L(\theta)=f(x|\theta)$$. It can be proportional if you omit the constants.

• alienflow and @whuber: I've added an image of the relevant paragraph to the OP. If $;$ and $|$ are interchangeable, or if $|$ is a typo, isn't it odd that $|$ and $;$ are used consistently, side by side, for different functions?
– mjc
Sep 13, 2022 at 2:28
• alienflow, re 3, I don't understand. If $L(\theta)=f(x|\theta)$ is the definition of $L$, how can their relationship be anything other than exact equality? Is there actually another function, $f':=kf$ for some $k\in\mathbb{R}$, which is being conflated with $f$ because the distinction is regarded as unimportant?
– mjc
Sep 13, 2022 at 2:33