# Likelihood $L(\theta; \mathbf{y})$: Is $\theta$ a vector of parameters or is it a single parameter?

I have the following definition of likelihood:

Let $$y_1, \dots, y_n$$ be a sample of observations taken on corresponding random variables $$Y_1, \dots, Y_n$$ whose distribution depends on the parameter(s) $$\theta \in \Theta$$.

If $$Y_1, \dots, Y_n$$ are discrete random variables, the likelihood of the sample, $$L(\theta) = L(\theta; y_1, \dots, y_n)$$, is defined to be the joint probability of $$y_1, \dots, y_n$$.

If $$Y_1, \dots, Y_n$$ are continuous random variables, the likelihood of the sample, $$L(\theta) = L(\theta; y_1, \dots, y_n)$$, is defined to be the joint density evaluated at $$y_1, \dots, y_n$$.

So, for $$L(\theta; \mathbf{y})$$, we clearly have that $$\mathbf{y}$$ is the vector of observations $$(y_1, \dots, y_n)$$, but I'm unsure about how to precisely conceptualise, describe, and denote the parameters ($$\theta$$ in this case). Specifically, is $$\theta$$ a vector of parameters, where each observation $$y_i$$ has its own parameter, or is it a single parameter that just belongs to the vector $$\mathbf{y}$$ (or the random variable $$\mathbf{Y}$$)? My question is analogous for all distributions (for instance, $$L(\mu, \sigma^2; \mathbf{y})$$ for the likelihood of normally distributed data).

## EDIT

I guess it depends on whether the random variables $$Y_1, \dots, Y_n$$ are i.i.d.. If so, then they all have the same parameter $$\theta$$, and so we would have $$L(\theta, \mathbf{y})$$, where $$\theta$$ is a single parameter (not a vector of parameters). But, if $$Y_1, \dots, Y_n$$ were not i.i.d., and so have different $$\theta$$ values, then I guess we would have $$L(\mathbf{\theta}, \mathbf{y})$$, where $$\mathbf{\theta}$$ (notice bolded) is a vector of parameters $$\mathbf{\theta} = (\theta_1, \dots, \theta_n)$$.

## EDIT2

But EDIT then begs another question: If we have the random variables $$Y_1, \dots, Y_n$$, then how does it make sense to say that $$y_1, \dots, y_n$$ are observations from these random variables? Shouldn't we have a matrix of random variables $$\mathbf{Y}$$ with $$n$$ columns, so that each column represents the observations of a random variable? In other words, we should have $$\mathbf{Y} = \begin{bmatrix} y_{11} & y_{12} & \dots & y_{1n} \\ y_{21} & y_{22} & \dots & y_{2n} \\ \vdots & \vdots & \vdots & \vdots \\ y_{n1} & y_{n2} & \dots & y_{nn} \end{bmatrix},$$ so that the observations for the first random variable $$Y_1$$ are $$y_{11}, y_{21}, \dots, y_{n1}$$ (in other words, the first column of the matrix)? And so, if the $$Y_1, \dots, Y_n$$ are not i.i.d., then the likelihood would be $$L(\mathbf{\theta}; \mathbf{Y})$$ where $$\mathbf{\theta}$$ (notice bolded) is a vector of parameters $$\mathbf{\theta} = (\theta_1, \dots, \theta_n)$$, and if the $$Y_1, \dots, Y_n$$ are i.i.d., then the likelihood would be, again, $$L(\mathbf{\theta}; \mathbf{Y})$$ where $$\mathbf{\theta}$$ (notice bolded) is a vector of parameters $$\mathbf{\theta} = (\theta_1, \dots, \theta_n)$$, but, in this case, we would have that $$\theta_1 = \theta_2 = \dots = \theta_n$$ (since the random variables are i.i.d.), and so, we can, for simplicity, just write $$L(\theta; \mathbf{Y})$$, where $$\theta$$ (notice not bolded) is a single parameter. Either way, it seems that we should have $$L( \cdot ; \mathbf{Y})$$, where $$\mathbf{Y}$$ is a random matrix (matrix of random variables), no?

## EDIT3

According to here, I am incorrect in EDIT2. It seems that I am getting confused and forgetting that a random variable only produces a single observation. A random vector is actually a vector of random variables, each of which produce a single realization/observation, which is why a random vector produces a vector of observations. So, if $$Y_1, \dots, Y_n$$ are random variables, then their random vector is $$\mathbf{Y} = (Y_1, \dots, Y_n)$$, where $$Y_1$$ then has the observation $$y_1$$ and $$Y_n$$ then has the observation $$y_n$$. This matches with the use of $$Y_1, \dots, Y_n$$ and $$y_1, \dots, y_n$$ in the above definition of likelihood.

• Could you explain what you might mean by "... has its own parameter"? Bear in mind that if a different parameter (or parameter value) governs the likelihood of each $y_i$ separately, you have no hope of estimating any of the parameters. Doesn't your Normal example at the end clarify -- and even answer -- your question?
– whuber
Commented Apr 17, 2021 at 18:51
• @whuber So, for instance, for $(y_1, \dots, y_n)$, we could have $(\theta_1, \dots, \theta_n)$, where $\theta_1$ is the parameter for $y_1$, $\theta_2$ is the parameter for $y_2$, and so on. Really, I'm just trying to make sure that I understand the language and concept here precisely. For instance, basically, if I write $L(\theta; \mathbf{y})$, we see that the $y$ is bolded to $\mathbf{y}$, since it is a vector, but this question relates to whether we should also have the $\theta$ bolded to $\mathbf{\theta}$. So I'm also trying to ensure that my notation is precisely correct. Commented Apr 17, 2021 at 18:55

It depends on the nature of the assumptions on your $$n$$ random variables $$Y_1, ..., Y_n$$, which in turn will influence the kind of parametric statistical model that is specified.

Here is a coin flipping example in context of maximum likelihood estimation that might be useful.

Suppose you have a single biased coin, and you flip it $$n$$ times, with a view to establishing how "biased" it is. You could do this by flipping it $$n$$ times, and recording the outcomes, tallying the fraction of heads recorded.

Formally, this situation could be modelled by assuming $$n$$ independent and identically distributed Bernoulli random variables, where $$Y_i \sim \text{Bernoulli}(p)$$. The above can be viewed from the perspective of parametric point estimation, whereby the statistical model is the family of Bernoulli distributions indexed by the scalar parameter $$p$$. Tallying the fraction of heads amounts to computing the scalar maximum likelihood estimator

$$\hat{p} = \frac{1}{n} \sum^n_{i=1} y_i$$

of the single scalar parameter $$p$$, which will maximise the likelihood

$$L(p; y_1, ..., y_n) = \prod^n_{i=1} p^{y_i} (1-p)^{1- y_i} =p^{\sum_i y_i} (1 - p)^{n - \sum_i y_i}$$

Now consider the somewhat perverse case where you have $$n$$ biased coins, each with their own degree of bias. You may only flip each of the $$n$$ separate coins once, for a total of $$n$$ flips altogether. Even though the situation is somewhat more restrictive in that you can only flip each biased coin once, you wish to continue using a principled method for ascertaining the bias of each individual coin.

Formally, this situation might be modelled by using $$n$$ independent random variables, $$X_1, ..., X_n$$ but the context does not allow for one to reasonably assume that the $$X_i$$ are identically distributed. In this case, one might say that $$X_i \sim \text{Bernoulli}(p_i)$$, where each coin's individual bias is parametrised by an individual indexed parameter $$p_i$$. In this case, we are postulating $$n$$ statistical models, that is, $$n$$ families of Bernoulli distributions each indexed by $$p_i$$. We could then view the maximum likelihood estimator as the vector

$$\hat{\mathbf{p}} = [\hat{p_1}, ..., \hat{p}_n]$$

where each of the $$n$$ individual maximum likelihood estimators is just the single observation of the respective biased coin $$\hat{p}_i = x_i$$. You will find that $$\hat{\mathbf{p}}$$ in this case maximises the likelihood

$$L(\mathbf{p}; x_1, ..., x_n) = \prod^n_{i=1}p_i^{x_i} (1 - p_i)^{1 - x_i}$$

• Yes, this gets to my point! Thanks for the clarification! Commented Apr 17, 2021 at 20:10

If you have a model with a single parameter, then you are looking to find the single parameter, if it has multiple parameter, you are looking to find multiple parameters. With maximum likelihood, you find the parameters by maximizing the likelihood that is a function of the parameters. So it can be a function of one or more parameters, depending on the particular problem you are trying to solve.

As about notation, there are many notation traditions, with using italics, bolding, using uppercase, decorating etc different things, there’s no single agreed notation. So some authors would use bold font there, some not.