I am a bit confused at how can i find the likelihood function and the solutions of likelihood function for a line equation, for example y=3x+15


The equation $y = 0.2x + 4$ is not a statistical model, but a mathematical equation. It doesn't even have parameters (unless you mean that $x$ is a parameter), so there is nothing to optimize. In case you meant $y = \beta_0 + \beta_1 x$, this would still be a deterministic function of the variables, that can be solved by pure algebra, that does not need statistics. Statistical model arises, when there is uncertanity about the data

$$ y = \beta_0 + \beta_1 x \color{red}{+ \varepsilon} $$

Assuming that by linear regression you mean the standard model minimizing the squared error (e.g. with ordinary least squares), then the objective function to minimize is

$$ \operatorname{SSE}(y, \hat y) = \sum_{i=1}^n (y_i - \hat y_i)^2 $$

To obtain a likelihood function, you would need a probability distribution. Usually we maximize the log-likelihood (or minimize the negative log-likelihood), rather then pure likelihood. The distribution that is equivalent to minimizing squared errors would be

$$ \log p(y) \propto \sum_{i=1}^n -\frac{(y_i - \hat y_i)^2}{\sigma^2} $$

where the proportionality ($\propto$) is due the fact, that for optimization we can drop the normalizing constant (the distribution does not have to integrate to one). As you can see, we introduced a new parameter for $\sigma^2$ for the variance of the residuals. If you take $\exp$ of it, to reverse the $\log$ we took, and get rid of the negative sign, this is a normal distribution

$$ p(y) \propto \exp\Big(-\frac{(y-\mu)^2}{2\sigma^2}\Big) $$

So, minimizing sum of squared errors is equivalent to maximizing the Gaussian likelihood. The above is a different way of saying, that the linear regression model assumes normally distributed noise $\varepsilon$ that has mean of zero,

$$\begin{align} y &= \beta_0 + \beta_1 x + \varepsilon \\ \varepsilon &\sim \mathcal{N}(0, \sigma^2) \end{align}$$

| cite | improve this answer | |
  • $\begingroup$ i wrote it as (1/[σ^2(2π)^n/2)] * exp[( -1/2σ^2)*Σ(yi-4-0.2xi)^2. is it false? $\endgroup$ – Sarah kenwich Oct 22 '19 at 6:36
  • $\begingroup$ @Sarahkenwich it is the same, but includes normalizing constant (not needed) and in re-arranged order. $\endgroup$ – Tim Oct 22 '19 at 6:37
  • $\begingroup$ in the way i wrote it, do i have to calculate the σ or should i let it as ''σ''? $\endgroup$ – Sarah kenwich Oct 22 '19 at 6:39
  • $\begingroup$ if i want for the same equation to justify why the solutions to the likelihood would not be the same if the linea regression model was fitted to two distinct dataset,would be enough if i say that they will not be the same, because evaluation of likelihood function is dependent on the data or do i have to write something more to justify it? thanks in advance $\endgroup$ – Sarah kenwich Oct 22 '19 at 6:42
  • $\begingroup$ @Sarahkenwich yes, it depends on data, so different datasets may lead to different results. As about $\sigma^2$, it is just another parameter to find via optimization. $\endgroup$ – Tim Oct 22 '19 at 6:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.