1
$\begingroup$

I am considering the linear regression model

\begin{equation} Y_{i} = A t_{i} + B + \varepsilon_{i} \tag{$1$} \end{equation}

where $A$ and $B$ are unknown real parameters and $(\varepsilon_{i})_{1 \leq i \leq n}$ are observed values of independent and identically distributed gaussian random variables with mean $0$ and variance $\sigma^{2}$ ($\sigma^{2}$ is unknown).

In matrix form, $(1)$ writes :

\begin{equation} Y=X\alpha + \varepsilon \tag{2} \end{equation}

where $X = \begin{bmatrix} t_{1} & 1 \\ \vdots & \vdots \\ t_{n} & 1 \end{bmatrix}$, $\alpha = \begin{bmatrix} A \\ B \end{bmatrix}$ and $\varepsilon \sim \mathcal{N}(0,\sigma^{2}\mathrm{I}_{n})$.

Using Matlab, I generated some data artificially : I took $n=100$, $A=3$, $B=-1$ and $\sigma^{2} = 0.64$ and the parameters estimates I obtain are : $\widehat{A}=3.0096$, $\widehat{B}=-0.9260$ and $\widehat{\sigma^{2}}=0.6337$.

The log-likelihood of $(2)$ is the function

$$ \ell(A,B,\sigma^{2}) = -n \log(\sigma) - \frac{1}{2\sigma^{2}} \Vert Y-X\alpha \Vert^{2} $$

This may be a silly question but I do not understand why the true values of the parameters do not maximize the log-liklihood in my case. Is it because the estimates I obtained using Matlab are observed values of random variables which asymptotically converge to the true values of the parameters and here, $n$ is not "big enough" ?

$\endgroup$

2 Answers 2

2
$\begingroup$

The data you simulated is not exactly equal to the true data generating process, i.e. it might be that the specific draw you did in terms of the error terms has a mean of say 0.0002 and a variance of 0.63. If you simulated a larger and larger data set you should get the true values.

$\endgroup$
2
$\begingroup$

When you calculate MLEs you maximize the likelihood for the observed data - the parameter estimate that will maximize the likelihood is what best fits the data, but the data is random; it's best fit is not at the population value.

So for example, consider MLE for $\mu$ in a $N(\mu,\sigma^2)$ is the sample mean. The sample mean is 'closer to the data' than the population mean (making the likelihood higher), specifically because that value is the one that's as close to the data as possible; any value other than the sample mean must be less likely.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.