Consider the following population regression model:

$$y_{i} = \beta _{1} + \beta_{2}x_{i} + \epsilon _{i},$$

where $i=1,...,n$. Assume $\epsilon \sim iid$, with the pdf in equation: $f(\epsilon ) = \alpha \epsilon$ for $0\leq \epsilon \leq 1$.

For $n=4$, we have $f(\epsilon _{1},\epsilon _{2},\epsilon _{3},\epsilon _{4}) = f_{1}(\epsilon _{1}) f_{2}(\epsilon _{2}) f_{3}(\epsilon _{3}) f_{4}(\epsilon _{4}) = \alpha \prod _{i=1}^{4} \epsilon _{i}$. This is because of no autocorrelation in errors.

Now we substitute $f(\epsilon _{1},\epsilon _{2},\epsilon _{3},\epsilon _{4})$ in the regression model and obtain

$$y_{i} = \beta _{1} + \beta_{2}x_{i}+\prod _{i=1}^{4} \epsilon _{i}$$.

Taking the natural logs, we obtain

$$ln(y_{i}) = ln(\beta _{1} + \beta_{2}x_{i}+\prod _{i=1}^{4} \epsilon _{i})$$.

What is a common name for the resulting function?

I am guessing it is the maximum log likelihood function. Am I correct?

  • 1
    $\begingroup$ Are you sure you're typing the question right? $\endgroup$ – wolfsatthedoor Nov 5 '14 at 23:34
  • 1
    $\begingroup$ Much of this makes no sense. You seem to be trying to write down a likelihood, but do not succeed due to a sequence of errors. Asking for the name of the resulting (meaningless) expression seems beside the point. What are you really trying to accomplish? $\endgroup$ – whuber Nov 5 '14 at 23:57

Sorry to say, but this is a totally confused mix up.

First, it appears that $f(\epsilon)$ is a probability density function. If this is the case it must intagrate to unity, over the specified domain of the variable. So we must have

$$\int_0^1 \alpha \epsilon d\epsilon = 1 \Rightarrow \frac{\alpha}{2}\epsilon^2|_0^1 = 1 \Rightarrow \alpha =2$$

Second, we do not insert the joint density of the four independent errors (they must be independent, not just non-autocorrelated) into the functional specification of the regression. What we do to arrive at the maximum likelihood estimator is to note that from the regression specification we obtain by solving for $\epsilon$,

$$y_{i} - \beta _{1} - \beta_{2}x_{i} = \epsilon _{i}$$

and substitute this into the joint probability density

$$L = \prod_{i=1}^4I_{\{\epsilon_i \in [0,1]\}}\cdot f(\epsilon_i) = \prod_{i=1}^4\alpha I_{\{y_{i} - \beta _{1} - \beta_{2}x_{i} \in [0,1]\}} \cdot \left(y_{i} - \beta _{1} - \beta_{2}x_{i}\right)$$

Note the inclusion of the indicator function that expresses the restriction on the support of $\epsilon_i$.

The log-likelihood is

$$\ln L = 4\ln2 + \sum_{i=1}^4\ln\left(y_{i} - \beta _{1} - \beta_{2}x_{i}\right) + \sum_{i=1}^4\ln I_{\{y_{i} - \beta _{1} - \beta_{2}x_{i} \in [0,1]\}}$$

The last term is translated into $2n$ inequality restrictions when we move to solve the above maximization problem with respect to the parameters, of the form

$$y_{i} - \beta _{1} - \beta_{2}x_{i} \geq 0,\;\;\; y_{i} - \beta _{1} - \beta_{2}x_{i} \leq 1$$ for each $i$.

  • $\begingroup$ Thanks. I guess I didn't understand the instructions properly. $\endgroup$ – OGC Nov 6 '14 at 0:09
  • 1
    $\begingroup$ Unfortunately, this version of the log likelihood is also incorrect. It is essential to incorporate the fact that the distribution is supported on the interval $[0,1]$. $\endgroup$ – whuber Nov 6 '14 at 0:22
  • 1
    $\begingroup$ @whuber I will update after I finish another answer I am writing, but you know as well as I do that we tend to consider the support as "understood" (and it is not the only bad practice we follow). $\endgroup$ – Alecos Papadopoulos Nov 6 '14 at 0:32
  • $\begingroup$ @AlecosPapadopoulos I have questions about the same model, which you might be interested in taking a look stats.stackexchange.com/questions/122860/…. Again thanks for the update. $\endgroup$ – OGC Nov 6 '14 at 1:23
  • $\begingroup$ @Alecos I'm not trying to be picky here. I pointed out that error because it has been the cause of many posts on this site concerning problems with ML. If you consider what might happen were someone to implement your (original) formula in software, you would immediately see the difficulty: it may try to take logarithms of negative numbers and, if it overcomes that difficulty, it will eventually produce near-infinite estimates of the coefficients. $\endgroup$ – whuber Nov 6 '14 at 14:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.