# Calculatation of Lagrange multiplier LM test for nonlinear regression model

I have the following non-linear model

$(y_t -X_t\alpha)^a=u_t$ where $u_t$ ~ $N(0,\sigma^2)$

Assume taht you estimated this model consitional on $a=1$.

Calculate an Lagrange Multiplier LM test which you could use to test this restriction without estimating the unrestricted model.

My attempt is as follows:

Model: $(y_t -X_t\alpha)^a=u_t$ where $u_t$ ~ $N(0,\sigma^2)$

The null hypothesis is $$H_0 : a=1$$

Log likelihood function is

$\ell=-n/2 ln(2\pi)- n/2 ln(\sigma^2)-1/2\sigma^2 [(y-X\alpha)^a]'[(y-X\alpha)^a]$

$\theta=(\alpha, a, \sigma^2)$

FOC differentiate it w.r.t. a

$g(a)=-\frac{1}{\sigma^2}[(y-X\alpha)^a]ln(y-X\alpha)$

Calculate the restricted estimate of a

$g(\tilde a)=-\frac{1}{\sigma^2}[(y-X\alpha)]ln(y-X\alpha)=0$

Now, I need to calculate information matrix $I(\tilde a)$

And then I need to find its inverse of this information matrix, $I(\tilde a)^{-1}$

And finally $LM=g(\tilde a) I(\tilde a)^{-1} g(\tilde a)$

I am not sure about my all solution. But my attempt is like that. And I stack at the point to calculate information matrix $I(\tilde a)$ and after that.

If what I did is correct, how can I calculate lm test, or if it is totally wrong, how can I solve it?

Thank you for all helps.

I am sorry to have to say that this problem may well not have a solution.

First, note that with the LR test, you're going to estimate the parameters of the restricted model, so there is no need to include $a$ in the parameters to be estimated. I'll redefine your parameter $\alpha$ to be $\beta$ so as to make the difference between it and the parameter of interest $a$ a little clearer to the eye. I'll also redefine $a$ as $1/\text{your}$ $a$. Given the Normality assumption, the MLE of $(\beta, \sigma^2)$ is found through estimation of the linear regression $y = X\beta$.

Now for the test. The relevant part of the log likelihood function - dropping all terms that aren't functions of $a$ - is:

$$l = -\frac{\Sigma e_t^{2a}}{2\hat{\sigma}^2}$$

The first and second derivatives w.r.t. $a$ are:

$$U(a) = -\frac{\Sigma e_t^{2a}\log e_t}{\hat{\sigma}^2}$$

and

$$U'(a)= -\frac{2\Sigma e_t^{2a}\log^2 e_t}{\hat{\sigma}^2}$$

... and now, the sticking point should be abundantly clear. How are you going to deal with the $\log$ of negative $e_t$?

The fundamental problem is that the MLE of $a$ doesn't exist except at rational values of $a$, worse yet only ones with odd denominators, because raising negative numbers / errors to fractional powers only gives real results for rational powers with odd denominators. (See https://math.stackexchange.com/questions/317528/how-do-you-compute-negative-numbers-to-fractional-powers for an explanation of this.) This breaks a lot of calculus: https://math.stackexchange.com/questions/1880741/why-cant-calculus-be-done-on-the-rational-numbers. With reference to your question, it breaks the part of calculus which is used to derive the LR test.

It may be that you can restate your problem so that the errors follow a symmetric stable distribution (https://en.wikipedia.org/wiki/Stable_distribution), but this limits you to having (your) $a \leq 1$. Another alternative might be to have $|u_t|^a \sim$ some distribution. You'd have to be careful when taking an expectation that has $\log |u_t|$ somewhere in it; for a lot of distributions, such an expectation would be infinite. It may be there is help to be found in nonstandard analysis, but that is well outside my set of mathematical expertise $> \epsilon$.

Edit in response to questions in comments:

If, somehow, you know $a$, you obviously don't have to do a test. Estimation of $\beta$ can proceed in any of several ways, but, basically, you'll have to transform the errors $e_t$ from the linear model $y - X\beta = e$ to the errors $u_t = e_t^{1/a}$. This is easily done using the relationship $u_t = \text{sign}(e_t)|e_t|^{1/a}$. You can implement the estimation as a straightforward nonlinear optimization problem using any of a number of solvers in R, Python, or your language of choice.

• I have one more question that I want to learn. How can I estimate model if I know a? And also I don't know a? – studying1234 Nov 18 '17 at 4:01
• Also, if the derivative of this log likelihood function was not negative, would my solution way be correct? – studying1234 Nov 18 '17 at 4:13
• And lastly, please can you expand in a bit details your answer by restating that the errors follow a symmetric stable distribution. I really want to learn. Really interesting question. Thank you so much for helps:) – studying1234 Nov 18 '17 at 4:18
• It's not that the derivative of the log likelihood is negative, it's that you have negative errors that you are trying to raise to the $a$ power, and you can only do this if $a$ is rational. I'll expand the answer later on to address your other two questions. – jbowman Nov 18 '17 at 16:27
• Okay I am waiting for expanding this question. Than you so much. – studying1234 Nov 18 '17 at 17:38