2
$\begingroup$

I'm trying to compute an estimate for the variance of the estimated coefficients in a non-linear regression using the formula described in link. I can't figure out how to build $F_{ij}$

Let's consider for simplicity a quadratic regression:

$f = \beta_0 + \beta_1 x + \beta_2 x^2$

and let's say we have computed the estimate $\hat{\beta}_0, \hat{\beta}_1, \hat{\beta}_2$ from a set of $m$ observations.

To me $$ F = \frac{\partial f(x_i)}{\partial \beta_j} = \left( \begin{array}{ccc} 1 & x_1 & x_1^2 \\ \vdots & \vdots &\vdots \\ 1 & x_m & x_m^2 \end{array} \right) $$

But of course it can't be cause it does not depend on $\hat{\beta}$ anymore...

$\endgroup$
1
  • 2
    $\begingroup$ This is not a non-linear regression. "Linearity" and non-linearity refer to how thw estimated unknown parameters enter the equation, not to whether there appear regressors that are non-linear transformations of other regressors. Set $x^2 \equiv z$ and re-write the equation. The estimator variance is the usual linear-regression one. $\endgroup$ Nov 20, 2014 at 0:11

1 Answer 1

1
$\begingroup$

The Matrix $F$ is actually correct and one can use it to build the covariance matrix of the regression coefficients: $$D(\hat{\beta}) = \sigma^2(F^T F)^{-1}$$ Now denoting with $d_{jj}$ the $j-th$ diagonal element of $D(\hat{\beta})$ we can specify the distribution of $\hat{\beta}_j$ as $$ \hat{\beta}_j \sim N(\beta_j, \sigma^2 d_{jj}) $$ where $\sigma^2$ can be estimated using the sum of squares of the residuals: $$ \hat{\sigma}^2 = \frac{1}{m - n} \sum_{i=1}^m \left( y_i - \hat{y}_i\right)$$

$\endgroup$
1
  • $\begingroup$ The normality result depends on assuming that the error term follows a normal distribution. This is not necessary for the derivation of the variance of the estimator. $\endgroup$ Nov 20, 2014 at 2:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.