0
$\begingroup$

I am running a weighted least-squares regression (where all weights are strictly positive), where my dependent variable is a cross-section of variance values.

Since variance is always positive (>=0), I would expect my fitted values to be positive as well, for them to be meaningful,

$\sigma^2_i = \alpha + \beta * x_i + \epsilon _i$

The problem is that I am getting some negative fitted values:

$\hat{\sigma}^2_i = \hat{\alpha} + \hat{\beta} * x_i $ for some $i$

Is there any suggestion as to how to constraint the predicted values to be positive?

Thanks!

Improve this question
$\endgroup$
  • $\begingroup$ You could regress $\log \sigma_i^2$, and then take the exponential of the prediction. $\endgroup$ – Arthur B. Nov 14 '14 at 15:24
  • $\begingroup$ I am not quite sure how that would solve the issue: you mean use $log\sigma^2_i)$ as my dependent variable, but keep the explanatory variable without log transformation? $\endgroup$ – Mayou Nov 14 '14 at 16:04
  • $\begingroup$ yes, I'll write an answer with more details $\endgroup$ – Arthur B. Nov 14 '14 at 16:14
1
$\begingroup$

Transform your dependent variable $\sigma^2$ with a logarithm and fit the following model

$$\log \sigma_i^2 = \alpha + \beta x_i + \epsilon_i$$

Get an estimate of the variance $\eta^2$ of the residuals as

$$\hat{\eta^2} = \frac{1}{N}\sum_{i=1}^N (\log \sigma_i^2 - \alpha - \beta x_i)^2$$

Finally, use the estimator

$$\hat{\sigma_i^2} = \exp\left(\hat{\alpha} + \hat{\beta} x_i + \frac{\hat{\eta^2}}{2}\right)$$

The reason there is a $\frac{1}{2}\hat{\eta^2}$ term is because if $\epsilon$ is normally distributed with mean 0, $e^\epsilon$ has expectation $e^{\eta^2/2}$.

Since $\hat{\sigma_i^2 }$ is an exponential, it will always be positive.

Improve this answer
$\endgroup$
  • $\begingroup$ Thanks for the thorough explanation. But how can that extra term be estimated? $\endgroup$ – Mayou Nov 14 '14 at 16:24
  • 1
    $\begingroup$ I'll edit the explanation to incorporate that $\endgroup$ – Arthur B. Nov 14 '14 at 16:25
  • $\begingroup$ Thank you very much. Is this what is called a GLS with log link function? Also, I know the expression of $\hat{\beta}$ in the case of a weighted-least squares, but once I add the log transformation, I don't know how to adjust the \beta expression. $\endgroup$ – Mayou Nov 14 '14 at 16:26
  • 1
    $\begingroup$ I'm not sure what it's called. The log transformation doesn't change anything about how you'd go about estimating beta or alpha, it's a linear model, but for the log of the $\sigma^2$ $\endgroup$ – Arthur B. Nov 14 '14 at 16:32
  • $\begingroup$ Can I do this transformation even if this is a WEIGHTED LEAST SQUARES regression, not a simple OLS? If so, how should the above formulas be adjusted to account for the weights? $\endgroup$ – Mayou Nov 14 '14 at 22:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.