Weighted least-squares negative fitted values

I am running a weighted least-squares regression (where all weights are strictly positive), where my dependent variable is a cross-section of variance values.

Since variance is always positive (>=0), I would expect my fitted values to be positive as well, for them to be meaningful,

$\sigma^2_i = \alpha + \beta * x_i + \epsilon _i$

The problem is that I am getting some negative fitted values:

$\hat{\sigma}^2_i = \hat{\alpha} + \hat{\beta} * x_i$ for some $i$

Is there any suggestion as to how to constraint the predicted values to be positive?

Thanks!

• You could regress $\log \sigma_i^2$, and then take the exponential of the prediction. – Arthur B. Nov 14 '14 at 15:24
• I am not quite sure how that would solve the issue: you mean use $log\sigma^2_i)$ as my dependent variable, but keep the explanatory variable without log transformation? – Mayou Nov 14 '14 at 16:04
• yes, I'll write an answer with more details – Arthur B. Nov 14 '14 at 16:14

Transform your dependent variable $\sigma^2$ with a logarithm and fit the following model

$$\log \sigma_i^2 = \alpha + \beta x_i + \epsilon_i$$

Get an estimate of the variance $\eta^2$ of the residuals as

$$\hat{\eta^2} = \frac{1}{N}\sum_{i=1}^N (\log \sigma_i^2 - \alpha - \beta x_i)^2$$

Finally, use the estimator

$$\hat{\sigma_i^2} = \exp\left(\hat{\alpha} + \hat{\beta} x_i + \frac{\hat{\eta^2}}{2}\right)$$

The reason there is a $\frac{1}{2}\hat{\eta^2}$ term is because if $\epsilon$ is normally distributed with mean 0, $e^\epsilon$ has expectation $e^{\eta^2/2}$.

Since $\hat{\sigma_i^2 }$ is an exponential, it will always be positive.

• Thanks for the thorough explanation. But how can that extra term be estimated? – Mayou Nov 14 '14 at 16:24
• I'll edit the explanation to incorporate that – Arthur B. Nov 14 '14 at 16:25
• Thank you very much. Is this what is called a GLS with log link function? Also, I know the expression of $\hat{\beta}$ in the case of a weighted-least squares, but once I add the log transformation, I don't know how to adjust the \beta expression. – Mayou Nov 14 '14 at 16:26
• I'm not sure what it's called. The log transformation doesn't change anything about how you'd go about estimating beta or alpha, it's a linear model, but for the log of the $\sigma^2$ – Arthur B. Nov 14 '14 at 16:32
• Can I do this transformation even if this is a WEIGHTED LEAST SQUARES regression, not a simple OLS? If so, how should the above formulas be adjusted to account for the weights? – Mayou Nov 14 '14 at 22:18