2
$\begingroup$

I am using an OLS regression to fit a model to some data. The estimated response is given by the usual

$$\mathbf{\hat{y}} = \mathbf{Py} = \mathbf{X}(\mathbf{X}^T\mathbf{X})^{-1}\mathbf{X}^T\mathbf{y}$$

If the design matrix $\mathbf{X}$ contains a column of $1s$ for the intercept term, then each row (and each column) of the projection matrix $\mathbf{P}$ sums to $1$. This means that the mean of $\mathbf{\hat{y}}$ is equal to the true mean of $\mathbf{y}$, i.e.

$$\mathbf{1}^T\mathbf{\hat{y}} = \mathbf{1}^T\mathbf{Py} = \mathbf{1}^T\mathbf{y}$$

The issue I face is that $\mathbf{\hat{y}}$ may have some values falling outside some allowed region (say, negative values), so I would like to apply a transformation to $\mathbf{\hat{y}}$ such that the mean is maintained but the values are all positive.

$\endgroup$
  • 1
    $\begingroup$ When you face this issue, you should seriously contemplate using a different model. If you "transform" $\hat y,$ then you risk losing every property of OLS for which it might have been indicated: it will not longer be unbiased among linear estimators; the p-values will be wrong; and you cannot interpret the coefficients in the standard way. $\endgroup$ – whuber Nov 2 '18 at 19:41
  • $\begingroup$ @whuber Thank you for taking time to read my post and your comment. Unfortunately, I often face constraints imposed externally (such as the use of OLS in this case) which cannot be overcome so easily and I have to come up with (at times "crude") "fixes". This is one of them. $\endgroup$ – Confounded Nov 2 '18 at 22:27
  • 1
    $\begingroup$ If you can do weighted least squares (and re-calculate weights based on the fit to re-fit) then you could replicate (for example) GLMs and nonlinear least squares using linear least squares and so end up using more suitable models in spite of the restriction. $\endgroup$ – Glen_b Nov 3 '18 at 2:34
  • $\begingroup$ @Glen_b Thank you for your comment. OLS is all I have to work with at the moment, but I would be interested in exploring your suggestions further. As far as I know, iteratively reweighted least squares is used when the errors are heteroscedastic, but I might be confusing it. Would you be able to point me to a resource / example of applying the technique? Thank you $\endgroup$ – Confounded Nov 3 '18 at 9:36
  • 1
    $\begingroup$ 1. You can do weighted LS using unweighted LS. 2. IRLS is used for much more than just heteroskedastic errors.See, for example, slide 14 here which shows how to set up the equations in the case of GLMs. Full details are in many places (I think including the classic McCullagh & Nelder book n GLMs but I don't have it to hand right now). $\endgroup$ – Glen_b Nov 3 '18 at 10:04
2
$\begingroup$

Let $Y_i^n$ be new $\hat Y_i$ value after transformation.

$$Y_i^n = \frac {\hat Y_i-\bar {\hat Y}}{\bar {\hat Y}- \min(\hat Y_1,...,\hat Y_n)}\bar{\hat Y} + \bar {\hat Y}$$

Then $Y_i^n \ge 0$ and $\bar Y^n = \bar {\hat Y}$ under the condition that $\bar {\hat Y} > 0$. Of course, if $\bar{\hat Y} \le 0 $, there is no solution.

$\endgroup$
  • $\begingroup$ What connection does your "$\bar Y$" have with the $\hat y$ of the question? $\endgroup$ – whuber Nov 2 '18 at 19:41
  • $\begingroup$ modified to reflect $\hat Y$. $\endgroup$ – user158565 Nov 2 '18 at 19:47
  • $\begingroup$ Okay, thank you. It makes one wonder what the resulting value would accomplish. How would it be interpreted? What would be the properties of this estimator? It's obviously at risk of being an extremely poor predictor! $\endgroup$ – whuber Nov 2 '18 at 19:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.