18
$\begingroup$

I have 5 variables and I'm trying to predict my target variable which must be within the range 0 to 70.

How do I use this piece of information to model my target better?

$\endgroup$
21
$\begingroup$

You don't necessarily have to do anything. It's possible the predictor will work fine. Even if the predictor extrapolates to values outside the range, possibly clamping the predictions to the range (that is, use $\max(0, \min(70, \hat{y}))$ instead of $\hat{y}$) will do well. Cross-validate the model to see whether this works.

However, the restricted range raises the possibility of a nonlinear relationship between the dependent variable ($y$) and the independent variables ($x_i$). Some additional indicators of this include:

  • Greater variation in residual values when $\hat{y}$ is in the middle of its range, compared to variation in residuals at either end of the range.

  • Theoretical reasons for specific non-linear relationships.

  • Evidence of model mis-specification (obtained in the usual ways).

  • Significance of quadratic or high-order terms in the $x_i$.

Consider a nonlinear re-expression of $y$ in case any of these conditions hold.

There are many ways to re-express $y$ to create more linear relationships with the $x_i$. For instance, any increasing function $f$ defined on the interval $[0,70]$ can be "folded" to create a symmetric increasing function via $y \to f(y) - f(70-y)$. If $f$ becomes arbitrarily large and negative as its argument approaches $0$, the folded version of $f$ will map $[0,70]$ into all the real numbers. Examples of such functions include the logarithm and any negative power. Using the logarithm is equivalent to the "logit link" recommended by @user603. Another way is to let $G$ be the inverse CDF of any probability distribution and define $f(y) = G(y/70)$. Using a Normal distribution gives the "probit" transformation.

One way to exploit families of transformations is to experiment: try a likely transformation, perform a quick regression of the transformed $y$ against the $x_i$, and test the residuals: they should appear to be independent of the predicted values of $y$ (homoscedastic and uncorrelated). These are signs of a linear relationship with the independent variables. It helps, too, if the residuals of the back-transformed predicted values tend to be small. This indicates the transformation has improved the fit. To resist the effects of outliers, use robust regression methods such as iteratively reweighted least squares.

$\endgroup$
  • 1
    $\begingroup$ +1 Great answer! Can you extrapolate on or give a citation for why "greater variation in residual values when y_hat is in the middle of its range, compared to variation in residuals at either end of the range" is an indication of non-linearity? $\endgroup$ – Andy McKenzie Jun 17 '11 at 3:44
  • 1
    $\begingroup$ @Andy In theory, such heteroscedasticity has no direct connection with nonlinearity, but in practice it is often observed that a variance-stabilizing transformation tends to linearize relationships. Any curve rising continuously from a minimum (like 0) to a maximum (like 70) will have a maximum slope somewhere in the middle of that range, often resulting in larger residual variance there too. That is why we would expect to see residuals to exhibit more variance in the middle and less at the ends. If that is not the case, we can hope for linear relationships with the untransformed variable. $\endgroup$ – whuber Jun 17 '11 at 13:18
5
$\begingroup$

It is important to consider why are your values bounded in the 0-70 range. For example, if they are the number of correct answers on a 70-question test, then you should consider models for "number of successes" variables, such as overdispersed binomial regression. Other reasons might lead you to other solutions.

$\endgroup$
2
$\begingroup$

Data transformation: rescale your data to lie in $[0,1]$ and model it using a glm model with a logit link.

Edit: When you re-scale a vector (ie divide all the elements by the largest entry), as a rule, before you do that, screen (eyeballs) for outliers.

UPDATE

Assuming you have access to R, i would carry the modeling part with a robust glm routine, see $\verb+glmrob()+$ in package $\verb+robustbase+$.

$\endgroup$
  • 3
    $\begingroup$ Clamping the data as recommended here will bias the slopes in a regression. $\endgroup$ – whuber Jun 16 '11 at 15:59
  • 1
    $\begingroup$ Also, I do not see the immediate value in clamping based on sample quantiles, when the true range of the data is known a priori. $\endgroup$ – cardinal Jun 16 '11 at 18:03
  • $\begingroup$ @Cardinal The point is that (e.g.) possibly 99% of the data lie in [0,1] and the remaining values equal 70: a compact constraint on the range does not assure absence of outliers! Therefore I agree with the spirit of the advice offered by @user603, despite my concern about the possible bias in the proposed approach. $\endgroup$ – whuber Jun 16 '11 at 19:24
  • $\begingroup$ @whuber: My inclination in such a setting would be to use a GLM that was resistant to outliers rather than this form of clamping. Then let the model fit adjust via the "intercept" and the "slope" coefficient. $\endgroup$ – cardinal Jun 16 '11 at 19:51
  • $\begingroup$ @Cardinal Yes, that's a valid solution. I hope the use of such a GLM would still be accompanied by diagnostic procedures to check for (approximate) linearity and independence of residuals. $\endgroup$ – whuber Jun 16 '11 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.