2
$\begingroup$

I've read through the most popular threads concerning confounding variables, but I haven't been able to find an answer to my specific question. Sorry for the wall of text, I hope it's clear enough. Thanks in advance for your time!

I'm attempting to build a predictive multiple regression model for $weight$ using a dataset with a large number of (mostly categorical) variables.

During my exploratory analysis I stumbled across the following problem. A simple scatter plot of my response vs $age$ revealed that there was no simple linear relation between these two variables. In fact, it seems like it would be possible to fit a straight line with a positive slope through the observations below a certain age. For older observations, there does not seem to be a relation between $age$ and $weight$. Of course this makes sense if you consider the fact that human beings stop growing after a certain age.

My question is: will it always be possible to perform a transformation on my response/predictors, so that I can fit a linear model on my data? Is there a boxcox-like procedure for predictor variables? I've attempted a log, square root and even cubed root transformation of age, but none of those seem to be the magic fix.

If I cannot find a useful transformation, does this then indicate that there is a confounding variable I should include in the model (or subset my data for)? There are plenty of examples on how adding a categorical variable (e.g. gender) could reveal a hidden association. In my case however, one possible categorical variable would simply be 'adult vs child'. Would it make sense to use an age category (adult/child) as a categorical factor in a model where $age$ is also my continuous predictor? While it would be strange to use the same variable twice (in a sense), this would allow me to properly capture the relationship I'm seeing.

Simple regression models of my other predictor variables were often only significant when performed on a certain age group. If I cannot fit an adequate curve on $age$, I will not be able to account for these differences between adults and children in my final multiple regression model, unless I use the categorical 'adult-child' variable. I would however lose out on the positive relation between $weight$ and $age$ (continuous) for children.

EDIT: Some additional information on why I believe I need to transform $age$: while the simple linear regression model of $weight$ ~ $age$ is significant, it has a low $R^2$ and the residual plot shows a parabola-like pattern.

Improve this question
$\endgroup$
3
  • $\begingroup$ Quick note 1: if you want a predictive model you should probably try to optimise some prediction-oriented criterion, like out-of-sample, or at least cross-validated error, instead of $R^2$. $\endgroup$
    – conjugateprior
    Commented Dec 10, 2014 at 15:57
  • $\begingroup$ Quick note 2: 'confounding', is a causal concept and doesn't make sense for purely predictive models. So don't worry about it. Throw in whatever works best according to your predictive criterion. $\endgroup$
    – conjugateprior
    Commented Dec 10, 2014 at 15:59
  • $\begingroup$ There is a detailed description of approaches to this type of problem at stats.stackexchange.com/questions/35711/… $\endgroup$
    – EdM
    Commented Dec 10, 2014 at 17:03

1 Answer 1

Reset to default
0
$\begingroup$

There's no fundamental problem with including an indicator variable for being an adult - that would simply allow the intercept to be different for children and adults. If you want the slope to be allowed to differ too, you could add an interaction term between age and the adult indicator. However, this approach requires you to select an age cutoff for being an adult. I guess you could do something like a grid search over different age cutoffs, and select the one that maximizes $R^2$. It also imposes a discontinuity in the slope of the age variable, and nature is usually not discontinuous.

Have you tried simply adding a polynomial in age? I would try putting both $age$ and $age^2$ in the regression, but you can also try adding higher-order terms. I would be surprised if a quadratic term wouldn't fit this data fairly well.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for the suggestion! I tried adding $age^2$ and it turned out to be a significant predictor. $R^2$ also increased quite a bit (although the comment by conjugateprior informed me this might not be the best criterion for my purposes). I'm a bit worried that this will make it harder to interpret interactions between $age$ and other predictors though. For example, I know that the effect of $gender$ differs between adults and children (modelled as an interaction effect $gender * categorical-age$). For the continuous case, only $gender * age^2$, and not $gender*age$, seems to be significant. $\endgroup$
    – Zenit
    Commented Dec 10, 2014 at 18:59
  • $\begingroup$ As conjugateprior implied, you have to decide what the purpose of your model is. If you are interested in prediction, you shouldn't be worrying about interpreting your coefficients. And if you're interested in interpreting your coefficients, you should drop the prediction goal. $\endgroup$
    – standard_error
    Commented Dec 11, 2014 at 9:17
  • $\begingroup$ That was eye-opening, thanks! It seems my statistical knowledge is still severely lacking, as I did not even realise that the differences between predictive and explanatory models carried that much weight. I was blindly following the model building strategy described in my course notes, which did not clearly differentiate between these two options. A quick search already led me to some interesting reading material (stats.stackexchange.com/questions/1194/…). I'll take a step back for now and re-evaluate my goals. $\endgroup$
    – Zenit
    Commented Dec 12, 2014 at 18:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.