Skip to main content
I have elaborated on the earlier very succinct comments, added references, and improved the formatting.
Source Link

linearLinear relations between each of the explanatory variables and the dependent variable will ensure also linear relations between the explanatory variables. The reverse is not of course true.

It is true that transformation(s) designed to give approximate linearity will increase collinearity. In the absence of such transformation(s), however, the collinearity is hidden. Insisting on keeping the collinearlity thus hidden can result in a complicated and uninterpretable regression equation, where a simple form of equation is available. 

Suppose also that yy is close to a linear function of log(x1)log(x1), in a case where x1x ranges over several powersvalues that differ by a factor of 10 or more. Then if x1x is used as a regressor, other explanatory variables will if at all possible be invoked to account for the nonlinearity in the relationship with x1. The result may be a very complicated regression relationship, with uninterpretable coefficients, in place of a simple form of regression equation that captures all the available explanatory power.

This can haveThe bizarre consequences, as that may result from failure to find and work with linearly related variables are well illustrated in the recent paper that claimed a femaleness of hurricane name effect in data on deaths from 94 Atlantic hurricanes that made landfall over the US over 1950-2012. If one regresses log See http://www.pnas.org/content/111/24/8782.abstract. The data are available as part of the supplementary information. Note that working with log(deaths) and using a normaL theory linear model (E[deaths]R's function lm()) is roughly equivalent to Jung et al's use of a negative binomial regression model.

If one regresses log(E[deaths]) on log(NDAM)log(NDAM), there is nothing left for the minimum pressure variable, the femaleness variable, and interactions, to explain. Regress logThe variable log(NDAM), not NDAM, appears in a scatterplot matrix as linearly related to the minimum pressure variable. Its distribution is also much less skew, much closer to symmetric.

Jung et al regressed log(E[deaths]) on NDAM (E[deaths]normalised damage) on NDAM, plus those other variables and theinteractions. The equation that emerges can bethen emerged was used to suggesttell a story in which the femaleness of the name has a large effect.

To see how bizarre it is to log(E[deaths]) on NDAMuse NDAM as an explanatory variable in a regression where the outcome variable is log(E[deaths]), plot log(deaths+0.5)log(deaths+0.5) or log(deaths+1)log(deaths+1) against NDAMNDAM. Then repeat the plot with log(NDAM) in place of NDAM. The contrast is even more striking if Katrina and Audrey, which Jung et al omitted as outliers, are included in the plot. By insisting on using NDAM as the explanatory variable, rather than log(NDAM), Jung et al passed up the opportunity to find a very simple form of regression relationship.

NB that E[deaths]E[deaths] is the number of deaths predicted by the model.

In the Jung et al data, the transformations needed can be identified from a scatterplot matrix of all variables. Try perhaps the R function spm() in the latest release of the car package for R, with transform=TRUE and (with deaths as a variable) family="yjPower". Or experiment with the transformations suggested by an initial scatterplot matrix. In general, the preferred advice may be to look first for explanatory variables that satisfy the linear predictors requirement, then attend to the outcome variable, perhaps using the car function invTranPlot().

See, in addition to "Data Analysis and Graphics Using R" that was referenced by the questioner:

  • Weisberg: Applied Linear Regression. 4th edn, Wiley 2014, pp.185-203.
  • Fox and Weisberg: An R Companion to Applied Regression. 2nd edn, Sage, 2011, pp.127-148.

linear relations between each of the explanatory variables and the dependent variable will ensure also linear relations between the explanatory variables. The reverse is not of course true.

It is true that transformation(s) designed to give approximate linearity will increase collinearity. In the absence of such transformation(s), however, the collinearity is hidden. Suppose also that y is close to a linear function of log(x1), in a case where x1 ranges over several powers of 10. Then if x1 is used as a regressor, other explanatory variables will if at all possible be invoked to account for the nonlinearity in the relationship with x1.

This can have bizarre consequences, as in the recent paper that claimed a femaleness of hurricane name effect in data on deaths from 94 Atlantic hurricanes that made landfall over the US over 1950-2012. If one regresses log(E[deaths]) on log(NDAM), there is nothing left for the minimum pressure variable, the femaleness variable, and interactions, to explain. Regress log(E[deaths]) on NDAM, and the equation that emerges can be used to suggest a story in which the femaleness of the name has a large effect.

To see how bizarre it is to log(E[deaths]) on NDAM, plot log(deaths+0.5) or log(deaths+1) against NDAM.

NB that E[deaths] is the number of deaths predicted by the model.

Linear relations between each of the explanatory variables and the dependent variable will ensure also linear relations between the explanatory variables. The reverse is not of course true.

It is true that transformation(s) designed to give approximate linearity will increase collinearity. In the absence of such transformation(s), however, the collinearity is hidden. Insisting on keeping the collinearlity thus hidden can result in a complicated and uninterpretable regression equation, where a simple form of equation is available. 

Suppose that y is close to a linear function of log(x1), in a case where x ranges over values that differ by a factor of 10 or more. Then if x is used as a regressor, other explanatory variables will if at all possible be invoked to account for the nonlinearity in the relationship with x1. The result may be a very complicated regression relationship, with uninterpretable coefficients, in place of a simple form of regression equation that captures all the available explanatory power.

The bizarre consequences that may result from failure to find and work with linearly related variables are well illustrated in the recent paper that claimed a femaleness of hurricane name effect in data on deaths from 94 Atlantic hurricanes that made landfall over the US over 1950-2012. See http://www.pnas.org/content/111/24/8782.abstract. The data are available as part of the supplementary information. Note that working with log(deaths) and using a normaL theory linear model (R's function lm()) is roughly equivalent to Jung et al's use of a negative binomial regression model.

If one regresses log(E[deaths]) on log(NDAM), there is nothing left for the minimum pressure variable, the femaleness variable, and interactions, to explain. The variable log(NDAM), not NDAM, appears in a scatterplot matrix as linearly related to the minimum pressure variable. Its distribution is also much less skew, much closer to symmetric.

Jung et al regressed log(E[deaths]) on NDAM (normalised damage), plus those other variables and interactions. The equation that then emerged was used to tell a story in which the femaleness of the name has a large effect.

To see how bizarre it is to use NDAM as an explanatory variable in a regression where the outcome variable is log(E[deaths]), plot log(deaths+0.5) or log(deaths+1) against NDAM. Then repeat the plot with log(NDAM) in place of NDAM. The contrast is even more striking if Katrina and Audrey, which Jung et al omitted as outliers, are included in the plot. By insisting on using NDAM as the explanatory variable, rather than log(NDAM), Jung et al passed up the opportunity to find a very simple form of regression relationship.

NB that E[deaths] is the number of deaths predicted by the model.

In the Jung et al data, the transformations needed can be identified from a scatterplot matrix of all variables. Try perhaps the R function spm() in the latest release of the car package for R, with transform=TRUE and (with deaths as a variable) family="yjPower". Or experiment with the transformations suggested by an initial scatterplot matrix. In general, the preferred advice may be to look first for explanatory variables that satisfy the linear predictors requirement, then attend to the outcome variable, perhaps using the car function invTranPlot().

See, in addition to "Data Analysis and Graphics Using R" that was referenced by the questioner:

  • Weisberg: Applied Linear Regression. 4th edn, Wiley 2014, pp.185-203.
  • Fox and Weisberg: An R Companion to Applied Regression. 2nd edn, Sage, 2011, pp.127-148.
Source Link

linear relations between each of the explanatory variables and the dependent variable will ensure also linear relations between the explanatory variables. The reverse is not of course true.

It is true that transformation(s) designed to give approximate linearity will increase collinearity. In the absence of such transformation(s), however, the collinearity is hidden. Suppose also that y is close to a linear function of log(x1), in a case where x1 ranges over several powers of 10. Then if x1 is used as a regressor, other explanatory variables will if at all possible be invoked to account for the nonlinearity in the relationship with x1.

This can have bizarre consequences, as in the recent paper that claimed a femaleness of hurricane name effect in data on deaths from 94 Atlantic hurricanes that made landfall over the US over 1950-2012. If one regresses log(E[deaths]) on log(NDAM), there is nothing left for the minimum pressure variable, the femaleness variable, and interactions, to explain. Regress log(E[deaths]) on NDAM, and the equation that emerges can be used to suggest a story in which the femaleness of the name has a large effect.

To see how bizarre it is to log(E[deaths]) on NDAM, plot log(deaths+0.5) or log(deaths+1) against NDAM.

NB that E[deaths] is the number of deaths predicted by the model.