2
$\begingroup$

Consider the following case: I am analyzing a the effect of (among other variables) the age of a firm on a specific binary event. Theoretically my perception is that age matters, but not linearly. That is, I don't believe that the age of firms e.g. 300 years matter 15 times more than a firm 20 years old. Can a transformation be used in such case?

Mathematically a log transformation has the properties that i look for as it evens out the weight of very high vs. low values. Below is a visual example. The transformed value chart demonstrates roughly the effect I want to model on the DV in the logistic regression.

To me, intuitively it makes perfect sense, however it is difficult for me to find any examples of this particular case as all sources I have been through covers transformation for normality and linearity in OLS regression, assumptions that do not apply to logistic regression.

I hope anyone can help with this most likely simple question.

y axis: value, x: axis (hypothetical observation, not relevant)

x axis: observation number y axis: variable value

Edit: how can the best transformation be identified when the DV is binary?

When attempting to find the best transformation for a continuous IV in relation to a binary DV, how is this best done? For a continuous DV in e.g. OLS regression, this is possible visually, but when attempting this for a binary variable, it obviously becomes difficult (see below).

enter image description here

Improve this question
$\endgroup$
5
  • 3
    $\begingroup$ This is answered pretty well at stats.stackexchange.com/questions/4831/… . $\endgroup$
    – EdM
    Dec 22, 2014 at 17:54
  • $\begingroup$ @EdM Ok,so 'transforming to achieve linearity' means exactly transforming to obtain a more accurate model of the phenomenon modeled. The problem is now: how can i visually diagnose the relationship ton decide on the most accurate transformation when the DV is binary? a scatter plot makes no sense with a binary DV. $\endgroup$
    – SuppaiKamo
    Dec 22, 2014 at 18:27
  • 2
    $\begingroup$ In R, the spineplot() function is a good way to start visualizing. With a numeric independent variable it displays a 'spinogram' of the relative frequencies of the binary DV outcomes for histogram-like groupings of the independent variable. Or you can do it yourself manually: break down your data into groups by Foreign Subsidiary Count, calculate the proportion of IsWOS for each group, and plot that against the mean or median value of Foreign Subsidiary Count for each group. $\endgroup$
    – EdM
    Dec 22, 2014 at 21:54
  • 1
    $\begingroup$ Upon smoothing the data and applying the logit, as suggested in Scortchi's answer, you can apply exploratory techniques to linearize bivariate relationships. An illustrated worked example is carried out in my posts at stats.stackexchange.com/a/35717 and stats.stackexchange.com/a/60455. A way to incorporate theoretical knowledge of a bivariate relationship into a GLM is outlined (with working code) at stats.stackexchange.com/a/64039. $\endgroup$
    – whuber ♦
    Jan 2, 2015 at 17:11
  • $\begingroup$ one way is also to look at the log odds response using for exemple the weight of evidence (woe) for lineraity of the reponse then depending on your aim you can check the relative ordering with a ROC curve. $\endgroup$
    – R. Prost
    Jan 29, 2018 at 9:53

1 Answer 1

Reset to default
2
$\begingroup$

Use a smoother on the binary reponse to visually investigate the relationship: see Creating univariable smoothed scatterplot on logit scale using R. Taking the logit (or another link function where applicable) of the smooth allows you to assess its linearity on the appropriate scale. Note in plots like these you're looking at the marginal effect of the predictor along the abscissa & not taking into account confounding effects of other predictors.

Note that investigating transformations in this way & picking the "best" introduces an optimistic bias into model performance measures (when the model's fitted to the same data that suggested the transformation) which can't be incorporated into resampling validation procedures. An alternative approach would be to consider that the relationship between any simple transformation of "age of firm" and the logit of the response isn't the kind of thing that's going to be exactly linear, & to incorporate your uncertainty about its nature into the model-fitting process by representing "age of firm" with a polynomial or spline basis. That doesn't preclude transforming the predictor beforehand when this seems sensible given its distribution—often done to reduce the influence of observations in the right tail of a very skewed distribution.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ My goal at this point (Master's thesis project in business admin) is not to achieve a perfectly fitted model, and I do not expect to apply advanced statistical techniques to obtain this neither. I would rather apply transformation to model influence of effects so that they conform to theoretical logic/empirics in the field, and do a simple test of fit of the transformed vs. non transformed variable in the model. The distribution of the variable in question is indeed right skewed, so I'll try this with a log. In your opinion as experienced in modelling, is this a sensible approach? $\endgroup$
    – SuppaiKamo
    Dec 23, 2014 at 19:59
  • $\begingroup$ The theoretical argument is that the effect of firm age diminishes after a certain point, which intuitively leads me to think that a log transform is sensible? $\endgroup$
    – SuppaiKamo
    Dec 23, 2014 at 20:11
  • 2
    $\begingroup$ (1) As @whuber says, "there is (usually) nothing special about how the data are originally expressed" - one man's taking logs is another's refraining from exponentiating - so don't be shy of specifying predictors in the form suggested by your subject-matter knowledge. (2) The problems I mentioned are exacerbated by searching too assiduously for "good" transformations for many predictors, but ameliorated by selecting from among a few well-founded competing forms. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Jan 2, 2015 at 16:35
  • 2
    $\begingroup$ (+1) Better yet, after smoothing, consider applying the link function to the smoothed values. For instance, with the usual logit link, take the logit (log odds) of the smooth. If that departs obviously from linearity, it is likely that after performing a logistic regression the post-fitting goodness-of-fit tests will suggest problems. In such cases, preliminary transformations of independent variables can be helpful. Note that such diagnoses can be made using relatively small subsets of the data (for "training") if one wants to preserve the ability to make formal tests after fitting. $\endgroup$
    – whuber ♦
    Jan 2, 2015 at 16:50
  • 1
    $\begingroup$ Certainly, @BrettMagill's answer illustrates taking the logit of the smooth - will edit. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Jan 2, 2015 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.