22
$\begingroup$

One of the predictors in my logistic model has been log transformed. How do you interpret the estimated coefficient of the log transformed predictor and how do you calculate the impact of that predictor on the odds ratio?

$\endgroup$
3

5 Answers 5

18
$\begingroup$

If you exponentiate the estimated coefficient, you'll get an odds ratio associated with a $b$-fold increase in the predictor, where $b$ is the base of the logarithm you used when log-transforming the predictor.

I usually choose to take logarithms to base 2 in this situation, so I can interpet the exponentiated coefficient as an odds ratio associated with a doubling of the predictor.

$\endgroup$
4
  • 4
    $\begingroup$ Interesting. I always use natural logs because many of the coefficients tend to be close to zero and then can be interpreted as proportional (relative) differences. That's not possible in any other base of logarithm. I see some merit in using other bases, but I think you need to clarify your response, because prima facie your interpretation doesn't use the value of the coefficient at all! $\endgroup$
    – whuber
    Mar 15, 2011 at 15:12
  • 1
    $\begingroup$ @whuber sorry what does prima facie mean? First face?? $\endgroup$
    – onestop
    Mar 16, 2011 at 7:56
  • 3
    $\begingroup$ google.com/search?q=define+"prima+facie" $\endgroup$
    – whuber
    Mar 16, 2011 at 14:26
  • $\begingroup$ @whuber in my answer to this question below I tried to formalize your comment here by applying the usual logic of log-log transformed regressions to this case, I also formalized the k-fold interpretation so we can compare. It looks like exponentiating the coefficient on the log-transformed variable in a log-log regression always gives you the k-fold interpretation -- it's not specific to the logistic case. But in the logistic case we're likely to be exponentiating anyway so maybe that's why the idea comes up more naturally here? $\endgroup$ Mar 3, 2020 at 3:03
10
$\begingroup$

@gung is completely correct, but, in case you do decide to keep it, you can interpret the coefficient has having an effect on each multiple of the IV, rather than each addition of the IV.

One IV that often should be transformed is income. If you included it untransformed, then each (say) \$1,000 increase in income would have an effect on the odds ratio as specified by the odds ratio. On the other hand, if you took log(10) of income, then each 10 fold increase in income would have the effect on the odds ratio specified in the odds ratio.

It makes sense to do this for income because, in many ways, an increase of \$1,000 in income is much bigger for someone who makes \$10,000 per year than someone who makes \$100,000.

One final note - although logistic regression makes no normality assumptions, even OLS regression doesn't make assumptions about the variables, it makes assumptions about the error, as estimated by the residuals.

$\endgroup$
3
  • 1
    $\begingroup$ +1, good points. I suppose I could have been more complete. In addition, I turned off the inadvertent mathjax by putting a backslash "\" immediately before the dollar signs. I hope you don't mind. $\endgroup$ Jul 18, 2012 at 1:05
  • $\begingroup$ What do you mean by 'logistic regression makes assumption about the errors' ? $\endgroup$
    – user83346
    Jan 16, 2016 at 8:20
  • $\begingroup$ No, OLS regression makes assumptions about the errors. That's what I said. $\endgroup$
    – Peter Flom
    Jan 16, 2016 at 14:08
7
$\begingroup$

This answer is adapted from The Statistical Sleuth by Fred L. Ramsey and Daniel W. Schafer.

If your model equation is:

$log(p/(1-p)) = \beta _{0} + \beta log(X)$

Then, each $k$-fold increase in $X$ is associated with a change in the odds by a multiplicative factor of $k^{\beta }$.

For example, I have the following model for presence of bed sores regressed on length of stay at a hospital.

$log(odds of bedsore)= -.44 + 0.45(length of stay)$

So my $\beta = 0.45$.

You can choose any $k$, based on what's works best for your model's interpretability.

I decide that $k=2$ and get the following:

$k^{\beta } = 2^{0.45} = 1.37$

Each doubling ($k=2$) of the length of stay is associated with a change in the odds of getting a bedsore by a factor of 1.37. Or if you double my length of stay, my odds of getting a bedsore will be 137% what they would have been otherwise.

Or if you decide $k=0.5$.

$k^{\beta } = 0.5^{0.45} = 0.73$

Each halving ($k=0.5$) of the length of stay is associated with a change in the odds of getting a bedsore by a factor of .73. Or if you cut my length of stay in half, my odds of getting a bedsore will only 73% of what they would have been otherwise.

$\endgroup$
1
  • $\begingroup$ does this require that the log used on the left side of your model equation is the same base as the log used on the right side? i think a logit regression would default the log on the left-side to log base e, so that would have to be changed to be k? $\endgroup$ Mar 2, 2020 at 19:18
5
$\begingroup$

The general model is

$ln(p/(1-p)) = \beta _{0} + \beta log_k(x)$

for some $k$, which could be $e$. I start by explaining the case of $k=e$, then consider general $k$.

Case 1: $k=e$, i.e. natural log transformed independent variable. Then if $\beta$ is close to zero we can say "a 1% increase in $x$ leads to a $\beta$ percent increase in the odds of the outcome." Details follow.

The model is

$ln(p/(1-p)) = \beta _{0} + \beta ln(x)$

where $ln()$ is the natural log.

@whuber's comment was that they always use natural logs for the independent variable, since in this case only, if $\beta$ is small then it is approximately the percentage change in the odds from a percentage increase in $x$.

To see this, it helps to define $odds(x) = p(x)/(1-p(x))$ as the odds of the dependent variable being 1 given the value x. Then the model is $ln(odd(x)) = \beta _{0} + \beta ln(x)$. Using usual arguments for log-transformed regressions (e.g. https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faqhow-do-i-interpret-a-regression-model-when-some-variables-are-log-transformed/), we can write for values $x_1$ and, say, $x_2 = 1.01 \times x_1$ ,

$odds(x_2)/odds(x_1) = (x_2/x_1)^\beta = (1.01)^\beta \approx 1 + \beta \times 0.01$

the last approximation requires $|\beta|$ to be small.

Thus we can write in this case, "a 1% increase in $x$ leads to a $\beta$ percent increase in the odds of the outcome." For example, if $\beta = 0.05$, then $\beta \times 0.01 = 0.0005$, and so a 1% increase in x leads to a 0.05% increase in the odds of the outcome being 1 (i.e. these odds are multiplied by 1.0005).

This argument rests on the base of the logarithm used for the independent variable being the same as the base used for the log odds ratio in the logit transformation. Since practically always the base used for the logit transformation is the natural log, then this argument rests on using the natural log to transform the independent variable. (If one were to make a modified logit regression that uses a different base for the logit transformation, it appears that the same argument would hold, but I do not think this is convention.)

Case 2: base $k$ transformed independent variable. Then the exponentiated coefficient, $e^\beta$, can be interpreted as the proportionate increase in the odds from a $k$-fold increase in the independent variable. Note that $k$ could be $e$, but $e$ would be a very strange choice given this interpretation.

The model is

$ln(p/(1-p)) = \beta _{0} + \beta log_k(x)$

where $ln()$ is the natural log and $log_k()$ is log base k. Notice that the logit transformation of the dependent variable remains using the natural log.

Again it helps to define $odds(x) = p(x)/(1-p(x))$ (see above). General derivations using the model equation yields that

$odds(log_k(x) + 1) / odds(log_k(x)) = e^\beta$

this is the usual interpretation of exponentiated coefficients, called "odds ratios" (e.g. in Stata, the relevant commands are -logit, or- where the "or" means "odds ratio", or -esttab, eform- where the "eform" means "exponentiate using e"). In words, the coefficient $e^\beta$ represents the proportional increase in the odds of the dependent variable being 1 from a unit increase in the independent variable. E.g. if $e^\beta = 1.10$ then the odds increase by 10% from a unit increase in the independent variable.

Since the independent variable is log transformed, we can use $1 = log_k(k)$ to find

$odds(log_k(x) + log_k(k)) / odds(log_k(x)) = e^\beta$

thus

$odds(log_k(kx)) / odds(log_k(x)) = e^\beta$

Hence the exponentiated coefficient represents the proportional increase in the odds from a k-fold increase in the $x$ (the non-log-transformed) variable.

$\endgroup$
1
  • $\begingroup$ +1 I like this answer. To clarify, if we are using k=10, as suggested by OP, OR changes for each 10-fold change in x. To scale OR to each 2-fold decrease, we would take OR to the power of log10(0.5). Is that a correct interpretation of your answer? $\endgroup$
    – bobmcpop
    May 25, 2023 at 7:25
0
$\begingroup$
Model

Assume the following model

$$ y_i \sim \text{Binomial}(n_i, p_i) \\ \log\left(\frac{p_i}{1-p_i}\right) = \eta = \beta_0 + \beta_1 \log(x_i). $$

How can we interpret the coefficient $\beta_1$?

Odds Ratio

We calculate the odds ratio between response $i$ and $j$. First note that the log odds ratio is given by

\begin{align*} \log(\text{OR}_{ij}) = \log\left(\frac{\frac{p_i}{1-p_i}}{\frac{p_j}{1-p_j}}\right) &= \log\left(\frac{p_i}{1-p_i}\right) - \log\left(\frac{p_j}{1-p_j}\right) \\ &= \beta_0 + \beta_1 \log(x_i) - \left(\beta_0 + \beta_1 \log(x_j)\right) \\ &= \beta_1 \log\left(\frac{x_i}{x_j}\right). \end{align*}

Therefore,

\begin{align*} \text{OR}_{ij} &= \exp\left(\beta_1 \log\left(\frac{x_i}{x_j}\right)\right) \\ &= \left(\exp\left(\log\left(\frac{x_i}{x_j}\right)\right)\right)^{\beta_1} \\ &= \left(\frac{x_i}{x_j}\right)^{\beta_1} \end{align*}

Interpretation

The interpretation of $\beta_1$ is simple if we define a specific ratio between $x_i$ and $x_j$. For example we could calculate the odds ratio associated with a doubling of predictor $x$ (i.e. $x_i/x_j = 2$) as

$$ 2^{\beta_1}. $$

Alternative

A widely used alternative is to transform $x$ with a logarithm to a specific base $b$ which later helps the interpretation of the coefficient. Let's say I want to determine the odds ratio associated with a doubling of predictor $x$. In this case we would transform $x$ with the logarithm to the base 2 changing the linear predictor to $\eta = \tilde\beta_0 + \tilde\beta_1 \log_2(x_i)$. This changes the formula above to

$$ \text{OR}_{ij} = \exp\left(\tilde\beta_1 \log_2\left(\frac{x_i}{x_j}\right)\right) \\ = \exp\left(\tilde\beta_1 \log_2(2)\right) \\ = \exp(\tilde\beta_1) \\ $$

i.e. the odds ratio associated with a doubling of predictor $x$ can be determined by calculating $e$ to the power of the coefficient.

  • Advantage
    • The same operation $\exp(\cdot)$ is applied to all coefficients in order to determine its effect on the odds ratio (also to coefficients associated to a predictor which was not log transformed)
  • Disadvantage
    • It is no longer simply possible to calculate the OR associated with another factor increase in $x$.
$\endgroup$