What is the significance of logistic regression coefficients?

Question

I am currently reading a paper concerning voting location and voting preference in the 2000 and 2004 election. In it, there is a chart which displays logistic regression coefficients. From courses years back and a little reading up, I understand logistic regression to be a way of describing the relationship between multiple independent variables and a binary response variable. What I'm confused about is, given the table below, because the South has a logistic regression coefficient of .903, does that mean that 90.3% of Southerners vote republican? Because of the logistical nature of the metric, that this direct correlation does not exist. Instead, I assume that you can only say that the south, with .903, votes Republican more than the Mountains/plains, with the regression of .506. Given the latter to be the case, how do I know what is significant and what is not and is it possible to extrapolate a percentage of republican votes given this logistic regression coefficient.

As a side note, please edit my post if anything is stated incorrectly

Answer 1

That the author has forced someone as thoughtful as you to have ask a question like this is compelling illustration of why the practice -- still way too common -- of confining reporting of regression model results to a table like this is so unacceptable.

1. You can, as pointed out, try to transform the logit coefficient into some meaningful indication of the effect being estimated for the predictor in question but that's cumbersome and doesn't convey information about the precision of the prediction, which is usually pretty important in a logistic regression model (on voting in particular).

2. Also, the use of multiple asterisks to report "levels" of significance reinforces the misconception that p-values are some meaningful index of effect size ("wow--that one has 3 asterisks!!"); for crying out loud, w/ N's of 10,000 to 20,000, completely trivial differences will be "significant" at p < .001 blah blah.

3. There is absolutely no need to mystify in this way. The logistic regression model is an equation that can be used (through determinate calculation or better still simulation) to predict probability of an outcome conditional on specified values for predictors, subject to measurement error. So the researcher should report what the impact of predictors of interest are on the probability of the outcome variable of interest, & associated CI, as measured in units the practical importance of which can readily be grasped. To assure ready grasping, the results should be graphically displayed. Here, for example, the researcher could report that being a rural as opposed to an urban voter increases the likelihood of voting Republican, all else equal, by X pct points (I'm guessing around 17 in 2000; "divide by 4" is a reasonable heuristic) +/- x% at 0.95 level of confidence-- if that's something that is useful to know.

4. The reporting of pseudo R^2 is also a sign that the modeler is engaged in statistical ritual rather than any attempt to illuminate. There are scores of ways to compute "pseudo R^2"; one might complain that the one used here is not specified, but why bother? All are next to meaningless. The only reason anyone uses pseudo R^2 is that they or the reviewer who is torturing them learned (likely 25 or more yrs ago) that OLS linear regression is the holy grail of statistics & thinks the only thing one is ever trying to figure out is "variance explained." There are plenty of defensible ways to assess the adequacy of overall model fit for logistic analysis, and likelihood ratio conveys meaningful information for comparing models that reflect alternative hypotheses. King, G. How Not to Lie with Statistics. Am. J. Pol. Sci. 30, 666-687 (1986).

5. If you read a paper in which reporting is more or less confined to a table like this don't be confused, don't be intimidated, & definitely don't be impressed; instead be angry & tell the researcher he or she is doing a lousy job (particularly if he or she is polluting your local intellectual environment w/ mysticism & awe--amazing how many completely mediocre thinkers trick smart people into thinking they know something just b/c they can produce a table that the latter can't understand). For smart, & temperate, expositions of these ideas, see King, G., Tomz, M. & Wittenberg., J. Making the Most of Statistical Analyses: Improving Interpretation and Presentation. Am. J. Pol. Sci. 44, 347-361 (2000); and Gelman, A., Pasarica, C. & Dodhia, R. Let's Practice What We Preach: Turning Tables into Graphs. Am. Stat. 56, 121-130 (2002).

Answer 2

The idea here is that in logistic regression, we predict not the actual probability that, say, a southerner votes Republican, but a transformed version of it, the "log odds". Instead of the probability $p$, we deal with $\log p/(1-p)$ and find linear regression coefficients for the log odds.

So for example, let's assume that an urban Northeasterner has probability 0.3 of voting for a Republican. (This would of course be part of the regression; I don't see it reported in this table, although I assume it's in the original paper.) Now, $x = 1/(1+e^{-z})$ gives $z = \log {x \over 1-x}$; that is, $f^{-1}(x) = \log {x \over 1-x}$, the "log odds" corresponding to $x$. These "log odds" are what behaves linearly; the log odds corresponding to $0.3$ are $\log 0.3/0.7 \approx -0.85$. So the log odds for an urban Southerner voting Republican are this (what Wikipedia calls the intercept, $\beta_0$) plus the logistic regression coefficient for the South, $0.903$ -- that is, $-0.85 + 0.904 = 0.05$. But you want an actual probability, so we need to invert the function $p \to \log p/(1-p)$. That gives $f(0.05) \approx 1/(1+e^{-0.05}) \approx 0.51$. The actual odds have gone from $0.43$ to $1$, to $1.05$ to $1$; the ratio $1.05/0.43$ is $e^{0.903}$, the exponential of the logistic regression coefficient.

Furthermore, the effects for, say, region of the country and urban/suburban/rural don't interact. So the log odds of a rural Midwesterner voting Republican, say, are $-0.85 + 0.37 + 0.68 = +0.20$ according to this model; the probability is $f(0.20) = 1/(1+e^{-0.20}) = 0.55$.

Answer 3

The coefficients in the logistic regression represent the tendency for a given region/demographic to vote Republican, compared to a reference category. A positive coefficent means that region is more likely to vote Republican, and vice-versa for a negative coefficient; a larger absolute value means a stronger tendency than a smaller value.

The reference categories are "Northeast" and "urban voter", so all the coefficients represent contrasts with this particular voter type.

In general, there's also no restriction on the coefficients in a logistic regression to be in [0, 1], even in absolute value. Notice that the Wikipedia article itself has an example of a logistic regression with coefficients of -5 and 2.

Answer 4

You also asked "how do I know what is significant and what is not." (I assume you mean statistically significant, since practical or substantive significance is another matter.) The asterisks in the table refer to the footnote: some effects are noted as having small p-values. These are obtained using a Wald test of the significance of each coefficient. Assuming random sampling, p<.05 means that, if there were no such effect in the larger population, the probability of seeing a connection as strong as the one observed, or stronger, in a sample of this size would be less than .05. You'll see many threads on this site discussing the subtle but important related point that p<.05 does not mean that there is a .05 probability of there being no connection in the larger population.

Answer 5

Let me just stress the importance of what rolando2 and dmk38 both noted: significance is commonly misread, and there is a high risk of that happening with that tabular presentation of results.

Paul Schrodt recently offered a nice description of the issue:

Researchers find it nearly impossible to adhere to the correct interpretation of the significance test. The p-value tells you only the likelihood that you would get a result under the [usually] completely unrealistic conditions of the null hypothesis. Which is not what you want to know—you usually want to know the magnitude of the effect of an independent variable, given the data. That’s a Bayesian question, not a frequentist question. Instead we see—constantly—the p-value interpreted as if it gave the strength of association: this is the ubiquitous Mystical Cult of the Stars and P-Values which permeates our journals.(fn) This is not what the p-value says, nor will it ever.

In my experience, this mistake is almost impossible to avoid: even very careful analysts who are fully aware of the problem will often switch modes when verbally discussing their results, even if they’ve avoided the problem in a written exposition. And let’s not even speculate on the thousands of hours and gallons of ink we’ve expended correcting this in graduate papers.

(fn) The footnote also informs on another issue, mentioned by dmk38: “[the ubiquitous Mystical Cult of the Stars and P-Values] supplanted the earlier—and equally pervasive—Cult of the Highest R2, demolished… by King (1986).”