88

With generalized linear models, there are three different types of statistical tests that can be run. These are: Wald tests, likelihood ratio tests, and score tests. The excellent UCLA statistics help site has a discussion of them here. The following figure (copied from their site) helps to illustrate them: The Wald test assumes that the likelihood is ...


41

Odds are a way to express chances. Odds ratios are just that: one odds divided by another. That means an odds ratio is what you multiply one odds by to produce another. Let's see how they work in this common situation. Converting between odds and probability The odds of a binary response $Y$ are the ratio of the chance it happens (coded with $1$), written ...


26

In this case you can collapse your data to $$ \begin{array}{c|cc} X \backslash Y & 0 & 1 \\ \hline 0 & S_{00} & S_{01} \\ 1 & S_{10} & S_{11} \end{array} $$ where $S_{ij}$ is the number of instances for $x = i$ and $y =j$ with $i,j \in \{0,1\}$. Suppose there are $n$ observations overall. If we fit the model $p_i = g^{-1}(x_i^T \beta)...


19

These odds ratios are the exponential of the corresponding regression coefficient: $$\text{odds ratio} = e^{\hat\beta}$$ For example, if the logistic regression coefficient is $\hat\beta=0.25$ the odds ratio is $e^{0.25} = 1.28$. The odds ratio is the multiplier that shows how the odds change for a one-unit increase in the value of the X. The odds ratio ...


15

an odds ratio of 2 means that the event is 2 time more probable given a one-unit increase in the predictor It means the odds would double, which is not the same as the probability doubling. In Cox regression, a hazard ratio of 2 means the event will occur twice as often at each time point given a one-unit increase in the predictor. Aside a bit of ...


15

That’s the logarithm of the odds ratio, not the odds ratio itself. An odds ratio less than zero is nonsense. Looking at the behavior of a logarithm function (the base could be 2, could be 10, could be $e$), the function achieves values less than zero when the argument is less than 1, so a negative log of the odds ratio means that the odds ratio is between 0 ...


14

Essentially, the Delta Method is a way of "linearizing" a non-linear function using a Taylor Series expansion so that you can find the variance and hence the standard error. For example, let's say you have a function $f(X) = Y$ that has a first and second order derivative. Then a first order Taylor Series Expansion centered around $\mu$ is given ...


12

From the help page for fisher.test(): Note that the conditional Maximum Likelihood Estimate (MLE) rather than the unconditional MLE (the sample odds ratio) is used.


12

Unadjusted OR is a simple ratio of probabilities of outcome in two groups $p_1, p_2$ (check here or here): $$OR = \frac{p_1/(1-p_1)}{p_2/(1-p_2)} $$ and it can be derived from the results of logistic regression (as opposed to counting a simple ratio calculated by hand from a $2 \times 2$ table). However, in logistic regression you can include other, ...


12

Is it okay that the direction of the fixed effects is opposite to the intercept in my model? Yes. The interpretation of the intercept is that it is the log-odds of the event (DV = 1) for the those in the none group (so, being negative, it is a protective effect), while the estimates for the other 2 groups are the log-odds of the event in each of those ...


11

If you write out the fitted model for the log odds of smoking $$\log \frac{\Pr(Y=1)}{\Pr(Y=0)} = -4.380\,1 + -0.324\,56\ I_\mathrm{teen} + 1.451\,19 \ I_\mathrm{mature} + -0.989\,1\ I_\mathrm{old}$$ where the dummies are $$I_\mathrm{teen}=\left\{ \begin{array}{l l} 0 & X\neq\mathrm{teenager}\\ 1& X=\mathrm{teenager}\\ \end{array}\right.$$ &c., ...


11

$\exp(1.4345) \approx 4.20$ $\exp(1.4345+1.96 \times 0.5346) \approx 11.97$ $\exp(1.4345-1.96 \times 0.5346) \approx 1.472$ In R > exp(summary(m)$coefficients["DSH",1] + + qnorm(c(0.025,0.5,0.975)) * summary(m)$coefficients["DSH",2]) [1] 1.472098 4.197368 11.967884


11

You can also use the confint.default function which is based on asymptotic normality. exp(cbind("Odds ratio" = coef(m), confint.default(m, level = 0.95)))


10

The coefficients that are returned standard with a logistic regression fit are not odds ratios. They represent the change in the log odds of 'success' associated with a one-unit change in their respective variable, when all else is held equal. If you exponentiate a coefficient, then you can interpret the result as an odds ratio (of course, this is not true ...


10

I too speculate at the prevalence of logistic models in the literature when a relative risk model would be more appropriate. We as statisticians are all too familiar with adherence to convention or sticking to "drop-down-menu" analyses. These create far more problems than they solve. Logistic regression is taught as a "standard off the shelf tool" for ...


10

I think that the first and biggest hurdle is making sure that people indeed understand logistic regression and what an odds ratio actually is. If they get that far, you simply need to explain that proportional odds models take logistic regression one step further to account for ordered categorical responses. A naive approach could be to run a logistic ...


10

A key step is to make sure people understand why log-odds-ratios are useful. To help motivate log-odds-ratios, try the tale of two principals: High School A reduced the dropout rate from 10% to 5%, a dramatic 50% decrease! High School B increased the graduation rate from 90% to 95%, a modest 5.5% increase. The first principal was lauded by the NYTimes ...


10

There are a number of alternative effects one can derive from the logistic regression model that do not suffer this same problem. One of the easiest is the average marginal effect of the variable. Assume the following logistic regression model: \begin{equation} \ln\Bigg[\frac{p}{1-p}\Bigg]=X\beta + \gamma d \end{equation} where $X$ is an $n$ (cases) by $k$ ...


9

Odds ratios are asymptotically Gaussian. Therefore their difference, as long as they are independent, is also asymptotically Gaussian, because the linear combination of independent Gaussian r.v.s is itself Gaussian. These are both fairly well-known and shouldn't require a citation. But just for assurance, both of those links are based on "authoritative" ...


9

I think it's a typo. The derivative of the logistic curve with respect to $x$ is: $$ \frac{\beta\mathrm{e}^{\alpha + \beta x}}{\left(1 + \mathrm{e}^{\alpha + \beta x}\right)^{2}} $$ So for their example where $\alpha = -1.40, \beta = 0.33$ it is: $$ \frac{0.33\mathrm{e}^{-1.40 + 0.33 x}}{\left(1 + \mathrm{e}^{-1.40 + 0.33 x}\right)^{2}} $$ Evaluated at the ...


8

Short answer no, different margins can produce the same odd's ratio and confidence interval. Some examples to follow. Here is a brief sketch of how to find the minimum possible N for the table. Note that as per your linked site, the standard error can be related to the cell contents by: $$\text{SE} = \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{...


8

The fisher's exact test in R by default tests whether the odds ratio associated with the first cell being 1 or not. That said, you can interpret the odds ratio 0.53 as: the odds of being male for a non-overwieght subject is 0.53 times that for an overweighted subject. Note the p-value is significant and the confidence interval doesn't contain 1. Therefore, ...


8

You are right that although you should be able to calculate the OLS coefficient estimate in logit space, you can't do it directly because the logit, $g(y) = \log \frac{p}{1-p}$, goes either to $-\infty$ for $y=0$ or $\infty$ for $y=1$. An added difficulty is that the variance in this model depends on $x$. The likelihood for logistic regression is optimized ...


8

You have chosen to do a one-sided test and, obviously, order is important in a one-sided test. Your first call to fisher.test is testing the null hypothesis Pct1 = Pct2 vs the alternative that Pct1 < Pct2. The second call is testing the same null vs the alternative that Pct2 < Pct1. The two alternatives are opposites of one another, so they give p-...


8

Given the comments, I have included the proof of the floating equation at the bottom of the response. Given a two-by-two contingency table where the OR is $\frac{a/b}{c/d}$, if you take the log of both sides, the log odds ratio is $\log(a) - \log(b) - (\log(c) - \log(d))$. Hence the formula for the log odds ratio is additive. Because it is additive, we can ...


8

Part of the problem is that you're taking a sentence from Gelman and Hill out of context. Here's a Google books screenshot: Note that the heading says "Interpreting Poisson regression coefficients" (emphasis added). Poisson regression uses a logarithmic link, in contrast to logistic regression, which uses a logit (log-odds) link. The interpretation of ...


7

The epiDisplay package does this very easily. library(epiDisplay) data(Wells, package="carData") glm1 <- glm(switch~arsenic+distance+education+association, family=binomial, data=Wells) logistic.display(glm1) Logistic regression predicting switch : yes vs no crude OR(95%CI) adj. OR(95%CI) P(Wald's test)...


7

If the confidence interval and the test are not quite based on the same calculation (in at least the somewhat loose sense that they give the same partial order to the sample space), then in some cases the two won't exactly correspond. There are a number of cases where the usual interval and test are based on different statistics that give similar but not ...


7

Regularized linear regression and regularized logistic regression can be interpreted nicely from a Bayesian point of view. The regularization parameter corresponds to a choice of prior distribution on the weights, for example, something like a normal distribution centered at zero with standard deviation given by the inverse of the regularization parameter. ...


7

The justification for the procedure is the asymptotic normality of the MLE for $\beta$ and results from arguments involving the Central Limit Theorem. The Delta method comes from a linear (i.e first order Taylor) expansion of the function around the MLE. Subsequently we appeal to the asymptotic normality and unbiasedness of the MLE. Asymptotically both give ...


Only top voted, non community-wiki answers of a minimum length are eligible