# Tag Info

66

With generalized linear models, there are three different types of statistical tests that can be run. These are: Wald tests, likelihood ratio tests, and score tests. The excellent UCLA statistics help site has a discussion of them here. The following figure (copied from their site) helps to illustrate them: The Wald test assumes that the likelihood is ...

50

It seems self-evident to me that $$\exp(\beta_0 + \beta_1x) \neq\frac{\exp(\beta_0 + \beta_1x)}{1+\exp(\beta_0 + \beta_1x)}$$ unless $\exp(\beta_0 + \beta_1x)=0$. So, I'm less clear about what the confusion might be. What I can say is that the left hand side (LHS) of the (not) equals sign is the odds of being undernourished, whereas the RHS is the ...

35

Odds are a way to express chances. Odds ratios are just that: one odds divided by another. That means an odds ratio is what you multiply one odds by to produce another. Let's see how they work in this common situation. Converting between odds and probability The odds of a binary response $Y$ are the ratio of the chance it happens (coded with $1$), ...

26

In this case you can collapse your data to $$\begin{array}{c|cc} X \backslash Y & 0 & 1 \\ \hline 0 & S_{00} & S_{01} \\ 1 & S_{10} & S_{11} \end{array}$$ where $S_{ij}$ is the number of instances for $x = i$ and $y =j$ with $i,j \in \{0,1\}$. Suppose there are $n$ observations overall. If we fit the model $p_i = g^{-1}(x_i^T \beta)... 25 You're on the right track, but always have a look at the documentation of the software you're using to see what model is actually fit. Assume a situation with a categorical dependent variable$Y$with ordered categories$1, \ldots, g, \ldots, k$and predictors$X_{1}, \ldots, X_{j}, \ldots, X_{p}$. "In the wild", you can encounter three equivalent choices ... 25 The odds is not the same as the probability. The odds is the number of "successes" (deaths) per "failure" (continue to live), while the probability is the proportion of "successes". I find it instructive to compare how one would estimate these two: An estimate of the odds would be the ratio of the number of successes over the number of failures, while an ... 22 If you're only putting that lone predictor into the model, then the odds ratio between the predictor and the response will be exactly equal to the exponentiated regression coefficient. I don't think a derivation of this result in present on the site, so I will take this opportunity to provide it. Consider a binary outcome$Y$and single binary predictor$X$... 14 The estimators$\widehat{OR}$have the asymptotic normal distribution around$OR$. Unless$n$is quite large, however, their distributions are highly skewed. When$OR=1$, for instance,$\widehat{OR}$cannot be much smaller than$OR$(since$\widehat{OR}\ge0$), but it could be much larger with non-negligible probability. The log transform, having an additive ... 14 That’s the logarithm of the odds ratio, not the odds ratio itself. An odds ratio less than zero is nonsense. Looking at the behavior of a logarithm function (the base could be 2, could be 10, could be$e$), the function achieves values less than zero when the argument is less than 1, so a negative log of the odds ratio means that the odds ratio is between 0 ... 13 These odds ratios are the exponential of the corresponding regression coefficient: $$\text{odds ratio} = e^{\hat\beta}$$ For example, if the logistic regression coefficient is$\hat\beta=0.25$the odds ratio is$e^{0.25} = 1.28$. The odds ratio is the multiplier that shows how the odds change for a one-unit increase in the value of the X. The odds ratio ... 12 an odds ratio of 2 means that the event is 2 time more probable given a one-unit increase in the predictor It means the odds would double, which is not the same as the probability doubling. In Cox regression, a hazard ratio of 2 means the event will occur twice as often at each time point given a one-unit increase in the predictor. Aside a bit of ... 12 Unadjusted OR is a simple ratio of probabilities of outcome in two groups$p_1, p_2$(check here or here): $$OR = \frac{p_1/(1-p_1)}{p_2/(1-p_2)}$$ and it can be derived from the results of logistic regression (as opposed to counting a simple ratio calculated by hand from a$2 \times 2$table). However, in logistic regression you can include other, ... 11 There are a number of alternative effects one can derive from the logistic regression model that do not suffer this same problem. One of the easiest is the average marginal effect of the variable. Assume the following logistic regression model: $$\ln\Bigg[\frac{p}{1-p}\Bigg]=X\beta + \gamma d$$ where$X$is an$n$(cases) by$k$... 11 Is it okay that the direction of the fixed effects is opposite to the intercept in my model? Yes. The interpretation of the intercept is that it is the log-odds of the event (DV = 1) for the those in the none group (so, being negative, it is a protective effect), while the estimates for the other 2 groups are the log-odds of the event in each of those ... 10 From the help page for fisher.test(): Note that the conditional Maximum Likelihood Estimate (MLE) rather than the unconditional MLE (the sample odds ratio) is used. 10 Hi Erica and welcome to the site. Have a look at this (page 3) document and this paper. The basic formula for the conversion is $$d=\mathrm{LogOR}\times \frac{\sqrt{3}}{\pi}$$ Applying the delta-method, we get the following expression for the the variance of$d$(the standard error of$d$is just the square root of its variance): $$\mathrm{Var}_{d}=\... 10 The coefficients that are returned standard with a logistic regression fit are not odds ratios. They represent the change in the log odds of 'success' associated with a one-unit change in their respective variable, when all else is held equal. If you exponentiate a coefficient, then you can interpret the result as an odds ratio (of course, this is not true ... 10 If you write out the fitted model for the log odds of smoking$$\log \frac{\Pr(Y=1)}{\Pr(Y=0)} = -4.380\,1 + -0.324\,56\ I_\mathrm{teen} + 1.451\,19 \ I_\mathrm{mature} + -0.989\,1\ I_\mathrm{old}$$where the dummies are$$I_\mathrm{teen}=\left\{ \begin{array}{l l} 0 & X\neq\mathrm{teenager}\\ 1& X=\mathrm{teenager}\\ \end{array}\right.$$&c., ... 10 A key step is to make sure people understand why log-odds-ratios are useful. To help motivate log-odds-ratios, try the tale of two principals: High School A reduced the dropout rate from 10% to 5%, a dramatic 50% decrease! High School B increased the graduation rate from 90% to 95%, a modest 5.5% increase. The first principal was lauded by the NYTimes ... 9 If you're using the equation you list below your code, I think you're OK. It's true that the numbers inside that equation are log odds, but once you've solved for \text{Pr}(Y=1), you do have a probability. As far as I can tell, you are not misinterpreting your results. 9 It's not completely clear from your question, but I'm assuming you're talking about the situation where you have a single binary response Y and a continuous predictor X and fit a logistic regression model:$$ \log \left( \frac{ P(Y_{i}=1|X_{i}) }{P(Y_{i}=0|X_{i})} \right) = \beta_{0} + \beta_{1} X_{i} $$Then the odds ratio, e^{\beta_{1}}, is the ... 9 I too speculate at the prevalence of logistic models in the literature when a relative risk model would be more appropriate. We as statisticians are all too familiar with adherence to convention or sticking to "drop-down-menu" analyses. These create far more problems than they solve. Logistic regression is taught as a "standard off the shelf tool" for ... 9 I think that the first and biggest hurdle is making sure that people indeed understand logistic regression and what an odds ratio actually is. If they get that far, you simply need to explain that proportional odds models take logistic regression one step further to account for ordered categorical responses. A naive approach could be to run a logistic ... 9 Odds ratios are asymptotically Gaussian. Therefore their difference, as long as they are independent, is also asymptotically Gaussian, because the linear combination of independent Gaussian r.v.s is itself Gaussian. These are both fairly well-known and shouldn't require a citation. But just for assurance, both of those links are based on "authoritative" ... 8 Short answer no, different margins can produce the same odd's ratio and confidence interval. Some examples to follow. Here is a brief sketch of how to find the minimum possible N for the table. Note that as per your linked site, the standard error can be related to the cell contents by:$$\text{SE} = \sqrt{\frac{1}{a} + \frac{1}{b} + \frac{1}{c} + \frac{1}{... 8 You have chosen to do a one-sided test and, obviously, order is important in a one-sided test. Your first call to fisher.test is testing the null hypothesis Pct1 = Pct2 vs the alternative that Pct1 < Pct2. The second call is testing the same null vs the alternative that Pct2 < Pct1. The two alternatives are opposites of one another, so they give p-... 7 The log odds ratio is the log of the odds ratio, not the odds ratio divided by a log. I don't know what problem the link you gave was trying to solve, but it wasn't this one. You take the log of the OR because the OR is bounded by 0 and infinity and is multiplicatively symmetric around 1; while the log(OR) is unbounded and additively symmetric around 0. 7 Thanks for adding the table Given this, I think the OR is fine. The logistic regression you posted also included another variable, but, in the table, the OR for L1 = 4 vs. L1 = 0 is$\frac{17537*1328}{1284*44} = 412.23\$ which is actually a little larger than the one from the regression. It's a very strong relationship.

7

An important point needs to be added to Peter's good answer: 372 times as likely is definitely not correct, although you would be about the ten gazillionth person to make this mistake. A number close to 400 is the ratio not of two probabilities but of two odds. The ratio of the two corresponding probabilities is [1328/(1328+44)] / [1284/(1284+17537)] =...

7

Regularized linear regression and regularized logistic regression can be interpreted nicely from a Bayesian point of view. The regularization parameter corresponds to a choice of prior distribution on the weights, for example, something like a normal distribution centered at zero with standard deviation given by the inverse of the regularization parameter. ...

Only top voted, non community-wiki answers of a minimum length are eligible