The Stack Overflow podcast is back! Listen to an interview with our new CEO.
37

The glm function in R allows 3 ways to specify the formula for a logistic regression model. The most common is that each row of the data frame represents a single observation and the response variable is either 0 or 1 (or a factor with 2 levels, or other varibale with only 2 unique values). Another option is to use a 2 column matrix as the response ...


34

It will take us a while to get there, but in summary, a one-unit change in the variable corresponding to B will multiply the relative risk of the outcome (compared to the base outcome) by 6.012. One might express this as a "5012%" increase in relative risk, but that's a confusing and potentially misleading way to do it, because it suggests we should be ...


32

Suppose the population, from which we assume you are sampling randomly, contains proportions $p_1$ of promoters, $p_0$ of passives, and $p_{-1}$ of detractors, with $p_1+p_0+p_{-1}=1$. To model the NPS, imagine filling a large hat with a huge number of tickets (one for each member of your population) labeled $+1$ for promoters, $0$ for passives, and $-1$ for ...


28

Suppose all $d=6$ sides have equal chances. Let's generalize and find the expected number of rolls needed until side $1$ has appeared $n_1$ times, side $2$ has appeared $n_2$ times, ..., and side $d$ has appeared $n_d$ times. Because the identities of the sides do not matter (they all have equal chances), the description of this objective can be condensed: ...


21

If $Y$ has more than two categories your question about "advantage" of one regression over the other is probably meaningless if you aim to compare the models' parameters, because the models will be fundamentally different: $\bf log \frac{P(i)}{P(not~i)}=logit_i=linear~combination$ for each $i$ binary logistic regression, and $\bf log \frac{P(i)}{P(r)}=...


18

You can approximate it with the multivariate normal distribution in the same way that binomial distribution is approximated by univariate normal distribution. Check Elements of Distribution Theory and Multinomial Distribution pages 15-16-17. Let $P=(p_1,...,p_k)$ be the vector of your probabilities. Then the mean vector of the multivariate normal ...


14

What about using z <- summary(test)$coefficients/summary(test)$standard.errors # 2-tailed Wald z tests to test significance of coefficients p <- (1 - pnorm(abs(z), 0, 1)) * 2 p Basically, this would be based on the estimated coefficients relative to their standard error, and would use a z test to test against a significant difference with zero based ...


14

Those the two distributions are different for every $n \geq 4$. Notation I'm going to rescale your simplex by a factor $n$, so that the lattice points have integer coordinates. This doesn't change anything, I just think it makes the notation a little less cumbersome. Let $S$ be the $(n-1)$-simplex, given as the convex hull of the points $(n,0,\ldots,0)$, ....


14

Call the number of pieces in each section $A$, $B$, and $C$. Because $A+B+C=6$, you are interested in $Pr(A=2, B=2) = Pr(B=2|A=2)Pr(A=2)$. $Pr(A=2)$ is a simple binomial calculation: $A\sim Binom(6, 1/3)$, so $Pr(A=2) = {6\choose 2}(1/3)^2(2/3)^4 = 80/243$. Conditioned on $A$ having two pieces, $B\sim Binom(4, 1/2)$, so $Pr(B=2|A=2) = {4\choose 2}(1/2)^4 = ...


13

In one dimension, this sounds like a job for beta regression (with or without variable dispersion). This is a regression model with beta-distributed dependent variable, naturally 0-1 constrained. An R package is betareg and a paper describing its use is here. For more than two proportions the usual extension of the Beta distribution leads to Dirichlet ...


13

Your formula is wrong (the upper limit of the sum). In logistic regression with $K$ classes ($K> 2$) you basically create $K-1$ binary logistic regression models where you choose one class as reference or pivot. Usually, the last class $K$ is selected as the reference. Thus, the probability of the reference class can be calculated by $$P(y_i = K | x_i) = ...


13

Because of the title, I'm assuming that "advantages of multiple logistic regression" means "multinomial regression". There are often advantages when the model is fit simultaneously. This particular situation is described in Agresti (Categorical Data Analysis, 2002) pg 273. In sum (paraphrasing Agresti), you expect the estimates from a joint model to be ...


13

Assume that you're fitting a multinomial regression model with J categories where the contrast between j and the last category J is modeled as $$ \log \frac{\mu_{ij}}{\mu_{iJ}} = \alpha_j + X_i \beta_j $$ where $X_i$ is a vector of covariates associated with the $i$th case. This is rather a hard optimisation problem because the parameters are coupled by ...


13

In the case of the multinomial one has no intrinsic ordering; in contrast in the case of ordinal regression there is an association between the levels. For example if you examine the variable $V1$ that has green, yellow and red as independent levels then $V1$ encodes a multinomial variable. If you have a new variable $V2$ were the levels green, yellow and ...


12

I don't think this is an "overparamaterized" model at all. I would argue that by placing a prior over the Dirichlet paramaters, you're being less committal about any particular outcome. In particular, as you probably know, for symmetric dirichlet distributions (i.e. $\alpha_1 = \alpha_2 = ... \alpha_K$) setting $\alpha<1$ gives more prior probability to ...


12

The original version of this question started life by asking: how many rolls are needed until each side has appeared 3 times Of course, that is a question that does not have an answer as @JuhoKokkala commented above: the answer is a random variable with a distribution that needs to be found. The question was then modified to ask: "What is the expected ...


11

There is a subtle bug. What is happening is the following: In your artificial data set, the three group means are on a line, and with the relatively small standard deviation used, the three groups become linearly separable in your 10-dimensional space. As a consequence, all parameters related to the second group are estimated to 0 for all $\lambda$. Check ...


11

In my opinion, loss function is the objective function that we want our neural networks to optimize its weights according to it. Therefore, it is task-specific and also somehow empirical. Just to be clear, Multinomial Logistic Loss and Cross Entropy Loss are the same (please look at http://ufldl.stanford.edu/wiki/index.php/Softmax_Regression). The cost ...


10

In the $2\times 2$ case the distributional assumption is given by two independent binomial random variables $X_1 \sim Bin(n_1, \theta_1)$ and $X_2 \sim Bin(n_2, \theta_2)$. The null hypothesis is the equality $\theta_1=\theta_2$. But Fisher's exact test is a conditional test: it relies on the conditional distribution of $X_1$ given $X_1+X_2$. This ...


10

Fisher's so-called "exact" test makes the same kind of subtle assumptions that $\chi^2$ tests make. The two variables being assessed for association are truly polytomous all-or-nothing variables such as dead/alive US/Europe. If one or both of the variables is a simplification of an underlying continuum, categorical data analysis should not be undertaken at ...


9

As a start, if you have a dependent variable that is a proportion, you can use Beta Regression. This doesn't extend (with my limited knowledge) to multiple proportions. For Beta Regression overview and an R implementation check out betareg.


9

Lets start with the good news: The proportional odds assumptions does not require that the distances between categories are the same. So what does the proportional odds assumption imply? Say we have three ordered outcomes (1, 2, 3), then we could model the choice of 1 versus 2 or 3 and the choice of 2 versus 3. The proportional odds assumption says that ...


9

This is correct, if $$ (X_1,\ldots,X_K)\sim\mathcal{M}(N;p_1,\ldots,p_K) $$ then \begin{align*} X_1 &\sim\mathcal{B}(N,p_1)\\ X_2|X_1 &\sim\mathcal{B}\{N-X_1,p_2/(1-p_1)\}\\ &\vdots\\ X_{K-1}|X_1,\ldots,X_{K-2} &\sim \mathcal{B}\{N-X_1-\cdots-X_{K-2},p_{K-1}/(p_{K-1}+p_K)\} \end{align*} as can be shown be equating $$ \mathbb{P}((X_1,…,X_K)=(...


9

You shouldn't use the binomial distribution here as it is a multinominal distribution problem (a generalization of the binomial). So let's gather what we have: n = 6 (total number of events) n1 = 2 in part 1 (pudding #1) n2 = 2 in part 2 (pudding #2) n3 = 2 in part 3 (pudding #3) p1 = 2/6 (probability to get 2 from n1) p2 = 2/6 (probability to get 2 from ...


8

Im sure you've already found your solutions as this post is very old, but for those of us who are still looking for solutions - I have found http://youtu.be/-Cp_KP9mq94 is a great source for instructions on how to run a multinomial logistic regression model in R using mlogit package. If you go to the econonometrics academy website she has all the scripts, ...


8

There are almost certainly easier ways, but one way of computing the value precisely is compute the number of ways of placing $n$ labeled balls in $m$ labeled bins such that no bin contains $k$ or more balls. We can compute this using a simple recurrence. Let $W(n,j,m',k)$ be the number of ways of placing exactly $j$ of the $n$ labeled balls in $m'$...


8

It depends whether you are doing... a) predictive research, where you don't care about what is causally responsible, only what serves as an efficient set of indicators, or b) explanatory research, where you want to disentangle causal relationships as much as you can. In the latter, when multiple correlated predictors vie for a role in your equation, you ...


8

One question that you ought to ask first is whether or not using a weighted score is correct for the kind of analysis that you want to do. There is a discussion of that in Parkin, D., Rice, N. and Devlin, N. (2010) Statistical analysis of EQ-5D profiles: Does the use of value sets bias inference? Medical Decision Making. 30(5), 556-565 doi: 10.1177/...


8

With a multinomial logit model you impose the constraint that all the predicted probabilities add up to 1. When you use separate binary logit model you can no longer impose that constraint, they are estimated in seperate models after all. So that would be the main difference between these two models. As you can see in the example below (In Stata, as that is ...


8

You could use ANY classifier. Including Linear Discriminants, multinomial logit as Bill pointed out, Support Vector Machines, Neural Nets, CART, random forest, C5 trees, there are a world of different models that can help you predict $v.a$ using $v.b$ and $v.c$. Here is an example using the R implementation of random forest: # packages library(randomForest) ...


Only top voted, non community-wiki answers of a minimum length are eligible