30

If $Y$ has more than two categories your question about "advantage" of one regression over the other is probably meaningless if you aim to compare the models' parameters, because the models will be fundamentally different: $\bf log \frac{P(i)}{P(not~i)}=logit_i=linear~combination$ for each $i$ binary logistic regression, and $\bf log \frac{P(i)}{P(...


28

Suppose all $d=6$ sides have equal chances. Let's generalize and find the expected number of rolls needed until side $1$ has appeared $n_1$ times, side $2$ has appeared $n_2$ times, ..., and side $d$ has appeared $n_d$ times. Because the identities of the sides do not matter (they all have equal chances), the description of this objective can be condensed: ...


23

What about using z <- summary(test)$coefficients/summary(test)$standard.errors # 2-tailed Wald z tests to test significance of coefficients p <- (1 - pnorm(abs(z), 0, 1)) * 2 p Basically, this would be based on the estimated coefficients relative to their standard error, and would use a z test to test against a significant difference with zero based ...


21

Because of the title, I'm assuming that "advantages of multiple logistic regression" means "multinomial regression". There are often advantages when the model is fit simultaneously. This particular situation is described in Agresti (Categorical Data Analysis, 2002) pg 273. In sum (paraphrasing Agresti), you expect the estimates from a joint model to be ...


15

In the case of the multinomial one has no intrinsic ordering; in contrast in the case of ordinal regression there is an association between the levels. For example if you examine the variable $V1$ that has green, yellow and red as independent levels then $V1$ encodes a multinomial variable. If you have a new variable $V2$ were the levels green, yellow and ...


14

Those the two distributions are different for every $n \geq 4$. Notation I'm going to rescale your simplex by a factor $n$, so that the lattice points have integer coordinates. This doesn't change anything, I just think it makes the notation a little less cumbersome. Let $S$ be the $(n-1)$-simplex, given as the convex hull of the points $(n,0,\ldots,0)$, ....


14

Call the number of pieces in each section $A$, $B$, and $C$. Because $A+B+C=6$, you are interested in $Pr(A=2, B=2) = Pr(B=2|A=2)Pr(A=2)$. $Pr(A=2)$ is a simple binomial calculation: $A\sim Binom(6, 1/3)$, so $Pr(A=2) = {6\choose 2}(1/3)^2(2/3)^4 = 80/243$. Conditioned on $A$ having two pieces, $B\sim Binom(4, 1/2)$, so $Pr(B=2|A=2) = {4\choose 2}(1/2)^4 = ...


13

Assume that you're fitting a multinomial regression model with J categories where the contrast between j and the last category J is modeled as $$ \log \frac{\mu_{ij}}{\mu_{iJ}} = \alpha_j + X_i \beta_j $$ where $X_i$ is a vector of covariates associated with the $i$th case. This is rather a hard optimisation problem because the parameters are coupled by ...


13

The sum of i.i.d. uniform random variables follows the Irwin–Hall distribution.


12

The original version of this question started life by asking: how many rolls are needed until each side has appeared 3 times Of course, that is a question that does not have an answer as @JuhoKokkala commented above: the answer is a random variable with a distribution that needs to be found. The question was then modified to ask: "What is the expected ...


11

The joint probability mass function of the $X_i$is $$p_{\mathbf X}(\mathbf x) = \prod_{i=1}^n e^{-\lambda}\frac{\lambda^{x_i}}{x_i!} = e^{-n\lambda}\frac{\lambda^{\sum_i x_i}}{x_1!x_2!\cdots x_n!}.$$ $Y = \sum_i X_i$ is a Poisson random variable with parameter $n\lambda$ and so $P\{Y = N\} = e^{-n\lambda}\frac{(n\lambda)^{N}}{N!}$. Now, $$P\left\{(X_1=x_1, ...


11

This is correct, if $$ (X_1,\ldots,X_K)\sim\mathcal{M}(N;p_1,\ldots,p_K) $$ then \begin{align*} X_1 &\sim\mathcal{B}(N,p_1)\\ X_2|X_1 &\sim\mathcal{B}\{N-X_1,p_2/(1-p_1)\}\\ &\vdots\\ X_{K-1}|X_1,\ldots,X_{K-2} &\sim \mathcal{B}\{N-X_1-\cdots-X_{K-2},p_{K-1}/(p_{K-1}+p_K)\} \end{align*} as can be shown be equating $$ \mathbb{P}((X_1,…,X_K)=(...


11

In the $2\times 2$ case the distributional assumption is given by two independent binomial random variables $X_1 \sim Bin(n_1, \theta_1)$ and $X_2 \sim Bin(n_2, \theta_2)$. The null hypothesis is the equality $\theta_1=\theta_2$. But Fisher's exact test is a conditional test: it relies on the conditional distribution of $X_1$ given $X_1+X_2$. This ...


11

In my opinion, loss function is the objective function that we want our neural networks to optimize its weights according to it. Therefore, it is task-specific and also somehow empirical. Just to be clear, Multinomial Logistic Loss and Cross Entropy Loss are the same (please look at http://ufldl.stanford.edu/wiki/index.php/Softmax_Regression). The cost ...


11

Deviance is a specific transformation of a likelihood ratio. In particular, we consider the model-based likelihood after some fitting has been done and compare this to the likelihood of what is called the saturated model. This latter is a model that has as many parameters as data points and achieves a perfect fit, so by looking at the likelihood ratio we'...


10

Im sure you've already found your solutions as this post is very old, but for those of us who are still looking for solutions - I have found http://youtu.be/-Cp_KP9mq94 is a great source for instructions on how to run a multinomial logistic regression model in R using mlogit package. If you go to the econonometrics academy website she has all the scripts, ...


10

Fisher's so-called "exact" test makes the same kind of subtle assumptions that $\chi^2$ tests make. The two variables being assessed for association are truly polytomous all-or-nothing variables such as dead/alive US/Europe. If one or both of the variables is a simplification of an underlying continuum, categorical data analysis should not be undertaken at ...


9

With a multinomial logit model you impose the constraint that all the predicted probabilities add up to 1. When you use separate binary logit model you can no longer impose that constraint, they are estimated in seperate models after all. So that would be the main difference between these two models. As you can see in the example below (In Stata, as that is ...


9

Lets start with the good news: The proportional odds assumptions does not require that the distances between categories are the same. So what does the proportional odds assumption imply? Say we have three ordered outcomes (1, 2, 3), then we could model the choice of 1 versus 2 or 3 and the choice of 2 versus 3. The proportional odds assumption says that ...


9

Please use with PyMC 2.3.2 It looks like there is a bug in 2.3.4 (the most recent version) that is causing the wrong inference. This took me a while to discover, but it was solved when I downgraded to PyMC 2.3.2. Model: import pymc as pm p = [ #brown, yellow, red, green, orange, tan, blue [.3, .2, .2, .1, .1, .1, .0 ], # 1994 bag [.13, .14, .13, ....


9

If I understand you correctly, you want to sample $x_1,\dots,x_k$ values from multinomial distribution with probabilities $p_1,\dots,p_k$ such that $\sum_i x_i = n$, however you want the distribution to be truncated so $a_i \le x_i \le b_i$ for all $x_i$. I see three solutions (neither as elegant as in non-truncated case): Accept-reject. Sample from non-...


9

You shouldn't use the binomial distribution here as it is a multinominal distribution problem (a generalization of the binomial). So let's gather what we have: n = 6 (total number of events) n1 = 2 in part 1 (pudding #1) n2 = 2 in part 2 (pudding #2) n3 = 2 in part 3 (pudding #3) p1 = 2/6 (probability to get 2 from n1) p2 = 2/6 (probability to get 2 from ...


8

You could use ANY classifier. Including Linear Discriminants, multinomial logit as Bill pointed out, Support Vector Machines, Neural Nets, CART, random forest, C5 trees, there are a world of different models that can help you predict $v.a$ using $v.b$ and $v.c$. Here is an example using the R implementation of random forest: # packages library(randomForest) ...


8

Let's make this simple, and do a very contrived two class case: Let's say we have three documents with the following words: Doc 1: "Food" occurs twice, "Meat" occurs once, "Brain" occurs once Class of Doc 1: "Health" Doc 2: "Food" occurs once, "Meat" occurs once, "Kitchen" occurs 9 times, "Job" occurs 5 times. Class of Doc 2: "Butcher" Doc 3: "Food" ...


8

It's called the categorical distribution (among other things). http://en.wikipedia.org/wiki/Categorical_distribution This article mentions the following names: categorical, multinoulli, generalized Bernoulli


8

Reparameterization means the substitution of a function for a parameter, where the parameters are the coefficients of a distribution. References on this do not help much. Parameterization is the explicit form for a distribution. For example, the gamma distribution has two different parameterizations that are in common use: 1) The probability density ...


8

The Gini impurity can definitely be used to quantify variance in a multi-class setting, not only in the binary case. Gini impurity is defined as $$ G(p) = \sum_{i=1}^{J}{p_i} \sum_{k \neq i}^{J}{p_k} = 1-\sum_{i=1}^{J}{(p_i)^{2}} $$ for the scenario with $J$ classes, each having probability $p_i...p_J$, where $|J|$ can be $>2$. More information can ...


8

I emailed kind Dr. Hastie who is the maintainer of the glmnet package and got the following answer: In the traditional case, the base category is arbitrary. In fact you can take a fitted model where say category one is the base category, and simply by subtraction of coefficients, make an equivalent model where another is the base (and the fit is identical)....


7

This is studied in compositional data analysis, there is a book by Aitchison: The Statistical Analysis Of Compositional Data. Define the simplex by $$ S^n =\{(x_1, \dots,x_{n+1}) \in {\mathbb R}^{n+1} \colon x_1>0,\dots, x_{n+1}>0, \sum_{i=1}^{n+1} x_i=1\}. $$ Note that we use the index $n$ to indicate dimension! Define the geometric mean of ...


7

Sorry for the late answer, but this bugged me as well and I found the answer. The distribution is indeed Dirichlet-Multinomial and the individual neg. binomial distributions don't even need to be identical, as long as their Fano factor (ratio of variance to mean) is identical. Long answer: If you parameterize NB as: $ p(X = x | \lambda, \theta) = NB(x | \...


Only top voted, non community-wiki answers of a minimum length are eligible