28
votes
Accepted
Expected number of times to roll a die until each side has appeared 3 times
Suppose all $d=6$ sides have equal chances. Let's generalize and find the expected number of rolls needed until side $1$ has appeared $n_1$ times, side $2$ has appeared $n_2$ times, ..., and side $d$ ...
- 323k
14
votes
Multinomial Logistic Loss vs (Cross Entropy vs Square Error)
In my opinion, loss function is the objective function that we want our neural networks to optimize its weights according to it. Therefore, it is task-specific and also somehow empirical. Just to be ...
- 241
14
votes
Accepted
Probability of each of the three Christmas puddings having exactly 2 coins
Call the number of pieces in each section $A$, $B$, and $C$. Because $A+B+C=6$, you are interested in $Pr(A=2, B=2) = Pr(B=2|A=2)Pr(A=2)$.
$Pr(A=2)$ is a simple binomial calculation: $A\sim Binom(6, ...
- 4,377
13
votes
Accepted
Is GINI limited to binary classifiers or can we use it for multi-class classifiers as well?
The Gini impurity can definitely be used to quantify variance in a multi-class setting, not only in the binary case. Gini impurity is defined as
$$ G(p) = \sum_{i=1}^{J}{p_i} \sum_{k \neq i}^{J}{p_k}...
- 298
13
votes
Accepted
Is there a probability distribution like the binomial distribution but with continuous rather than binary trial outputs?
The sum of i.i.d. uniform random variables follows the Irwin–Hall distribution.
Tim♦
- 138k
12
votes
Expected number of times to roll a die until each side has appeared 3 times
The original version of this question started life by asking:
how many rolls are needed until each side has appeared 3 times
Of course, that is a question that does not have an answer as @...
- 7,673
11
votes
Accepted
How to sample $n$ observations from a multinomial distribution using binomial (or poisson) sampling?
You can do it by progressing conditionally through the categories. I'm going to work from the last category backward (for a particular reason) but it can be done in any order as long as you're ...
- 283k
10
votes
How to simulate Likert-scale data in R?
A likert scale, as the term is typically used, is just an ordinal rating scale. The phrase is often used for a single rating, which might have been called a likert item. Traditionally, the idea was ...
10
votes
How to interpret coefficients of a multinomial elastic net (glmnet) regression
I emailed kind Dr. Hastie who is the maintainer of the glmnet package and got the following answer:
In the traditional case, the base category is arbitrary.
In ...
- 649
9
votes
How to sample a truncated multinomial distribution?
If I understand you correctly, you want to sample $x_1,\dots,x_k$ values from multinomial distribution with probabilities $p_1,\dots,p_k$ such that $\sum_i x_i = n$, however you want the distribution ...
Tim♦
- 138k
9
votes
Interpreting multinomial logistic regression in scikit-learn
As the probabilities of each class must sum to one, we can either define n-1 independent coefficients vectors, or n coefficients ...
- 591
9
votes
Probability of each of the three Christmas puddings having exactly 2 coins
You shouldn't use the binomial distribution here as it is a multinominal distribution problem (a generalization of the binomial).
So let's gather what we have:
...
- 569
9
votes
GLMNET: Weights and imbalanced data
Yes, you should provide weights. I assign weights $1 - \frac{\text{# of class members}}{\text{# of total members}}$. Glmnet rescales them to sum to the total number of class members anyway.
Here's an ...
- 342
9
votes
Accepted
Dirichlet distribution vs Multinomial distribution?
Multinomial distribution is a discrete, multivariate distribution for $k$ variables $x_1,x_2,\dots,x_k$ where each $x_i \in \{0,1,\dots,n\}$ and $\sum_{i=1}^k x_i = n$. Dirichlet distribution is a ...
Tim♦
- 138k
8
votes
What are some distributions over the probability simplex?
This is studied in compositional data analysis, there is a book by Aitchison: The Statistical Analysis Of Compositional Data.
Define the simplex by
$$
S^n =\{(x_1, \dots,x_{n+1}) \in {\mathbb ...
- 78.1k
8
votes
Accepted
conditional on the total, what is the distribution of negative binomials
Sorry for the late answer, but this bugged me as well and I found the answer. The distribution is indeed Dirichlet-Multinomial and the individual neg. binomial distributions don't even need to be ...
- 2,572
8
votes
How to threshold multiclass probability prediction to get confusion matrix?
According to @cangrejo's answer: https://stats.stackexchange.com/a/310956/194535, suppose the original output probability of your model is the vector $v$, and then you can define the prior ...
- 181
8
votes
Accepted
What is distribution parameterization?
Reparameterization means the substitution of a function for a parameter, where the parameters are the coefficients of a distribution. References on this do not help much. Parameterization is the ...
- 13.1k
8
votes
Accepted
MCMC sampling for a model with a multinomial choice--so the parameters need to sum to 1
The problem does not seem to stand with MCMC but with the prior modelling. If the data comes from a Multinomial distribution
$$\mathcal D_k(n,p_1,\ldots,p_k)$$
where the probability vector $\mathbf{p}=...
- 106k
8
votes
Accepted
Detailed derivation for the log likelihood of a logistic multinomial model
You have two nested $j$ variables in your likelihood. The likelihood is more correctly
$$\prod_i \prod_j \frac{\exp(x_i\beta_j)}{\sum_k \exp(x_i\beta_k)}$$
giving a loglikelihood
$$\sum_i\sum_j y_{ij}...
- 38.8k
7
votes
Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters
To demonstrate a solution to this hyperprior problem, I implemented an hierarchical gamma-Dirichlet-multinomial model in PyMC3. The gamma prior for the Dirichlet is specified and sampled per Ted ...
- 71
7
votes
Accepted
Correlation multinomial distribution
The probability generating function is
$$\eqalign{
f(x_1,\ldots, x_c) &= \sum_{k_1, \ldots, k_c} \Pr((X_1,\ldots,X_c)=(k_1,\ldots, k_c)) x_1^{k_1}\cdots x_c^{k_c}\\
&= \sum_{k_1,\ldots,k_c} \...
- 323k
7
votes
Accepted
Inferring alleles distribution from the blood types distribution
The probability of the blood types can be defined in terms of the alleles:
$$O = o^2$$
$$A=a^2+2oa$$
$$B=b^2+2ob$$
$$AB=2ab$$
These are 4 equations with 3 variables, and thus a solution is not ...
- 86
7
votes
Accepted
In multinomial logistic regression, why do the decision boundaries tend to be parallel to each other?
In multinomial logistic regression,
$$
p(k) = \frac{e^{x\beta_k}}{\sum_i e^{x\beta_i}}
$$
where $i, k$ are possible class labels, $x$ - input data, $\beta_i$ - coefficient vector for the class $i$.
...
- 2,301
7
votes
Multinomial & Covariances
The moment generating function is
$$\begin{aligned}
\phi(t_1,\ldots, t_k) &= E\left[\exp\left(t_1 X_1 + \cdots + t_k X_k\right)\right]\\
&= \sum_\mathbf{x} \binom{n}{\mathbf x} (p_1 e^{t_1 x_1}...
- 323k
7
votes
Help with rigorous derivation of multinomial distribution
Even under the rigorous measure-theoretic framework, your proof is overly verbose, probably due to that you confused the underlying probability space $(\Omega, \mathscr{F}, P)$, where $X_1, X_2, \...
- 18.8k
7
votes
Accepted
R - multinomial logistic regression with relative frequencies as response variable
The regular multinomial regression model is usable as long as the model-fitting procedure accepts fractional responses and robust standard errors are used. This is known as the fractional multinomial ...
- 1,544
6
votes
Multinomial logistic regression vs one-vs-rest binary logistic regression
I don't think the previous answers really capture the key difference, although it is implicit in the discussion of Independence of Irrelevant Alternatives (which is a social sciences term rather than ...
- 6,779
6
votes
Sum of coefficients of multinomial distribution
The present question is a specific case where you are dealing with a quantity that is a linear function of a multinomial random variable. It is possible to solve your problem exactly, by enumerating ...
- 125k
6
votes
Accepted
Entropy of the multinomial distribution
Ok so I guess I should have done a bit of experimentation before posting this question. I just assumed that since the Wikipedia article for the multinomial distribution didn't mention entropy, and ...
- 1,453
Only top scored, non community-wiki answers of a minimum length are eligible