14
votes
Accepted
Probability of each of the three Christmas puddings having exactly 2 coins
Call the number of pieces in each section $A$, $B$, and $C$. Because $A+B+C=6$, you are interested in $Pr(A=2, B=2) = Pr(B=2|A=2)Pr(A=2)$.
$Pr(A=2)$ is a simple binomial calculation: $A\sim Binom(6, ...
- 4,389
13
votes
Accepted
Is GINI limited to binary classifiers or can we use it for multi-class classifiers as well?
The Gini impurity can definitely be used to quantify variance in a multi-class setting, not only in the binary case. Gini impurity is defined as
$$ G(p) = \sum_{i=1}^{J}{p_i} \sum_{k \neq i}^{J}{p_k}...
- 308
13
votes
Accepted
Is there a probability distribution like the binomial distribution but with continuous rather than binary trial outputs?
The sum of i.i.d. uniform random variables follows the Irwin–Hall distribution.
- 140k
11
votes
Accepted
How to sample $n$ observations from a multinomial distribution using binomial (or poisson) sampling?
You can do it by progressing conditionally through the categories. I'm going to work from the last category backward (for a particular reason) but it can be done in any order as long as you're ...
- 287k
10
votes
How to simulate Likert-scale data in R?
A likert scale, as the term is typically used, is just an ordinal rating scale. The phrase is often used for a single rating, which might have been called a likert item. Traditionally, the idea was ...
10
votes
How to interpret coefficients of a multinomial elastic net (glmnet) regression
I emailed kind Dr. Hastie who is the maintainer of the glmnet package and got the following answer:
In the traditional case, the base category is arbitrary.
In ...
- 669
10
votes
Accepted
Dirichlet distribution vs Multinomial distribution?
Multinomial distribution is a discrete, multivariate distribution for $k$ variables $x_1,x_2,\dots,x_k$ where each $x_i \in \{0,1,\dots,n\}$ and $\sum_{i=1}^k x_i = n$. Dirichlet distribution is a ...
- 140k
9
votes
What are some distributions over the probability simplex?
This is studied in compositional data analysis, there is a book by Aitchison: The Statistical Analysis Of Compositional Data.
Define the simplex by
$$
S^n =\{(x_1, \dots,x_{n+1}) \in {\mathbb ...
- 81.7k
9
votes
How to sample a truncated multinomial distribution?
If I understand you correctly, you want to sample $x_1,\dots,x_k$ values from multinomial distribution with probabilities $p_1,\dots,p_k$ such that $\sum_i x_i = n$, however you want the distribution ...
- 140k
9
votes
Interpreting multinomial logistic regression in scikit-learn
As the probabilities of each class must sum to one, we can either define n-1 independent coefficients vectors, or n coefficients ...
- 591
9
votes
Probability of each of the three Christmas puddings having exactly 2 coins
You shouldn't use the binomial distribution here as it is a multinominal distribution problem (a generalization of the binomial).
So let's gather what we have:
...
- 569
9
votes
GLMNET: Weights and imbalanced data
Yes, you should provide weights. I assign weights $1 - \frac{\text{# of class members}}{\text{# of total members}}$. Glmnet rescales them to sum to the total number of class members anyway.
Here's an ...
- 352
9
votes
Help with rigorous derivation of multinomial distribution
Even under the rigorous measure-theoretic framework, your proof is overly verbose, probably due to that you confused the underlying probability space $(\Omega, \mathscr{F}, P)$, where $X_1, X_2, \...
- 20.6k
8
votes
Accepted
conditional on the total, what is the distribution of negative binomials
Sorry for the late answer, but this bugged me as well and I found the answer. The distribution is indeed Dirichlet-Multinomial and the individual neg. binomial distributions don't even need to be ...
- 2,934
8
votes
How to threshold multiclass probability prediction to get confusion matrix?
According to @cangrejo's answer: https://stats.stackexchange.com/a/310956/194535, suppose the original output probability of your model is the vector $v$, and then you can define the prior ...
- 181
8
votes
Accepted
What is distribution parameterization?
Reparameterization means the substitution of a function for a parameter, where the parameters are the coefficients of a distribution. References on this do not help much. Parameterization is the ...
- 13.2k
8
votes
Accepted
MCMC sampling for a model with a multinomial choice--so the parameters need to sum to 1
The problem does not seem to stand with MCMC but with the prior modelling. If the data comes from a Multinomial distribution
$$\mathcal D_k(n,p_1,\ldots,p_k)$$
where the probability vector $\mathbf{p}=...
- 107k
8
votes
Accepted
Detailed derivation for the log likelihood of a logistic multinomial model
You have two nested $j$ variables in your likelihood. The likelihood is more correctly
$$\prod_i \prod_j \frac{\exp(x_i\beta_j)}{\sum_k \exp(x_i\beta_k)}$$
giving a loglikelihood
$$\sum_i\sum_j y_{ij}...
- 43.4k
7
votes
Multinomial-Dirichlet model with hyperprior distribution on the concentration parameters
To demonstrate a solution to this hyperprior problem, I implemented an hierarchical gamma-Dirichlet-multinomial model in PyMC3. The gamma prior for the Dirichlet is specified and sampled per Ted ...
- 71
7
votes
Accepted
Correlation multinomial distribution
The probability generating function is
$$\eqalign{
f(x_1,\ldots, x_c) &= \sum_{k_1, \ldots, k_c} \Pr((X_1,\ldots,X_c)=(k_1,\ldots, k_c)) x_1^{k_1}\cdots x_c^{k_c}\\
&= \sum_{k_1,\ldots,k_c} \...
- 330k
7
votes
Accepted
Inferring alleles distribution from the blood types distribution
The probability of the blood types can be defined in terms of the alleles:
$$O = o^2$$
$$A=a^2+2oa$$
$$B=b^2+2ob$$
$$AB=2ab$$
These are 4 equations with 3 variables, and thus a solution is not ...
- 86
7
votes
Accepted
In multinomial logistic regression, why do the decision boundaries tend to be parallel to each other?
In multinomial logistic regression,
$$
p(k) = \frac{e^{x\beta_k}}{\sum_i e^{x\beta_i}}
$$
where $i, k$ are possible class labels, $x$ - input data, $\beta_i$ - coefficient vector for the class $i$.
...
- 2,311
7
votes
Multinomial & Covariances
The moment generating function is
$$\begin{aligned}
\phi(t_1,\ldots, t_k) &= E\left[\exp\left(t_1 X_1 + \cdots + t_k X_k\right)\right]\\
&= \sum_\mathbf{x} \binom{n}{\mathbf x} (p_1 e^{t_1 x_1}...
- 330k
7
votes
Accepted
R - multinomial logistic regression with relative frequencies as response variable
The regular multinomial regression model is usable as long as the model-fitting procedure accepts fractional responses and robust standard errors are used. This is known as the fractional multinomial ...
- 1,926
6
votes
Multinomial logistic regression vs one-vs-rest binary logistic regression
I don't think the previous answers really capture the key difference, although it is implicit in the discussion of Independence of Irrelevant Alternatives (which is a social sciences term rather than ...
- 7,266
6
votes
Deriving the MAP estimate for Multinomial-Dirichlet
You can impose the constraint $\sum \theta_i = 1$ by specifying $\theta_k = 1 - \sum_{i<k}\theta_i$ in your likelihood function. This results in:
$$l(\theta) = \sum_{i=1}^{k-1}(x_i+a_i-1)\log \...
- 40.6k
6
votes
Looking to see if random sample is uniform or not
You have seen 112*35 = 3920 colors. The expected frequency of each color is then 784. You can use a Chi-square test to see if the colors are randomly distributed in your sample.
You saw 790 Blue, ...
- 1,548
6
votes
Accepted
Distribution of number of objects in Simple Random Sampling with Replacement (SRSWR)
The probability that $K = k$ is given by
$$
p(k) = \frac{\binom{m}{k} f(n,k)}{m^n}
$$
where $f(n,k)$ is the number of sequences consisting of only the integers $i = 1, \ldots, k$ of length $n$ in ...
- 9,002
6
votes
Accepted
How to decide on whether it is a hypergeometric or a multinomial?
One important difference is that the hypergeometric distribution assumes sampling without replacement, and the multinomial assumes sampling with replacement.
A second important difference is that ...
- 57.1k
6
votes
Accepted
How to simulate Likert-scale data in R?
To perform the simulation, here is a one line solution using the sample function:
...
- 1,746
Only top scored, non community-wiki answers of a minimum length are eligible