Questions tagged [mixture-distribution]
A mixture distribution is one that is written as a convex combination of other distributions. Use the "compound-distributions" tag for "concatenations" of distributions (where a parameter of a distribution is itself a random variable).
495
questions
0
votes
2
answers
41
views
Posterior of binomial and mixed prior
I'm currently studying posterior distribution with likelihood $y|\theta \sim B(n,\theta)$ and mixture of prior distribution $\theta \sim \pi Beta(\alpha_1, \beta_1) + (1-\pi)Beta(\alpha_2, \beta_2)$. ...
- 101
0
votes
1
answer
107
views
Generate marginally dependent (with predetermined covariance) but conditionally independent data from a Mixture of Gaussians
Suppose you have three variables $y\in\{0,1\}$ and $x_1\in\mathbb{R}$ and $x_2\in\mathbb{R}$. I want to produce data with the following generative process which corresponds to a Mixture of Gaussians (...
- 316
0
votes
0
answers
11
views
Calculating Mean Vector and Covariance Matrix of Mixture of Multivariate Normal Distributions [duplicate]
In an effort to better understand multivariate normal distributions, I am attempting to derive the mean vector and covariance matrix of multivariate random vector defined by a mixture distribution. ...
0
votes
0
answers
7
views
Combining factors, represented as normal distributions, to one combined factor, normally distributed
I'm trying to combine the different factors that may affect running pace, such as GPS-measured distance, grade, terrain, heat and other factors (such as wind etc.). Each factor is represented as a ...
0
votes
0
answers
39
views
How do we obtain the posterior of a beta binomial mixture of continuous and a discrete density?
In section 3.6 of Jim Albert's 2009 book "Bayesian Computation with R" he describes a test of whether a coin is fair using a mixture of priors. The coin tossing follows a binomial ...
- 101
0
votes
2
answers
44
views
Flexmix maxima are not where they are expected to be
For my dataset I have plotted the density with ggplot.
As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus.
...
- 111
4
votes
2
answers
496
views
Kolmognorov Smirnov test p values is 0
I am trying to use Kolmogorov-Smirnov test to check the goodness of fit of the distributions for the dataset. I have dataset consisting of 100,000 samples and I apply expectation-maximization ...
- 85
0
votes
0
answers
14
views
Cluster based on random effects, STAN
I have a problem where I measure repeated responses in condition A and in condition B for a set of individuals $i=1,...,n$. I am interested in learning about the effect of the condition in the ...
- 448
0
votes
0
answers
31
views
Log likelihood of mixture distributions
If I have a gamma mixture distribution represented as,
$$f(x;\alpha,\beta) = \sum_{i=1}^{N}\pi_if(x;\alpha_i,\beta_i)$$
Where $\pi$ represents the weights of the $N$ components. For $N=2$ and the ...
- 85
1
vote
1
answer
58
views
Understanding gamma mixture model
I am trying to understand the gamma mixture models, especially the significance of the 'loc' parameter in scipy.stats. In the code below, I generate a mixture ...
- 85
0
votes
0
answers
5
views
Finding subgroups in population, using individual effects of hierarchical model
I want to know how to look for effects both at a population level, and at an individual level in an experiment. I was wondering if I can do this with hierarchical Bayesian models as follows.
In a ...
- 448
1
vote
0
answers
23
views
Bayesian conjugate updating when the likelihood can be approximated by a finite mixture of normals?
I'm facing a situation where I'd like to do Bayesian conjugate updating, but both the prior and the likelihood (a Student-t) can only be approximated by a finite mixture of normals. I know that a ...
- 29.3k
0
votes
0
answers
10
views
Infer a K-multinomial mixture model from "group-less" data
I want to infer a multinomial mixture model consisting of $K$ components where each component has the same number of trials, denoted as $N$. Such a multinomial mixture model can be written as:
$$
\...
- 422
0
votes
0
answers
58
views
Bayesian Gaussian mixture - is my prior correct?
I'd like to sample from the Bayesian Posterior of a Gaussian mixture model, but I am not sure about the correct Bayesian formulation of the latter. Is the following correct?
I consider the 1-...
- 1
2
votes
1
answer
98
views
Modeling outliers in maximum likelihood estimation with gradient descent
Consider a set of 3D points $X = \{x_1, x_2, ...x_n\} $ with $ x_i\in\mathbb{R}^3$ on which we want to fit an arbitrary probability distribution. The distribution we want to fit models some ...
- 5,576
0
votes
0
answers
18
views
Identifying a mixture model given the mixing and the outcome distribution
Consider a mixture model involving an observed random variable $Y$ taking values in $\mathbb{R}$ with cdf $F$, a mixing random variable $\alpha$ taking values in a set $\mathcal{A}$ with cdf $G$ and a ...
0
votes
0
answers
26
views
Is every piecewise distribution a mixture distribution?
If I have a piecewise distribution with density $f$ on $[0,1)$ so that $f=f_1$ on $[0,0.5)$ and $f=f_2$ on $[0.5,1)$, with both $f_1$ and $f_2$ densities, is it necessarily a mixture distribution? My ...
- 126
0
votes
0
answers
10
views
What assumptions are made about categorical variables in regression models? [duplicate]
This is the formalization for a continuous regression model as I understand it:
Assume that your outcome $Y$ is normally distributed. Assume that your predictors, $X_1, X_2, \cdots X_n$ are normally ...
- 1
0
votes
1
answer
22
views
Efficiently calculating comparisons between finite mixtures of Gaussians
Say I have two finite mixtures, each consisting of an equally weighted mix of $n$ Gaussians each with a known mean and standard deviation.
Is there an efficient method of calculating the probability ...
- 293
0
votes
0
answers
34
views
Bayes estimate of mixture of exponential under the the Kullback-Leibler divergence loss function
Model framework:
Suppose that the loss function is given by the Kullback-Leibler divergence (KLD) as follows:
\begin{equation}
\text{KL}(\Theta \parallel \hat{\Theta}) = \text{KL}\big(f(x;\Theta) \...
- 143
2
votes
1
answer
108
views
When two distributions overlap, how to separate one distribution from the mixture distribution if I the other distribution is known?
I have a data which can be classified into two groups. As you see, Figure(a) shows that they are easily classified into group A and B. However, sometimes they are overlapped and it is impossible set a ...
0
votes
0
answers
71
views
Hypothesis testing for random effect term in linear mixed model
In linear mixed model, we have
$$
y = X\alpha + Z u + \epsilon;
$$
which can be written in Multi-variate normal form:
$$
y \sim MVN(X\alpha, \tau_1V_1 + \tau_2V_2 + ... + \sigma^2 I) \\
\text{where } ...
- 313
0
votes
0
answers
25
views
Are marginal probabilities a mixture model of the conditional probabilities?
In a contingency table, is it correct to say that the marginal probabilities are a mixture model of the conditional probabilities?
Also, is it correct to say that these marginal and conditinal ...
- 11
0
votes
1
answer
50
views
Maximum likelihood estimate for mixture with components using both cartesian and polar coordinates
I have a set of points (x,y) that were generated from a mixture of two components: one component uses Cartesian coordinates, and the other polar coordinates.
For example, with probability $\gamma$ I ...
- 15
0
votes
0
answers
33
views
Mixture Model on Time Series with different number of observations on each time point
Suppose that we a dynamic series of values over time points $t=1,2,...,T$ and observations on each time point being $n_{t}$. So, we have a collection of values $y_{i,t}$ with $i=1,2,...,n_{t}$ that ...
- 2,142
0
votes
0
answers
48
views
Approximaion with uniform mixture density
Assume that a RV is drawn from a distribution with PDF $f(x)$. I would like to approximate this distribution as a mixture of infinitely many uniform distributions. Without loss of generality, assume ...
1
vote
0
answers
42
views
Are there any fast algorithm to estimate a Bernoulli mixture model?
In a problem I need to estimate a Bernoulli mixture model with 3 mixing components. More specifically, we have a random vector $\mathbf{D}=(D_1,D_2,D_3,D_4,D_5)$. $D_1,D_2,D_3,D_4,D_5$ are drawn ...
- 2,516
0
votes
0
answers
65
views
Loss Function for Neural Network with High density at 0
I am working on a time series project and looking to use a transformer based Neural Network (specifically, temporal fusion transformer).
My data is extremely heavily at 0 (the use case is that most ...
- 21
0
votes
0
answers
84
views
Is every Compound Poisson distribution a Mixture model?
We have two models:
Let $N \sim \hbox Poisson (\lambda)$ and let $(X_k ; k =1,2,3,...)$ be a a sequence of independent and identically distributed random objects (random variables, vectors or ...
- 997
0
votes
0
answers
88
views
Extracting distributions from R density function
I am looking for a not terribly complicated solution to deconvolute a series of overlapping distributions. The data may be noisy and I am considering using R's density function.
...
everett.jk
1
vote
1
answer
119
views
Maximum likelihood estimate for mixture of different distributions
I'd like to estimate the parameters of a mixture model using MLE. The density is:
$$
f(x,y) = \mathcal{N}(x, y; \boldsymbol{\mu}, \Sigma) \cdot \alpha + \mathcal{N}(x; \mu, \sigma^2) \cdot \mathcal{U}...
- 15
1
vote
1
answer
93
views
Generating random variables from a mixture of Normal distributions and Exponential distribution using composition method
How can I sample from a mixture distribution in particular a mixture of Normal distributions and Exponential distribution in R using composition method?
For instance if I want to sample from:
$0.3\...
- 39
0
votes
0
answers
82
views
Model Comparison within Bayesian Gaussian Mixture Model framework
Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model.
$$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
- 2,142
2
votes
1
answer
48
views
MLE for a mixture of betas without using EM algorithm
Suppose I have a mixture of two Beta densities say $f_1 = \text{Beta}(1,1)$ and $f_2= \text{Beta}(1,\beta)$ where $\beta$ is unknown. The sample $X_1,....,X_n$ is observed based on latent Bernoulli ...
- 217
1
vote
1
answer
40
views
What is the skewness of the difference between two binomial random variables?
I have two binomial random variables:
$$
X_1 \sim \text{Binom}(n_1,p_1), \\[6pt]
X_2 \sim \text{Binom}(n_2,p_2).
$$
I know that the individual skewnesses are:
$$
\mathbb{Skew}(X_1) = \frac{1 - 2p_1}{\...
- 123
0
votes
1
answer
13
views
How to test whether one cohort is a mixture of two other cohorts for a set of binary variables?
I'll illustrate my question with an example. Suppose I have 25 observations for 3 binary variables. The observations belong to 3 pre-labelled cohorts 1, 2, and 3. Let's say:
...
- 13
0
votes
1
answer
489
views
Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]
In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
0
votes
0
answers
58
views
Mixture distribution mean and variance
I'm currently studying zero-modified exponential distributions and working with neither continuous nor discrete distributions, specifically the following general case:
$$ F_{p,m}(x) = \begin{cases}
0, ...
- 13
1
vote
0
answers
23
views
What is the best way to calculate the fit of the mixture distribution to the actual data?
I have fitted the mixture of Gaussians to the natural log of the data. I know that the model is not a very good fit to the data in the tail region, however in the high density region the actual data ...
- 85
1
vote
1
answer
122
views
Is there a way to fit two distributions from exponential family and overcome discontinuity in parameters?
I have data that follows LogNormal distribution in the body (high density region) of the distribution and it seems like it has an Exponential tail after a particular 'cut' point. Given that these two ...
- 85
0
votes
0
answers
37
views
Probability of mixture distribution
Let
\begin{equation}
p_1=\Pr(0.8z_1^2+0.2z_2^2 > c), \qquad p_2=\Pr(0.2z_1^2+0.2z_2^2 > c),
\end{equation}
where $z_1,z_2 \sim N(0,1)$.
Then, can I have $p_1 >p_2$ since $0.8z_1^2+0.2z_2^2 &...
- 345
0
votes
0
answers
55
views
How to perform calculations when the integral of a mixed exponential distribution pdf does not give 1?
Let Y be the time, for a mixture distribution with two exponential distributions, each multiplied by a and b and having 2 different parameters. How can calculations such as mean, variance and ...
- 23
3
votes
1
answer
160
views
Maximum likelihood for mixture of Bernoullis with known mixture proportions
Given the hierarchical model
$$
\begin{align}
k & \sim \text{Categorical}(\pi_1, \dots, \pi_K) \\
X \mid k & \sim \text{Bernoulli}(\theta_{k})
\end{align}
$$
and an i.i.d. sample $X_1, \dots, ...
- 3,337
1
vote
1
answer
66
views
Understanding conditional probability formulas in the context of class-conditionals in generative models
I am trying to understand the theory behind probabilistic generative models a bit better.
If I model the class-conditionals as Gaussians, the formula is this:
$$
\frac{1}{2\pi^{\frac{D}{2}}|\Sigma|^...
- 219
1
vote
1
answer
71
views
Are discrete mixtures Gauss quadrature-like integral approximations?
I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution.
Let $p_{comp}(x)$ be the PDF of a compound ...
- 310
2
votes
0
answers
44
views
Bayesian inference when distribution depends on unobserved outcome with known distribution
Let's say we have an observed outcome $Y_i$ for an object $i=1,\ldots,I$ that arises like this:
For each object a coin is tossed (outcome $X_i$ = $H$ or $T$).
We know the coin is fair, so $X_i \sim \...
- 29.3k
0
votes
0
answers
102
views
Calculating Critical Value for Mixture of Two Chi-Square Distributions
Let's say I have a statistic X which is distributed as a mixture of two chi-square distributions; $$X \sim 0.5\chi^2_1 + 0.5\chi^2_2.$$
I'm wondering how I can calculate the critical value(and p-value)...
- 321
1
vote
1
answer
91
views
Mixtures vs Multi-level models?
I'm confused on how mixture models and multi-level models are different (if at all.) Are there general rules for when to use one and not the other, pros/cons, etc?
- 2,377
2
votes
0
answers
175
views
E-Step of the EM algorithm on mixture
Im trying to implement the EM algorithm on mixture model: $pg(x) + (1-p)h(x)$ where the sample $\bar{x} = (x_1, \ldots, x_n)$, is independently generated from the mixture, and $g(x) = e^{-x}$ and $h(x)...
- 161
1
vote
0
answers
381
views
Likelihood in mixture models
As per my understanding, normally, when we talk about Bayes rule, we write:
p(z|x) = [p(x|z) * p(z)] / p(x)
where,
p(z|x) is called posterior
p(x|z) is called likelihood
p(z) is called prior
p(x) is ...
- 411