Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
607
questions
1
vote
0
answers
16
views
What does mixture refer to in LCMM and HLME in R
I am trying to use the HLME and LCMM functions to fit latent class mixed models to my data. Here are the documentations to both of them:
https://www.rdocumentation.org/packages/lcmm/versions/1.8.1.1/...
- 183
1
vote
1
answer
51
views
Gaussian mixture model probabilities
I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
- 113
0
votes
0
answers
36
views
Ask for help understanding a sentence in Bishop's PRML book on soft weight sharing
It's about section 5.5.7 of Christopher M. Bishop's "Pattern Recognition and Machine Learning" on soft weight sharing. The sentence is the first three lines of page 271. First the author ...
- 183
1
vote
0
answers
19
views
How to solve gamma from GMM
GMM refers to the Gaussian mixture model, cf. here.
Suppose we have $N$ data points (or observations in Statistics) and $K$ Gaussian models to mix. After going through Maximum Likelihood Estimation, ...
- 177
4
votes
1
answer
275
views
Mixture of Gaussian is not log-concave
I've encountered the statement:
For $p\in(0,1),$ the location mixture of standard univariate normal densities
$f(x)=p\phi(x)+(1-p)\phi(x-\mu)$ is log-concave if and only if $\Vert\mu\Vert \leq 2.$
...
- 163
0
votes
0
answers
9
views
Do the Mixing Coefficients in a Gaussian Mixture Model Depend on the Datapoint?
Given a Gaussian mixture model with $K$ clusters, the probability of sampling a point $x\in\mathbb{R}^d$ is given by
$$
p(x) = \sum_{k=1}^K\pi_k\mathcal{N}(x;\boldsymbol\mu_k,\boldsymbol\sigma_k)
$$
...
- 101
0
votes
1
answer
29
views
Mixture of Two Normals
Suppose we have a data which consists of two normals,
x = rnorm(50,mean=1,sd=2)
y = rnorm(50,mean=2,sd=3)
z = sample( c(x,y) , size = 100, replace=FALSE )
The goal ...
- 1,365
0
votes
0
answers
33
views
Model Comparison within Bayesian Gaussian Mixture Model framework
Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model.
$$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
- 2,028
7
votes
1
answer
166
views
EM algorithm for Bivariate Normal
Consider a random sample $X_i = (U_i,V_i)$ where $i=1,2,...,n$ from a bivariate normal population with mean $(\mu_1,\mu_2)$ and variances $(\sigma_1 ^2, \sigma_2 ^2)$ and correlation $\rho$. Let's ...
- 217
0
votes
1
answer
103
views
Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]
In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
0
votes
0
answers
9
views
EM algorithm for finding discrete latent variables with gaussian child in Python
I have the following Bayesian network model:
$P(A, B, C, D) = P(D|B)P(C|B)P(B|A)P(A)$
A and B are discrete variables, while C and D are continuous variables. I've looked around and it seems in a ...
- 61
0
votes
0
answers
23
views
Inferring class probabilities in Gaussian Mixture Model
Let's say I go shopping and always buy a few fruits and a few candies. Candies are typically more expensive than fruits. The prices of fruits and candies are both well modeled by gaussian ...
- 1,739
1
vote
0
answers
13
views
What is the best way to calculate the fit of the mixture distribution to the actual data?
I have fitted the mixture of Gaussians to the natural log of the data. I know that the model is not a very good fit to the data in the tail region, however in the high density region the actual data ...
- 75
1
vote
1
answer
48
views
Closed form solution of multivariate Gaussian over mixture of multivariate Gaussians
Suppose I have three variables $X_{1}, X_{2}$ and $Y$, where $X_{1}, X_{2}$ are continuous and $Y$ is binary. The conditional distribution of $X_{1}, X_{2}$, given $Y$ is a multivariate Gaussian ...
- 306
1
vote
1
answer
43
views
Coefficients of Gaussian mixture
This is in context of Gaussian mixtures
$$p(\boldsymbol{x}) = \sum_{k=1}^K \pi_k\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})$$
Bishop mentions on Page-111
Also, the requirement ...
- 131
2
votes
2
answers
221
views
p(x) in Gaussian mixture model
I've recently learned about Gaussian mixture models (GMM) in school. The formula for a GMM goes something like this:
$$p(x) = \sum_{k=1}^{K} \pi_i\mathcal{N}(x|\mu_i,\,\sigma^{2}_i)$$
Now, I know from ...
- 123
1
vote
0
answers
13
views
When imputing missing labels Y1 == NaN during training, how do additional target vectors (Y2 != NaN) impact learning Y1==NaN?
I am training a Mixture Density Network (MDN) to map from continuous input vectors X to continuous targets Y [i.e. X -> Y]. There are missing labels on vector Yi, which I impute from the mdn (as ...
- 11
0
votes
1
answer
50
views
Could the likelihood increase monotonically in a misspecified EM algorithm?
I am dealing with the estimation of a Gaussian Hidden Markov Model with conditional distribution given the first-order Markov state $S_t = j,\ j=1,...,J$
$$
Y_t|S_t=j\sim N(0,\sigma^2_j)
$$
where the ...
- 319
0
votes
0
answers
20
views
How to use GMM for clustering?
After running FAMD, the scatterplot of PC1 vs. PC2 looks like this:
It seems like if I want to do a clustering on this data, GMM is the best option (if not, please let me know what to use).
Using BIC,...
- 91
0
votes
0
answers
23
views
Metric to check the number of clusters in one dimentional data
I have a set of 1D data as shown below.
I want to find a metric that represents the number of clusters in data.
Is there any suitable metric that matches my criteria?
Example 1D data list:
[68, 3, 3, ...
- 1
0
votes
0
answers
13
views
Is there methods for splitting a Guassian Mixture model into separate unimodal distributions?
I want to apply an algorithm which usually works on unimodal distributions on multimodal distributions, I'm considering doing this by splitting the multimodal distributions into multiple unimodal ...
1
vote
0
answers
9
views
How do I make a table that prints the Mean and its corresponding weight of a Gaussian Mixture model?
I am successfully generating weights and means for a GMM, but I'm trying to get the results to print in a way that shows the corresponding mean for a weight. Here's what I have now:
...
- 11
0
votes
0
answers
27
views
In what situations can you use EM, but where you cannot compute the marginal likelihood?
Expectation Maximisation is used to find the parameters of a hierarchical model with some nuisance parameters, that need to be integrated out. The typical example is a Gaussian Mixture Model, where ...
- 101
1
vote
0
answers
55
views
Finding Probability that data is taken from a Gaussian Mixture Model
I'm using scikit's Gaussian Mixture to obtain a 2 component mixture model for some 1-D data. Having obtained a model, I want to test how well a different data set matches this mixture model, or more ...
- 33
0
votes
0
answers
49
views
Random Intercept in a Random Coefficient Logit
I want to estimate a random coefficients logit where the intercept is random and normally distributed, not the slopes of the variables. I understand this boils down a "fixed effects logit model&...
- 111
1
vote
1
answer
225
views
Calculating weighted covariance matrix of a weighted finite mixture of multivariate normal distributions
I am trying to calculate the weighted covariance matrix for a finite mixture of multivariate normal distributions. I read this post here and this one here, but the first post is focused on uniformly ...
4
votes
0
answers
113
views
Can GMM approximate any given probability density function?
I am currently studying on Bayesian models, and still new to probability theory.
I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of ...
- 63
4
votes
2
answers
161
views
Interpolating Gaussian mixture. How many points is sufficient for complete reconstruction
Consider the following Gaussian mixture of $N$ components:
\begin{align}
f(x)= \sum_{i=1}^N p (s_i) e^{-\frac{(x-s_i)^2}{2}}\big/ \sqrt{2\pi}
\end{align}
where we assume that
$\max_{i} |s_i| \le C$
...
- 215
2
votes
1
answer
28
views
Is there a relationship between the number of the mixture components and the overfiting of the model?
I read the following:
To prevent overfitting we would like to work with as few components as possible".
How does the number of the mixture component affect the fit of the model? Is that because ...
- 1,368
1
vote
1
answer
32
views
Can the gaussian mixture model combined in clustering?
Suppose I have a data with two clusters. Suppose further that I cluster the data using, for example, K-means. Then, can I fit a mixture model to each cluster? That is, can I fit a gaussian mixture ...
- 507
2
votes
1
answer
39
views
Determine if high dimensional data is multimodal
I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n.
What are ...
- 21
1
vote
0
answers
62
views
Is obtaining maximum likelihood estimate of Gaussian mixture via clustering possible?
Let's say I have a data set $ X = \{ x_1, \dots, x_n \}$ with underlying probability density
$$ p(x; \mu_1, \sigma_1^2, \dots, \mu_k, \sigma_k^2) = \sum_{i=1}^k \alpha_i p(x; \mu_i, \sigma_i^2), \quad ...
- 31
1
vote
0
answers
205
views
Likelihood in mixture models
As per my understanding, normally, when we talk about Bayes rule, we write:
p(z|x) = [p(x|z) * p(z)] / p(x)
where,
p(z|x) is called posterior
p(x|z) is called likelihood
p(z) is called prior
p(x) is ...
- 401
0
votes
1
answer
52
views
What is the average bias in MLE of 2-component univariate Gaussian mixture model?
Imagine that you have a standard 2-component univariate Gaussian mixture model:
$$p(x_i∣θ)=(1-λ)N(x_i|μ_1,σ_1^2 )+λN(x_i|μ_2,σ_2^2 )$$
$$θ=\{μ_1,μ_2,σ_1,σ_2,λ\}$$
$$L(θ;x)=∏_{i=1}^N p(x_i |θ)$$
The ...
2
votes
0
answers
43
views
AIC vs BIC for time series clustering and descriptive purposes
I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
- 21
0
votes
0
answers
28
views
Test for gaussian mixture fit when component assignment is known?
I have a process 𝑃 generating random variables X_1, ... X_n. From each of these I've sampled a set of samples ...
- 191
0
votes
0
answers
88
views
MLE for Two component mixture model
Chapter 8 section 8.5.1 of the Elements of Statistical Learning book describes a simple mixture model for density estimation and the associated EM algorithm for carrying out maximum likelihood ...
- 133
0
votes
1
answer
47
views
How to obtain the p-values of a gamlssMX model?
I am working with a dataset that includes a binary target variable (0 or 1).
I have built a model with the gamlssMX() function included on the "gamlss.mx" package to explain a continuous ...
0
votes
0
answers
28
views
Are gaussian mixture models for clustering robust to data sparsity?
I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
- 101
0
votes
0
answers
46
views
How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?
Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
- 1
0
votes
0
answers
32
views
In a mixture model should I update the parameters of variance jointly or one-by-one?
Suppose that I have the following mixture model, where I know the true values of $(\pi_{1},\pi_{2},\pi_{3},\mu_{1},\mu_{2},\mu_{3})$ (I know them for the simulation that I build)
$$f(x) = \pi_{1}N(x;\...
- 2,028
0
votes
0
answers
36
views
How to conduct EM algorithm when there are some outliers in GMM Models?
I'm just confused about the problem of adding an outlier component directly to the primary form of GMM models:
Suppose that the observed data contains several outliers. The mixture model could be:
$$
...
- 1
2
votes
0
answers
35
views
High dimensional behavior of Dirichlet Process-based clustering?
I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed.
Let $D$ be the dimensionality ...
- 741
2
votes
0
answers
31
views
Confusion Regarding Bayesian Mixture of Gaussians
Following is the screenshot from the paper "Variational Inference: A Review for Statisticians".
I am having confusion understanding equations (7) and (8).
Can anyone please let me know ...
- 401
0
votes
0
answers
110
views
Selecting the optimal bandwidth in kernel density estimation
I have a question regarding kernel density estimation.
At the moment I have a set of sample date $V$, where for each $v \in V$ I have an associated standard deviation $\sigma_v$ (some measurements ...
- 1
2
votes
1
answer
214
views
Fitting truncated normal mixtures in R
I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
- 371
4
votes
1
answer
187
views
Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?
In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of
Multimodality of the Likelihood. ...
- 3,930
0
votes
0
answers
119
views
My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?
I use DeLonge method to compare two ROC AUCS. The result of it is Z-score.
Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
- 1
2
votes
1
answer
262
views
Why cannot MLE be implemented for Gaussian mixture model directly?
Consider the following density, the mixture of two Gaussian distributions,
\begin{align*}
p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) ,
\end{align*}
where $p(k=1)+p(k=0)=\pi_1+\...
- 105
0
votes
1
answer
391
views
Creating a probability density function from a Gaussian Mixture Model
I have some daily timeseries (27 right now but will be over 200 when I get more data) for electricity consumption. For each hour I want to know what the probability density function looks like.
What I ...
- 35