Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
589
questions
1
vote
1
answer
11
views
Is there a relationship between the number of the mixture components and the overfiting of the model?
I read the following:
To prevent overfitting we would like to work with as few components as possible".
How does the number of the mixture component affect the fit of the model? Is that because ...
- 1,116
0
votes
1
answer
19
views
Can the gaussian mixture model combined in clustering?
Suppose I have a data with two clusters. Suppose further that I cluster the data using, for example, K-means. Then, can I fit a mixture model to each cluster? That is, can I fit a gaussian mixture ...
- 477
2
votes
1
answer
14
views
Determine if high dimensional data is multimodal
I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n.
What are ...
- 21
1
vote
0
answers
41
views
Is obtaining maximum likelihood estimate of Gaussian mixture via clustering possible?
Let's say I have a data set $ X = \{ x_1, \dots, x_n \}$ with underlying probability density
$$ p(x; \mu_1, \sigma_1^2, \dots, \mu_k, \sigma_k^2) = \sum_{i=1}^k \alpha_i p(x; \mu_i, \sigma_i^2), \quad ...
- 11
1
vote
0
answers
34
views
Likelihood in mixture models
As per my understanding, normally, when we talk about Bayes rule, we write:
p(z|x) = [p(x|z) * p(z)] / p(x)
where,
p(z|x) is called posterior
p(x|z) is called likelihood
p(z) is called prior
p(x) is ...
- 161
0
votes
1
answer
22
views
What is the average bias in MLE of 2-component univariate Gaussian mixture model?
Imagine that you have a standard 2-component univariate Gaussian mixture model:
$$p(x_i∣θ)=(1-λ)N(x_i|μ_1,σ_1^2 )+λN(x_i|μ_2,σ_2^2 )$$
$$θ=\{μ_1,μ_2,σ_1,σ_2,λ\}$$
$$L(θ;x)=∏_{i=1}^N p(x_i |θ)$$
The ...
2
votes
0
answers
27
views
AIC vs BIC for time series clustering and descriptive purposes
I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
- 21
0
votes
0
answers
18
views
Test for gaussian mixture fit when component assignment is known?
I have a process 𝑃 generating random variables X_1, ... X_n. From each of these I've sampled a set of samples ...
- 191
0
votes
0
answers
19
views
MLE for Two component mixture model
Chapter 8 section 8.5.1 of the Elements of Statistical Learning book describes a simple mixture model for density estimation and the associated EM algorithm for carrying out maximum likelihood ...
- 123
0
votes
1
answer
11
views
How to obtain the p-values of a gamlssMX model?
I am working with a dataset that includes a binary target variable (0 or 1).
I have built a model with the gamlssMX() function included on the "gamlss.mx" package to explain a continuous ...
0
votes
0
answers
12
views
Are gaussian mixture models for clustering robust to data sparsity?
I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
- 101
0
votes
0
answers
33
views
How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?
Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
- 1
0
votes
0
answers
17
views
In a mixture model should I update the parameters of variance jointly or one-by-one?
Suppose that I have the following mixture model, where I know the true values of $(\pi_{1},\pi_{2},\pi_{3},\mu_{1},\mu_{2},\mu_{3})$ (I know them for the simulation that I build)
$$f(x) = \pi_{1}N(x;\...
- 1,681
0
votes
0
answers
20
views
How to conduct EM algorithm when there are some outliers in GMM Models?
I'm just confused about the problem of adding an outlier component directly to the primary form of GMM models:
Suppose that the observed data contains several outliers. The mixture model could be:
$$
...
- 1
2
votes
0
answers
27
views
High dimensional behavior of Dirichlet Process-based clustering?
I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed.
Let $D$ be the dimensionality ...
- 699
2
votes
0
answers
25
views
Confusion Regarding Bayesian Mixture of Gaussians
Following is the screenshot from the paper "Variational Inference: A Review for Statisticians".
I am having confusion understanding equations (7) and (8).
Can anyone please let me know ...
- 161
0
votes
0
answers
26
views
Selecting the optimal bandwidth in kernel density estimation
I have a question regarding kernel density estimation.
At the moment I have a set of sample date $V$, where for each $v \in V$ I have an associated standard deviation $\sigma_v$ (some measurements ...
- 1
1
vote
1
answer
112
views
Fitting truncated normal mixtures in R
I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
- 179
0
votes
0
answers
14
views
Conditional distribution of GMM with given distribution?
I'm currently looking for the propagation of Gaussian mixture noise in a linear system.
I have a question regards how to calculate the following equations:
Suppose there is an GMM $f_1(x,y)$ = $\sum_{...
- 1
3
votes
1
answer
106
views
Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?
In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of
Multimodality of the Likelihood. ...
- 3,778
0
votes
0
answers
33
views
My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?
I use DeLonge method to compare two ROC AUCS. The result of it is Z-score.
Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
- 1
1
vote
1
answer
74
views
Why cannot MLE be implemented for Gaussian mixture model directly?
Consider the following density, the mixture of two Gaussian distributions,
\begin{align*}
p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) ,
\end{align*}
where $p(k=1)+p(k=0)=\pi_1+\...
- 95
0
votes
1
answer
98
views
Creating a probability density function from a Gaussian Mixture Model
I have some daily timeseries (27 right now but will be over 200 when I get more data) for electricity consumption. For each hour I want to know what the probability density function looks like.
What I ...
- 35
0
votes
0
answers
28
views
What is the number of free parameters in an n-component GMM?
I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom.
If I am fitting guassian mixture model to the data, ...
- 1
0
votes
2
answers
106
views
Is using a fixed random seed in production okay?
I have a dataset and am trying to use GMM to cluster it. The algorithm works well but when I run it multiple times I get different results. While the clusters produced in each run are valid my users ...
- 11
0
votes
0
answers
77
views
How can I fit a 1d, two-component Gaussian mixture with very uneven weights?
Say I have data generated by, with probability $p$, sampling from one normal distribution, and with probability $1-p$, from a different normal distribution. I would like to estimate the means of the ...
- 389
2
votes
1
answer
115
views
PCA as Pre-Processing before Clustering through GMM
Suppose I start with a data matrix $X \in \mathbb{R}^{N \times D}$, where each row $x_i$ is $D$-dimensional sample. I would like to cluster this data through a Gaussian Mixture Model (GMM).
If I pre-...
- 149
0
votes
0
answers
11
views
mixture of finite regressions without a response variable
In finite mixture modelling, in particular mixture of regressions modelling, we are interested in finding latent trajectories against a response variable. But what if there is no known response ...
- 155
0
votes
0
answers
27
views
Generating a simulated dataset from a mixture of two-trait Gaussian distributions
How can I generate a random variable which follows the mixture Gaussian distributions:
I found the answer and tried to simulate it, but I think it is not the right dataset
I wanted.
Here is my code:
<...
- 23
0
votes
1
answer
72
views
Why Gibbs Sampling for mixture models?
I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
0
votes
1
answer
34
views
For univariate data, why do we need the normalmixEM function in R instead of just computing the mean and variance with the basic methods?
I can understand why if from your univariate data (1 column?) you plot a histogram which seems to have 2+ peaks ie a mix of more than 1 gaussian. But what if you plot a histogram and there looks to be ...
- 5
1
vote
0
answers
58
views
Problems with convergence of the EM algorithm for a gaussian mixture regression
I have been implementing a EM-algorithm for a latent-class regression model, where every individual has a vector of observations.
Currenly, I have the problem that the model does not converge. The log ...
- 21
2
votes
0
answers
60
views
What is the distribution of the Poisson Sum of gaussians?
I know that the sum of two independent normal random variables is normal. Particulary, when one is copy of other, i.e., if $X_1, X_2 \sim \mathcal{N}(0,\sigma^2)$, independent, we have:
$$X_1 + X_2 \...
- 223
2
votes
2
answers
80
views
what is the difference between mixture of two normal distributions and sum of two independent variables
The following denotes a mixture of a standard normal with a
normal with the same mean but 100 times the variance:
$0.95 \mathrm{~N}(0,1)+ 0.05 \mathrm{~N}(0,100)$
Let Y = 0.95 X + 0.05 Z with X,Z are ...
1
vote
0
answers
63
views
MAD & Median of weighted GMM
What is the median and median-absolute-deviation of a weighted GMM in terms of component mean and variance? For example, three normal distributions $A$, $B$, $C$ with means $\mu_a,\mu_b,\mu_c$, ...
- 93
1
vote
1
answer
187
views
Derivation of the M-step in EM algorithm for a three-dimensional panel mixture model
I have a question regarding the estimation of a latent-class gaussian mixture model, where the model is for three dimensional panel data set with individuals $i$, in country $j$ in time $t$.
I want ...
- 21
1
vote
0
answers
39
views
A mixture of discrete and continuous components
Suppose that random variable $X$ is sampled from Bernoulli with probability $\pi$. Let $Y\sim N(\mu, \sigma^2)$ and denote $Z=XY$. Then, when $X=0$, $Z$ equals 0 (and this happens with probability $1-\...
- 789
0
votes
0
answers
86
views
Is there any way to perform MLE on a 1d gaussian mixture model comprised of two gaussians?
The MLE of the mean of a single gaussian distribution is the mean of the sample. But is there any way to do this when you have two gaussians such as in the figure attached? I know how to estimate the ...
15
votes
2
answers
669
views
Finding category with maximum likelihood method
Let's say that we had an information for men and women heights.
R code:
...
- 153
1
vote
0
answers
23
views
What determines performance in recoverying K in Gaussian Mixture Model?
My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm.
For simplicity, let's consider the case in ...
- 653
1
vote
1
answer
34
views
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Recently, I came across the use of Gaussian Mixture Distributions being used to model the response ...
- 5,922
2
votes
0
answers
86
views
How to calculate the covariance matrix in (Xu and Knight 2010)?
Setup
I'm reading (Xu and Knight 2010), which is a paper about estimating finite Gaussian mixture models using the CECF (Continuous Empirical Characteristic Function) method. The basic idea is to ...
- 148
5
votes
1
answer
525
views
K Means as a special case of GMM (using EM Algorithm)
I am looking for a tutorial/gentle introduction (preferably with mathematics/proofs) on K-means as a special case of Gaussian Mixture Model using the EM Algorithm.
I have found this: https://www....
- 7,875
0
votes
0
answers
133
views
Compute log-likelihood in Bernoulli Gaussian Mixture
I'm working on this exercise about Gaussian Mixtures:
Here's part of the solution:
I don't understand how they came up with this equation for the log-likelihood (red arrow).
From what I know, the ...
- 1
0
votes
0
answers
26
views
Help students understand Gaussian Kernel in mean shift segmentation
I am creating a image mean shift segmentation algorithm for a class.
So far what I've done is basically
Iterate through each point/pixel on the image and with a given bandwidth(radius) calculate the ...
1
vote
2
answers
223
views
Are linear combinations (i.e. "sums") of gaussian distributions also gaussian?
I have always assumed that this is fact : but are there any mathematical theorems that state: finite sums of gaussian distributions are gaussian themselves?
In the above picture, does p(x) have a ...
- 5,922
0
votes
0
answers
95
views
Conjugate prior bayesian inference on multivariate GMM
I am trying to understand how the posterior looks like when running Bayesian inference on a multivariate Gaussian-mixture model.
$p(\mathbf{x}) \propto \sum_{i=1}^M w_iN(\mathbf{x}|\mu_i,\Sigma_i)$.
...
- 1
0
votes
0
answers
14
views
Is there a formula for the optimal weights for a Gaussian mixture model when the components are considered "correct"?
For simplicity consider two models predicting $Y=Z_0 + \sigma Z_1$, where $Z_i$ are independent $\mathcal{N}(0, 1)$ and each model only take one of the $Z_i$ as a factor. So we have $Y|Z_0$ and $Y|Z_1$...
- 238
1
vote
1
answer
161
views
Assumption of Gaussian mixture model [closed]
I am still confused when I read about the Gaussian Mixture Model and how does it work.
One thing that I do not understand is that "GMM assumes the data is a mixture of Gaussian or
i.i.d (...
- 173
0
votes
0
answers
13
views
Are the subsets of Mixture Models necessarily parametrized?
Im just learning about mixture models and they are described as a "mix of parametric and non parametric models".
My question is how a non-parametric subset would look like? How would e.g. EM-...
- 1