Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
589
questions
0
votes
0
answers
5
views
How to use GMM for clustering?
After running FAMD, the scatterplot of PC1 vs. PC2 looks like this:
It seems like if I want to do a clustering on this data, GMM is the best option (if not, please let me know what to use).
Using BIC,...
- 91
0
votes
0
answers
22
views
Metric to check the number of clusters in one dimentional data
I have a set of 1D data as shown below.
I want to find a metric that represents the number of clusters in data.
Is there any suitable metric that matches my criteria?
Example 1D data list:
[68, 3, 3, ...
- 1
0
votes
0
answers
9
views
Is there methods for splitting a Guassian Mixture model into separate unimodal distributions?
I want to apply an algorithm which usually works on unimodal distributions on multimodal distributions, I'm considering doing this by splitting the multimodal distributions into multiple unimodal ...
1
vote
0
answers
7
views
How do I make a table that prints the Mean and its corresponding weight of a Gaussian Mixture model?
I am successfully generating weights and means for a GMM, but I'm trying to get the results to print in a way that shows the corresponding mean for a weight. Here's what I have now:
...
- 11
0
votes
0
answers
26
views
In what situations can you use EM, but where you cannot compute the marginal likelihood?
Expectation Maximisation is used to find the parameters of a hierarchical model with some nuisance parameters, that need to be integrated out. The typical example is a Gaussian Mixture Model, where ...
- 101
1
vote
0
answers
20
views
Finding Probability that data is taken from a Gaussian Mixture Model
I'm using scikit's Gaussian Mixture to obtain a 2 component mixture model for some 1-D data. Having obtained a model, I want to test how well a different data set matches this mixture model, or more ...
- 33
0
votes
0
answers
34
views
Random Intercept in a Random Coefficient Logit
I want to estimate a random coefficients logit where the intercept is random and normally distributed, not the slopes of the variables. I understand this boils down a "fixed effects logit model&...
- 101
1
vote
1
answer
99
views
Calculating weighted covariance matrix of a weighted finite mixture of multivariate normal distributions
I am trying to calculate the weighted covariance matrix for a finite mixture of multivariate normal distributions. I read this post here and this one here, but the first post is focused on uniformly ...
4
votes
0
answers
77
views
Can GMM approximate any given probability density function?
I am currently studying on Bayesian models, and still new to probability theory.
I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of ...
- 63
4
votes
2
answers
145
views
Interpolating Gaussian mixture. How many points is sufficient for complete reconstruction
Consider the following Gaussian mixture of $N$ components:
\begin{align}
f(x)= \sum_{i=1}^N p (s_i) e^{-\frac{(x-s_i)^2}{2}}\big/ \sqrt{2\pi}
\end{align}
where we assume that
$\max_{i} |s_i| \le C$
...
- 203
1
vote
1
answer
16
views
Is there a relationship between the number of the mixture components and the overfiting of the model?
I read the following:
To prevent overfitting we would like to work with as few components as possible".
How does the number of the mixture component affect the fit of the model? Is that because ...
- 1,180
1
vote
1
answer
25
views
Can the gaussian mixture model combined in clustering?
Suppose I have a data with two clusters. Suppose further that I cluster the data using, for example, K-means. Then, can I fit a mixture model to each cluster? That is, can I fit a gaussian mixture ...
- 507
2
votes
1
answer
28
views
Determine if high dimensional data is multimodal
I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n.
What are ...
- 21
1
vote
0
answers
52
views
Is obtaining maximum likelihood estimate of Gaussian mixture via clustering possible?
Let's say I have a data set $ X = \{ x_1, \dots, x_n \}$ with underlying probability density
$$ p(x; \mu_1, \sigma_1^2, \dots, \mu_k, \sigma_k^2) = \sum_{i=1}^k \alpha_i p(x; \mu_i, \sigma_i^2), \quad ...
- 31
1
vote
0
answers
83
views
Likelihood in mixture models
As per my understanding, normally, when we talk about Bayes rule, we write:
p(z|x) = [p(x|z) * p(z)] / p(x)
where,
p(z|x) is called posterior
p(x|z) is called likelihood
p(z) is called prior
p(x) is ...
- 161
0
votes
1
answer
37
views
What is the average bias in MLE of 2-component univariate Gaussian mixture model?
Imagine that you have a standard 2-component univariate Gaussian mixture model:
$$p(x_i∣θ)=(1-λ)N(x_i|μ_1,σ_1^2 )+λN(x_i|μ_2,σ_2^2 )$$
$$θ=\{μ_1,μ_2,σ_1,σ_2,λ\}$$
$$L(θ;x)=∏_{i=1}^N p(x_i |θ)$$
The ...
2
votes
0
answers
33
views
AIC vs BIC for time series clustering and descriptive purposes
I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
- 21
0
votes
0
answers
20
views
Test for gaussian mixture fit when component assignment is known?
I have a process 𝑃 generating random variables X_1, ... X_n. From each of these I've sampled a set of samples ...
- 191
0
votes
0
answers
46
views
MLE for Two component mixture model
Chapter 8 section 8.5.1 of the Elements of Statistical Learning book describes a simple mixture model for density estimation and the associated EM algorithm for carrying out maximum likelihood ...
- 133
0
votes
1
answer
30
views
How to obtain the p-values of a gamlssMX model?
I am working with a dataset that includes a binary target variable (0 or 1).
I have built a model with the gamlssMX() function included on the "gamlss.mx" package to explain a continuous ...
0
votes
0
answers
19
views
Are gaussian mixture models for clustering robust to data sparsity?
I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
- 101
0
votes
0
answers
38
views
How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?
Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
- 1
0
votes
0
answers
21
views
In a mixture model should I update the parameters of variance jointly or one-by-one?
Suppose that I have the following mixture model, where I know the true values of $(\pi_{1},\pi_{2},\pi_{3},\mu_{1},\mu_{2},\mu_{3})$ (I know them for the simulation that I build)
$$f(x) = \pi_{1}N(x;\...
- 1,913
0
votes
0
answers
30
views
How to conduct EM algorithm when there are some outliers in GMM Models?
I'm just confused about the problem of adding an outlier component directly to the primary form of GMM models:
Suppose that the observed data contains several outliers. The mixture model could be:
$$
...
- 1
2
votes
0
answers
31
views
High dimensional behavior of Dirichlet Process-based clustering?
I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed.
Let $D$ be the dimensionality ...
- 709
2
votes
0
answers
28
views
Confusion Regarding Bayesian Mixture of Gaussians
Following is the screenshot from the paper "Variational Inference: A Review for Statisticians".
I am having confusion understanding equations (7) and (8).
Can anyone please let me know ...
- 161
0
votes
0
answers
70
views
Selecting the optimal bandwidth in kernel density estimation
I have a question regarding kernel density estimation.
At the moment I have a set of sample date $V$, where for each $v \in V$ I have an associated standard deviation $\sigma_v$ (some measurements ...
- 1
1
vote
1
answer
150
views
Fitting truncated normal mixtures in R
I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
- 179
3
votes
1
answer
139
views
Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?
In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of
Multimodality of the Likelihood. ...
- 3,850
0
votes
0
answers
80
views
My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?
I use DeLonge method to compare two ROC AUCS. The result of it is Z-score.
Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
- 1
1
vote
1
answer
122
views
Why cannot MLE be implemented for Gaussian mixture model directly?
Consider the following density, the mixture of two Gaussian distributions,
\begin{align*}
p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) ,
\end{align*}
where $p(k=1)+p(k=0)=\pi_1+\...
- 95
0
votes
1
answer
191
views
Creating a probability density function from a Gaussian Mixture Model
I have some daily timeseries (27 right now but will be over 200 when I get more data) for electricity consumption. For each hour I want to know what the probability density function looks like.
What I ...
- 35
0
votes
0
answers
61
views
What is the number of free parameters in an n-component GMM?
I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom.
If I am fitting guassian mixture model to the data, ...
- 1
0
votes
2
answers
227
views
Is using a fixed random seed in production okay?
I have a dataset and am trying to use GMM to cluster it. The algorithm works well but when I run it multiple times I get different results. While the clusters produced in each run are valid my users ...
- 11
0
votes
0
answers
173
views
How can I fit a 1d, two-component Gaussian mixture with very uneven weights?
Say I have data generated by, with probability $p$, sampling from one normal distribution, and with probability $1-p$, from a different normal distribution. I would like to estimate the means of the ...
- 409
2
votes
1
answer
198
views
PCA as Pre-Processing before Clustering through GMM
Suppose I start with a data matrix $X \in \mathbb{R}^{N \times D}$, where each row $x_i$ is $D$-dimensional sample. I would like to cluster this data through a Gaussian Mixture Model (GMM).
If I pre-...
- 149
0
votes
0
answers
12
views
mixture of finite regressions without a response variable
In finite mixture modelling, in particular mixture of regressions modelling, we are interested in finding latent trajectories against a response variable. But what if there is no known response ...
- 167
0
votes
0
answers
61
views
Generating a simulated dataset from a mixture of two-trait Gaussian distributions
How can I generate a random variable which follows the mixture Gaussian distributions:
I found the answer and tried to simulate it, but I think it is not the right dataset
I wanted.
Here is my code:
<...
- 23
0
votes
1
answer
112
views
Why Gibbs Sampling for mixture models?
I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
0
votes
1
answer
36
views
For univariate data, why do we need the normalmixEM function in R instead of just computing the mean and variance with the basic methods?
I can understand why if from your univariate data (1 column?) you plot a histogram which seems to have 2+ peaks ie a mix of more than 1 gaussian. But what if you plot a histogram and there looks to be ...
- 5
1
vote
0
answers
92
views
Problems with convergence of the EM algorithm for a gaussian mixture regression
I have been implementing a EM-algorithm for a latent-class regression model, where every individual has a vector of observations.
Currenly, I have the problem that the model does not converge. The log ...
- 21
2
votes
0
answers
72
views
What is the distribution of the Poisson Sum of gaussians?
I know that the sum of two independent normal random variables is normal. Particulary, when one is copy of other, i.e., if $X_1, X_2 \sim \mathcal{N}(0,\sigma^2)$, independent, we have:
$$X_1 + X_2 \...
- 222
3
votes
3
answers
195
views
what is the difference between mixture of two normal distributions and sum of two independent variables
The following denotes a mixture of a standard normal with a
normal with the same mean but 100 times the variance:
$0.95 \mathrm{~N}(0,1)+ 0.05 \mathrm{~N}(0,100)$
Let Y = 0.95 X + 0.05 Z with X,Z are ...
1
vote
0
answers
66
views
MAD & Median of weighted GMM
What is the median and median-absolute-deviation of a weighted GMM in terms of component mean and variance? For example, three normal distributions $A$, $B$, $C$ with means $\mu_a,\mu_b,\mu_c$, ...
- 93
1
vote
1
answer
231
views
Derivation of the M-step in EM algorithm for a three-dimensional panel mixture model
I have a question regarding the estimation of a latent-class gaussian mixture model, where the model is for three dimensional panel data set with individuals $i$, in country $j$ in time $t$.
I want ...
- 21
1
vote
0
answers
62
views
A mixture of discrete and continuous components
Suppose that random variable $X$ is sampled from Bernoulli with probability $\pi$. Let $Y\sim N(\mu, \sigma^2)$ and denote $Z=XY$. Then, when $X=0$, $Z$ equals 0 (and this happens with probability $1-\...
- 799
0
votes
0
answers
132
views
Is there any way to perform MLE on a 1d gaussian mixture model comprised of two gaussians?
The MLE of the mean of a single gaussian distribution is the mean of the sample. But is there any way to do this when you have two gaussians such as in the figure attached? I know how to estimate the ...
15
votes
2
answers
674
views
Finding category with maximum likelihood method
Let's say that we had an information for men and women heights.
R code:
...
- 153
1
vote
0
answers
26
views
What determines performance in recoverying K in Gaussian Mixture Model?
My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm.
For simplicity, let's consider the case in ...
- 685
1
vote
1
answer
41
views
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Recently, I came across the use of Gaussian Mixture Distributions being used to model the response ...
- 6,078