Questions tagged [gaussian-mixture-distribution]
A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.
586
questions
0
votes
0
answers
16
views
MLE for Two component mixture model
Chapter 8 section 8.5.1 of the Elements of Statistical Learning book describes a simple mixture model for density estimation and the associated EM algorithm for carrying out maximum likelihood ...
- 123
0
votes
0
answers
3
views
How to obtain the p-values of a gamlssMX model?
I am working with a dataset that includes a binary target variable (0 or 1).
I have built a model with the gamlssMX() function included on the "gamlss.mx" package to explain a continuous ...
0
votes
0
answers
8
views
Are gaussian mixture models for clustering robust to data sparsity?
I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
- 101
0
votes
0
answers
26
views
How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?
Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
- 1
0
votes
0
answers
16
views
In a mixture model should I update the parameters of variance jointly or one-by-one?
Suppose that I have the following mixture model, where I know the true values of $(\pi_{1},\pi_{2},\pi_{3},\mu_{1},\mu_{2},\mu_{3})$ (I know them for the simulation that I build)
$$f(x) = \pi_{1}N(x;\...
- 1,679
0
votes
0
answers
15
views
How to conduct EM algorithm when there are some outliers in GMM Models?
I'm just confused about the problem of adding an outlier component directly to the primary form of GMM models:
Suppose that the observed data contains several outliers. The mixture model could be:
$$
...
- 1
0
votes
0
answers
7
views
True and Estimated Posterior of a Mixture of Gaussians in Bayesian Learning via SGLD
I am trying to recreate one of the experiments in this paper, (Bayesian Learning via Stochastic Gradient Langevin Dynamics). To be exact experiment 5.1. I am pretty sure, I am missing something here ...
2
votes
0
answers
25
views
High dimensional behavior of Dirichlet Process-based clustering?
I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed.
Let $D$ be the dimensionality ...
- 699
2
votes
0
answers
21
views
Confusion Regarding Bayesian Mixture of Gaussians
Following is the screenshot from the paper "Variational Inference: A Review for Statisticians".
I am having confusion understanding equations (7) and (8).
Can anyone please let me know ...
- 61
0
votes
0
answers
12
views
Selecting the optimal bandwidth in kernel density estimation
I have a question regarding kernel density estimation.
At the moment I have a set of sample date $V$, where for each $v \in V$ I have an associated standard deviation $\sigma_v$ (some measurements ...
- 1
1
vote
1
answer
104
views
Fitting truncated normal mixtures in R
I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
- 189
0
votes
0
answers
10
views
Conditional distribution of GMM with given distribution?
I'm currently looking for the propagation of Gaussian mixture noise in a linear system.
I have a question regards how to calculate the following equations:
Suppose there is an GMM $f_1(x,y)$ = $\sum_{...
- 1
3
votes
1
answer
92
views
Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?
In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of
Multimodality of the Likelihood. ...
- 3,768
0
votes
0
answers
26
views
My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?
I use DeLonge method to compare two ROC AUCS. The result of it is Z-score.
Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
- 1
1
vote
1
answer
50
views
Why cannot MLE be implemented for Gaussian mixture model directly?
Consider the following density, the mixture of two Gaussian distributions,
\begin{align*}
p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) ,
\end{align*}
where $p(k=1)+p(k=0)=\pi_1+\...
- 95
0
votes
1
answer
75
views
Creating a probability density function from a Gaussian Mixture Model
I have some daily timeseries (27 right now but will be over 200 when I get more data) for electricity consumption. For each hour I want to know what the probability density function looks like.
What I ...
- 35
0
votes
0
answers
10
views
What is the number of free parameters in an n-component GMM?
I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom.
If I am fitting guassian mixture model to the data, ...
- 1
0
votes
2
answers
68
views
Is using a fixed random seed in production okay?
I have a dataset and am trying to use GMM to cluster it. The algorithm works well but when I run it multiple times I get different results. While the clusters produced in each run are valid my users ...
- 11
0
votes
0
answers
46
views
How can I fit a 1d, two-component Gaussian mixture with very uneven weights?
Say I have data generated by, with probability $p$, sampling from one normal distribution, and with probability $1-p$, from a different normal distribution. I would like to estimate the means of the ...
- 389
0
votes
0
answers
73
views
PCA as Pre-Processing before Clustering through GMM
Suppose I start with a data matrix $X \in \mathbb{R}^{N \times D}$, where each row $x_i$ is $D$-dimensional sample. I would like to cluster this data through a Gaussian Mixture Model (GMM).
If I pre-...
- 127
0
votes
0
answers
11
views
mixture of finite regressions without a response variable
In finite mixture modelling, in particular mixture of regressions modelling, we are interested in finding latent trajectories against a response variable. But what if there is no known response ...
- 155
0
votes
0
answers
19
views
Generating a simulated dataset from a mixture of two-trait Gaussian distributions
How can I generate a random variable which follows the mixture Gaussian distributions:
I found the answer and tried to simulate it, but I think it is not the right dataset
I wanted.
Here is my code:
<...
- 23
0
votes
1
answer
56
views
Why Gibbs Sampling for mixture models?
I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
0
votes
1
answer
29
views
For univariate data, why do we need the normalmixEM function in R instead of just computing the mean and variance with the basic methods?
I can understand why if from your univariate data (1 column?) you plot a histogram which seems to have 2+ peaks ie a mix of more than 1 gaussian. But what if you plot a histogram and there looks to be ...
- 5
1
vote
0
answers
48
views
Problems with convergence of the EM algorithm for a gaussian mixture regression
I have been implementing a EM-algorithm for a latent-class regression model, where every individual has a vector of observations.
Currenly, I have the problem that the model does not converge. The log ...
- 21
2
votes
0
answers
54
views
What is the distribution of the Poisson Sum of gaussians?
I know that the sum of two independent normal random variables is normal. Particulary, when one is copy of other, i.e., if $X_1, X_2 \sim \mathcal{N}(0,\sigma^2)$, independent, we have:
$$X_1 + X_2 \...
- 223
2
votes
2
answers
56
views
what is the difference between mixture of two normal distributions and sum of two independent variables
The following denotes a mixture of a standard normal with a
normal with the same mean but 100 times the variance:
$0.95 \mathrm{~N}(0,1)+ 0.05 \mathrm{~N}(0,100)$
Let Y = 0.95 X + 0.05 Z with X,Z are ...
1
vote
0
answers
57
views
MAD & Median of weighted GMM
What is the median and median-absolute-deviation of a weighted GMM in terms of component mean and variance? For example, three normal distributions $A$, $B$, $C$ with means $\mu_a,\mu_b,\mu_c$, ...
- 93
1
vote
1
answer
159
views
Derivation of the M-step in EM algorithm for a three-dimensional panel mixture model
I have a question regarding the estimation of a latent-class gaussian mixture model, where the model is for three dimensional panel data set with individuals $i$, in country $j$ in time $t$.
I want ...
- 21
1
vote
0
answers
37
views
A mixture of discrete and continuous components
Suppose that random variable $X$ is sampled from Bernoulli with probability $\pi$. Let $Y\sim N(\mu, \sigma^2)$ and denote $Z=XY$. Then, when $X=0$, $Z$ equals 0 (and this happens with probability $1-\...
- 789
0
votes
0
answers
64
views
Is there any way to perform MLE on a 1d gaussian mixture model comprised of two gaussians?
The MLE of the mean of a single gaussian distribution is the mean of the sample. But is there any way to do this when you have two gaussians such as in the figure attached? I know how to estimate the ...
15
votes
2
answers
667
views
Finding category with maximum likelihood method
Let's say that we had an information for men and women heights.
R code:
...
- 153
1
vote
0
answers
21
views
What determines performance in recoverying K in Gaussian Mixture Model?
My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm.
For simplicity, let's consider the case in ...
- 653
1
vote
1
answer
33
views
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?
Recently, I came across the use of Gaussian Mixture Distributions being used to model the response ...
- 5,754
2
votes
0
answers
83
views
How to calculate the covariance matrix in (Xu and Knight 2010)?
Setup
I'm reading (Xu and Knight 2010), which is a paper about estimating finite Gaussian mixture models using the CECF (Continuous Empirical Characteristic Function) method. The basic idea is to ...
- 148
4
votes
1
answer
444
views
K Means as a special case of GMM (using EM Algorithm)
I am looking for a tutorial/gentle introduction (preferably with mathematics/proofs) on K-means as a special case of Gaussian Mixture Model using the EM Algorithm.
I have found this: https://www....
- 7,585
0
votes
0
answers
109
views
Compute log-likelihood in Bernoulli Gaussian Mixture
I'm working on this exercise about Gaussian Mixtures:
Here's part of the solution:
I don't understand how they came up with this equation for the log-likelihood (red arrow).
From what I know, the ...
- 1
0
votes
0
answers
25
views
Help students understand Gaussian Kernel in mean shift segmentation
I am creating a image mean shift segmentation algorithm for a class.
So far what I've done is basically
Iterate through each point/pixel on the image and with a given bandwidth(radius) calculate the ...
1
vote
2
answers
197
views
Are linear combinations (i.e. "sums") of gaussian distributions also gaussian?
I have always assumed that this is fact : but are there any mathematical theorems that state: finite sums of gaussian distributions are gaussian themselves?
In the above picture, does p(x) have a ...
- 5,754
0
votes
0
answers
75
views
Conjugate prior bayesian inference on multivariate GMM
I am trying to understand how the posterior looks like when running Bayesian inference on a multivariate Gaussian-mixture model.
$p(\mathbf{x}) \propto \sum_{i=1}^M w_iN(\mathbf{x}|\mu_i,\Sigma_i)$.
...
- 1
0
votes
0
answers
11
views
Is there a formula for the optimal weights for a Gaussian mixture model when the components are considered "correct"?
For simplicity consider two models predicting $Y=Z_0 + \sigma Z_1$, where $Z_i$ are independent $\mathcal{N}(0, 1)$ and each model only take one of the $Z_i$ as a factor. So we have $Y|Z_0$ and $Y|Z_1$...
- 238
1
vote
1
answer
123
views
Assumption of Gaussian mixture model [closed]
I am still confused when I read about the Gaussian Mixture Model and how does it work.
One thing that I do not understand is that "GMM assumes the data is a mixture of Gaussian or
i.i.d (...
- 173
0
votes
0
answers
12
views
Are the subsets of Mixture Models necessarily parametrized?
Im just learning about mixture models and they are described as a "mix of parametric and non parametric models".
My question is how a non-parametric subset would look like? How would e.g. EM-...
- 1
1
vote
1
answer
109
views
Conditional distribution of the weight of a mixture gaussian with data augmentation using gibbs sampling
This question is relate to Differenciate between two distributions using gibbs sampling .
For $t=1,\,\dots,\,n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl}
\...
- 793
0
votes
1
answer
54
views
Differenciate between two distributions using gibbs sampling [closed]
This question is relate to the post :
" Conditional distribution for Gibbs sampling for Gaussian mixture " but is a little bit different. My objective is to know why the algorithm (which is ...
- 793
0
votes
0
answers
33
views
Interpretation of test set negative log likelihood in neural density estimation applications
I have seen people splitting a dataset into a train and test sets and learning the parameters of a mixture density network using the negative log likelihood cost function on the train set and ...
user297721
1
vote
1
answer
299
views
Kernel design for Gaussian processes with multiple inputs
How can I design a kernel function when there are multiple input variables and their degree of influence on the covariance with the target variable is different from each other?
For example, if the ...
- 155
0
votes
0
answers
35
views
Mixture Density Network on Unbalanced Dataset
I want to use a mixture density network to make predictions with NLL loss (which is working) but my dataset is unbalanced so my network tends to make estimations that look like the distribution of ...
- 11
4
votes
1
answer
141
views
Gaussian distribution with poisson variance
Let's $r_t\sim \mathcal{N}(0,V_t)$ where $V_t = (1+m)^{J_t}$, $m\geq0$ and $J_t\sim \mathcal{P}oi(\lambda)$. How can we compute the distribution of $r_t$ when the parameters are $m$ and $\lambda$. In ...
- 793
0
votes
0
answers
11
views
Serie calculation - Normal poisson distribution
I would like to compute that serie if it is possible :
$\sum_{j\geq 0}\frac{(\lambda m^{-1/2})^j}{j!} \exp{\left(-B\,m^{-j}\right)}$ where $B \geq 0,\, \lambda > 0,\, m>1$.
I know it converge as ...
- 793