Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

How to use GMM for clustering?

After running FAMD, the scatterplot of PC1 vs. PC2 looks like this: It seems like if I want to do a clustering on this data, GMM is the best option (if not, please let me know what to use). Using BIC,...
0 votes
0 answers
22 views

Metric to check the number of clusters in one dimentional data

I have a set of 1D data as shown below. I want to find a metric that represents the number of clusters in data. Is there any suitable metric that matches my criteria? Example 1D data list: [68, 3, 3, ...
0 votes
0 answers
9 views

Is there methods for splitting a Guassian Mixture model into separate unimodal distributions?

I want to apply an algorithm which usually works on unimodal distributions on multimodal distributions, I'm considering doing this by splitting the multimodal distributions into multiple unimodal ...
1 vote
0 answers
7 views

How do I make a table that prints the Mean and its corresponding weight of a Gaussian Mixture model?

I am successfully generating weights and means for a GMM, but I'm trying to get the results to print in a way that shows the corresponding mean for a weight. Here's what I have now: ...
0 votes
0 answers
26 views

In what situations can you use EM, but where you cannot compute the marginal likelihood?

Expectation Maximisation is used to find the parameters of a hierarchical model with some nuisance parameters, that need to be integrated out. The typical example is a Gaussian Mixture Model, where ...
1 vote
0 answers
20 views

Finding Probability that data is taken from a Gaussian Mixture Model

I'm using scikit's Gaussian Mixture to obtain a 2 component mixture model for some 1-D data. Having obtained a model, I want to test how well a different data set matches this mixture model, or more ...
  • 33
0 votes
0 answers
34 views

Random Intercept in a Random Coefficient Logit

I want to estimate a random coefficients logit where the intercept is random and normally distributed, not the slopes of the variables. I understand this boils down a "fixed effects logit model&...
  • 101
1 vote
1 answer
99 views

Calculating weighted covariance matrix of a weighted finite mixture of multivariate normal distributions

I am trying to calculate the weighted covariance matrix for a finite mixture of multivariate normal distributions. I read this post here and this one here, but the first post is focused on uniformly ...
4 votes
0 answers
77 views

Can GMM approximate any given probability density function?

I am currently studying on Bayesian models, and still new to probability theory. I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of ...
  • 63
4 votes
2 answers
145 views

Interpolating Gaussian mixture. How many points is sufficient for complete reconstruction

Consider the following Gaussian mixture of $N$ components: \begin{align} f(x)= \sum_{i=1}^N p (s_i) e^{-\frac{(x-s_i)^2}{2}}\big/ \sqrt{2\pi} \end{align} where we assume that $\max_{i} |s_i| \le C$ ...
  • 203
1 vote
1 answer
16 views

Is there a relationship between the number of the mixture components and the overfiting of the model?

I read the following: To prevent overfitting we would like to work with as few components as possible". How does the number of the mixture component affect the fit of the model? Is that because ...
  • 1,180
1 vote
1 answer
25 views

Can the gaussian mixture model combined in clustering?

Suppose I have a data with two clusters. Suppose further that I cluster the data using, for example, K-means. Then, can I fit a mixture model to each cluster? That is, can I fit a gaussian mixture ...
  • 507
2 votes
1 answer
28 views

Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n. What are ...
  • 21
1 vote
0 answers
52 views

Is obtaining maximum likelihood estimate of Gaussian mixture via clustering possible?

Let's say I have a data set $ X = \{ x_1, \dots, x_n \}$ with underlying probability density $$ p(x; \mu_1, \sigma_1^2, \dots, \mu_k, \sigma_k^2) = \sum_{i=1}^k \alpha_i p(x; \mu_i, \sigma_i^2), \quad ...
  • 31
1 vote
0 answers
83 views

Likelihood in mixture models

As per my understanding, normally, when we talk about Bayes rule, we write: p(z|x) = [p(x|z) * p(z)] / p(x) where, p(z|x) is called posterior p(x|z) is called likelihood p(z) is called prior p(x) is ...
  • 161
0 votes
1 answer
37 views

What is the average bias in MLE of 2-component univariate Gaussian mixture model?

Imagine that you have a standard 2-component univariate Gaussian mixture model: $$p(x_i∣θ)=(1-λ)N(x_i|μ_1,σ_1^2 )+λN(x_i|μ_2,σ_2^2 )$$ $$θ=\{μ_1,μ_2,σ_1,σ_2,λ\}$$ $$L(θ;x)=∏_{i=1}^N p(x_i |θ)$$ The ...
2 votes
0 answers
33 views

AIC vs BIC for time series clustering and descriptive purposes

I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
  • 21
0 votes
0 answers
20 views

Test for gaussian mixture fit when component assignment is known?

I have a process 𝑃 generating random variables X_1, ... X_n. From each of these I've sampled a set of samples ...
  • 191
0 votes
0 answers
46 views

MLE for Two component mixture model

Chapter 8 section 8.5.1 of the Elements of Statistical Learning book describes a simple mixture model for density estimation and the associated EM algorithm for carrying out maximum likelihood ...
0 votes
1 answer
30 views

How to obtain the p-values ​of a gamlssMX model?

I am working with a dataset that includes a binary target variable (0 or 1). I have built a model with the gamlssMX() function included on the "gamlss.mx" package to explain a continuous ...
0 votes
0 answers
19 views

Are gaussian mixture models for clustering robust to data sparsity?

I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
0 votes
0 answers
38 views

How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?

Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
  • 1
0 votes
0 answers
21 views

In a mixture model should I update the parameters of variance jointly or one-by-one?

Suppose that I have the following mixture model, where I know the true values of $(\pi_{1},\pi_{2},\pi_{3},\mu_{1},\mu_{2},\mu_{3})$ (I know them for the simulation that I build) $$f(x) = \pi_{1}N(x;\...
  • 1,913
0 votes
0 answers
30 views

How to conduct EM algorithm when there are some outliers in GMM Models?

I'm just confused about the problem of adding an outlier component directly to the primary form of GMM models: Suppose that the observed data contains several outliers. The mixture model could be: $$ ...
  • 1
2 votes
0 answers
31 views

High dimensional behavior of Dirichlet Process-based clustering?

I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed. Let $D$ be the dimensionality ...
2 votes
0 answers
28 views

Confusion Regarding Bayesian Mixture of Gaussians

Following is the screenshot from the paper "Variational Inference: A Review for Statisticians". I am having confusion understanding equations (7) and (8). Can anyone please let me know ...
  • 161
0 votes
0 answers
70 views

Selecting the optimal bandwidth in kernel density estimation

I have a question regarding kernel density estimation. At the moment I have a set of sample date $V$, where for each $v \in V$ I have an associated standard deviation $\sigma_v$ (some measurements ...
1 vote
1 answer
150 views

Fitting truncated normal mixtures in R

I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
  • 179
3 votes
1 answer
139 views

Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?

In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of Multimodality of the Likelihood. ...
  • 3,850
0 votes
0 answers
80 views

My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
1 vote
1 answer
122 views

Why cannot MLE be implemented for Gaussian mixture model directly?

Consider the following density, the mixture of two Gaussian distributions, \begin{align*} p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) , \end{align*} where $p(k=1)+p(k=0)=\pi_1+\...
  • 95
0 votes
1 answer
191 views

Creating a probability density function from a Gaussian Mixture Model

I have some daily timeseries (27 right now but will be over 200 when I get more data) for electricity consumption. For each hour I want to know what the probability density function looks like. What I ...
0 votes
0 answers
61 views

What is the number of free parameters in an n-component GMM?

I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom. If I am fitting guassian mixture model to the data, ...
  • 1
0 votes
2 answers
227 views

Is using a fixed random seed in production okay?

I have a dataset and am trying to use GMM to cluster it. The algorithm works well but when I run it multiple times I get different results. While the clusters produced in each run are valid my users ...
  • 11
0 votes
0 answers
173 views

How can I fit a 1d, two-component Gaussian mixture with very uneven weights?

Say I have data generated by, with probability $p$, sampling from one normal distribution, and with probability $1-p$, from a different normal distribution. I would like to estimate the means of the ...
  • 409
2 votes
1 answer
198 views

PCA as Pre-Processing before Clustering through GMM

Suppose I start with a data matrix $X \in \mathbb{R}^{N \times D}$, where each row $x_i$ is $D$-dimensional sample. I would like to cluster this data through a Gaussian Mixture Model (GMM). If I pre-...
0 votes
0 answers
12 views

mixture of finite regressions without a response variable

In finite mixture modelling, in particular mixture of regressions modelling, we are interested in finding latent trajectories against a response variable. But what if there is no known response ...
0 votes
0 answers
61 views

Generating a simulated dataset from a mixture of two-trait Gaussian distributions

How can I generate a random variable which follows the mixture Gaussian distributions: I found the answer and tried to simulate it, but I think it is not the right dataset I wanted. Here is my code: <...
  • 23
0 votes
1 answer
112 views

Why Gibbs Sampling for mixture models?

I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
0 votes
1 answer
36 views

For univariate data, why do we need the normalmixEM function in R instead of just computing the mean and variance with the basic methods?

I can understand why if from your univariate data (1 column?) you plot a histogram which seems to have 2+ peaks ie a mix of more than 1 gaussian. But what if you plot a histogram and there looks to be ...
1 vote
0 answers
92 views

Problems with convergence of the EM algorithm for a gaussian mixture regression

I have been implementing a EM-algorithm for a latent-class regression model, where every individual has a vector of observations. Currenly, I have the problem that the model does not converge. The log ...
2 votes
0 answers
72 views

What is the distribution of the Poisson Sum of gaussians?

I know that the sum of two independent normal random variables is normal. Particulary, when one is copy of other, i.e., if $X_1, X_2 \sim \mathcal{N}(0,\sigma^2)$, independent, we have: $$X_1 + X_2 \...
  • 222
3 votes
3 answers
195 views

what is the difference between mixture of two normal distributions and sum of two independent variables

The following denotes a mixture of a standard normal with a normal with the same mean but 100 times the variance: $0.95 \mathrm{~N}(0,1)+ 0.05 \mathrm{~N}(0,100)$ Let Y = 0.95 X + 0.05 Z with X,Z are ...
1 vote
0 answers
66 views

MAD & Median of weighted GMM

What is the median and median-absolute-deviation of a weighted GMM in terms of component mean and variance? For example, three normal distributions $A$, $B$, $C$ with means $\mu_a,\mu_b,\mu_c$, ...
1 vote
1 answer
231 views

Derivation of the M-step in EM algorithm for a three-dimensional panel mixture model

I have a question regarding the estimation of a latent-class gaussian mixture model, where the model is for three dimensional panel data set with individuals $i$, in country $j$ in time $t$. I want ...
1 vote
0 answers
62 views

A mixture of discrete and continuous components

Suppose that random variable $X$ is sampled from Bernoulli with probability $\pi$. Let $Y\sim N(\mu, \sigma^2)$ and denote $Z=XY$. Then, when $X=0$, $Z$ equals 0 (and this happens with probability $1-\...
  • 799
0 votes
0 answers
132 views

Is there any way to perform MLE on a 1d gaussian mixture model comprised of two gaussians?

The MLE of the mean of a single gaussian distribution is the mean of the sample. But is there any way to do this when you have two gaussians such as in the figure attached? I know how to estimate the ...
15 votes
2 answers
674 views

Finding category with maximum likelihood method

Let's say that we had an information for men and women heights. R code: ...
1 vote
0 answers
26 views

What determines performance in recoverying K in Gaussian Mixture Model?

My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm. For simplicity, let's consider the case in ...
  • 685
1 vote
1 answer
41 views

Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?

Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions? Recently, I came across the use of Gaussian Mixture Distributions being used to model the response ...
  • 6,078

1
2 3 4 5
12