Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
1 vote
0 answers
16 views

What does mixture refer to in LCMM and HLME in R

I am trying to use the HLME and LCMM functions to fit latent class mixed models to my data. Here are the documentations to both of them: https://www.rdocumentation.org/packages/lcmm/versions/1.8.1.1/...
  • 183
1 vote
1 answer
51 views

Gaussian mixture model probabilities

I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
0 votes
0 answers
36 views

Ask for help understanding a sentence in Bishop's PRML book on soft weight sharing

It's about section 5.5.7 of Christopher M. Bishop's "Pattern Recognition and Machine Learning" on soft weight sharing. The sentence is the first three lines of page 271. First the author ...
  • 183
1 vote
0 answers
19 views

How to solve gamma from GMM

GMM refers to the Gaussian mixture model, cf. here. Suppose we have $N$ data points (or observations in Statistics) and $K$ Gaussian models to mix. After going through Maximum Likelihood Estimation, ...
  • 177
4 votes
1 answer
275 views

Mixture of Gaussian is not log-concave

I've encountered the statement: For $p\in(0,1),$ the location mixture of standard univariate normal densities $f(x)=p\phi(x)+(1-p)\phi(x-\mu)$ is log-concave if and only if $\Vert\mu\Vert \leq 2.$ ...
  • 163
0 votes
0 answers
9 views

Do the Mixing Coefficients in a Gaussian Mixture Model Depend on the Datapoint?

Given a Gaussian mixture model with $K$ clusters, the probability of sampling a point $x\in\mathbb{R}^d$ is given by $$ p(x) = \sum_{k=1}^K\pi_k\mathcal{N}(x;\boldsymbol\mu_k,\boldsymbol\sigma_k) $$ ...
  • 101
0 votes
1 answer
29 views

Mixture of Two Normals

Suppose we have a data which consists of two normals, x = rnorm(50,mean=1,sd=2) y = rnorm(50,mean=2,sd=3) z = sample( c(x,y) , size = 100, replace=FALSE ) The goal ...
0 votes
0 answers
33 views

Model Comparison within Bayesian Gaussian Mixture Model framework

Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model. $$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
  • 2,028
7 votes
1 answer
166 views

EM algorithm for Bivariate Normal

Consider a random sample $X_i = (U_i,V_i)$ where $i=1,2,...,n$ from a bivariate normal population with mean $(\mu_1,\mu_2)$ and variances $(\sigma_1 ^2, \sigma_2 ^2)$ and correlation $\rho$. Let's ...
0 votes
1 answer
103 views

Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]

In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
0 votes
0 answers
9 views

EM algorithm for finding discrete latent variables with gaussian child in Python

I have the following Bayesian network model: $P(A, B, C, D) = P(D|B)P(C|B)P(B|A)P(A)$ A and B are discrete variables, while C and D are continuous variables. I've looked around and it seems in a ...
  • 61
0 votes
0 answers
23 views

Inferring class probabilities in Gaussian Mixture Model

Let's say I go shopping and always buy a few fruits and a few candies. Candies are typically more expensive than fruits. The prices of fruits and candies are both well modeled by gaussian ...
1 vote
0 answers
13 views

What is the best way to calculate the fit of the mixture distribution to the actual data?

I have fitted the mixture of Gaussians to the natural log of the data. I know that the model is not a very good fit to the data in the tail region, however in the high density region the actual data ...
  • 75
1 vote
1 answer
48 views

Closed form solution of multivariate Gaussian over mixture of multivariate Gaussians

Suppose I have three variables $X_{1}, X_{2}$ and $Y$, where $X_{1}, X_{2}$ are continuous and $Y$ is binary. The conditional distribution of $X_{1}, X_{2}$, given $Y$ is a multivariate Gaussian ...
  • 306
1 vote
1 answer
43 views

Coefficients of Gaussian mixture

This is in context of Gaussian mixtures $$p(\boldsymbol{x}) = \sum_{k=1}^K \pi_k\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})$$ Bishop mentions on Page-111 Also, the requirement ...
  • 131
2 votes
2 answers
221 views

p(x) in Gaussian mixture model

I've recently learned about Gaussian mixture models (GMM) in school. The formula for a GMM goes something like this: $$p(x) = \sum_{k=1}^{K} \pi_i\mathcal{N}(x|\mu_i,\,\sigma^{2}_i)$$ Now, I know from ...
  • 123
1 vote
0 answers
13 views

When imputing missing labels Y1 == NaN during training, how do additional target vectors (Y2 != NaN) impact learning Y1==NaN?

I am training a Mixture Density Network (MDN) to map from continuous input vectors X to continuous targets Y [i.e. X -> Y]. There are missing labels on vector Yi, which I impute from the mdn (as ...
  • 11
0 votes
1 answer
50 views

Could the likelihood increase monotonically in a misspecified EM algorithm?

I am dealing with the estimation of a Gaussian Hidden Markov Model with conditional distribution given the first-order Markov state $S_t = j,\ j=1,...,J$ $$ Y_t|S_t=j\sim N(0,\sigma^2_j) $$ where the ...
  • 319
0 votes
0 answers
20 views

How to use GMM for clustering?

After running FAMD, the scatterplot of PC1 vs. PC2 looks like this: It seems like if I want to do a clustering on this data, GMM is the best option (if not, please let me know what to use). Using BIC,...
0 votes
0 answers
23 views

Metric to check the number of clusters in one dimentional data

I have a set of 1D data as shown below. I want to find a metric that represents the number of clusters in data. Is there any suitable metric that matches my criteria? Example 1D data list: [68, 3, 3, ...
0 votes
0 answers
13 views

Is there methods for splitting a Guassian Mixture model into separate unimodal distributions?

I want to apply an algorithm which usually works on unimodal distributions on multimodal distributions, I'm considering doing this by splitting the multimodal distributions into multiple unimodal ...
1 vote
0 answers
9 views

How do I make a table that prints the Mean and its corresponding weight of a Gaussian Mixture model?

I am successfully generating weights and means for a GMM, but I'm trying to get the results to print in a way that shows the corresponding mean for a weight. Here's what I have now: ...
0 votes
0 answers
27 views

In what situations can you use EM, but where you cannot compute the marginal likelihood?

Expectation Maximisation is used to find the parameters of a hierarchical model with some nuisance parameters, that need to be integrated out. The typical example is a Gaussian Mixture Model, where ...
1 vote
0 answers
55 views

Finding Probability that data is taken from a Gaussian Mixture Model

I'm using scikit's Gaussian Mixture to obtain a 2 component mixture model for some 1-D data. Having obtained a model, I want to test how well a different data set matches this mixture model, or more ...
  • 33
0 votes
0 answers
49 views

Random Intercept in a Random Coefficient Logit

I want to estimate a random coefficients logit where the intercept is random and normally distributed, not the slopes of the variables. I understand this boils down a "fixed effects logit model&...
  • 111
1 vote
1 answer
225 views

Calculating weighted covariance matrix of a weighted finite mixture of multivariate normal distributions

I am trying to calculate the weighted covariance matrix for a finite mixture of multivariate normal distributions. I read this post here and this one here, but the first post is focused on uniformly ...
4 votes
0 answers
113 views

Can GMM approximate any given probability density function?

I am currently studying on Bayesian models, and still new to probability theory. I learned that Gaussian Mixture Model is used to represent the distribution of given population as a weighted sum of ...
  • 63
4 votes
2 answers
161 views

Interpolating Gaussian mixture. How many points is sufficient for complete reconstruction

Consider the following Gaussian mixture of $N$ components: \begin{align} f(x)= \sum_{i=1}^N p (s_i) e^{-\frac{(x-s_i)^2}{2}}\big/ \sqrt{2\pi} \end{align} where we assume that $\max_{i} |s_i| \le C$ ...
  • 215
2 votes
1 answer
28 views

Is there a relationship between the number of the mixture components and the overfiting of the model?

I read the following: To prevent overfitting we would like to work with as few components as possible". How does the number of the mixture component affect the fit of the model? Is that because ...
  • 1,368
1 vote
1 answer
32 views

Can the gaussian mixture model combined in clustering?

Suppose I have a data with two clusters. Suppose further that I cluster the data using, for example, K-means. Then, can I fit a mixture model to each cluster? That is, can I fit a gaussian mixture ...
  • 507
2 votes
1 answer
39 views

Determine if high dimensional data is multimodal

I have p-dimensional data and I need to determine if that data has significant modes or if it’s clustered in any way. Here p=50, (dense embedding), we have n samples and p <<< n. What are ...
  • 21
1 vote
0 answers
62 views

Is obtaining maximum likelihood estimate of Gaussian mixture via clustering possible?

Let's say I have a data set $ X = \{ x_1, \dots, x_n \}$ with underlying probability density $$ p(x; \mu_1, \sigma_1^2, \dots, \mu_k, \sigma_k^2) = \sum_{i=1}^k \alpha_i p(x; \mu_i, \sigma_i^2), \quad ...
  • 31
1 vote
0 answers
205 views

Likelihood in mixture models

As per my understanding, normally, when we talk about Bayes rule, we write: p(z|x) = [p(x|z) * p(z)] / p(x) where, p(z|x) is called posterior p(x|z) is called likelihood p(z) is called prior p(x) is ...
  • 401
0 votes
1 answer
52 views

What is the average bias in MLE of 2-component univariate Gaussian mixture model?

Imagine that you have a standard 2-component univariate Gaussian mixture model: $$p(x_i∣θ)=(1-λ)N(x_i|μ_1,σ_1^2 )+λN(x_i|μ_2,σ_2^2 )$$ $$θ=\{μ_1,μ_2,σ_1,σ_2,λ\}$$ $$L(θ;x)=∏_{i=1}^N p(x_i |θ)$$ The ...
2 votes
0 answers
43 views

AIC vs BIC for time series clustering and descriptive purposes

I'm in the process of fitting a hidden markov model with gaussian mixtures to time series health data. The primary purpose of this is descriptive, not predictive – I'm using the fitted model to give a ...
  • 21
0 votes
0 answers
28 views

Test for gaussian mixture fit when component assignment is known?

I have a process 𝑃 generating random variables X_1, ... X_n. From each of these I've sampled a set of samples ...
  • 191
0 votes
0 answers
88 views

MLE for Two component mixture model

Chapter 8 section 8.5.1 of the Elements of Statistical Learning book describes a simple mixture model for density estimation and the associated EM algorithm for carrying out maximum likelihood ...
0 votes
1 answer
47 views

How to obtain the p-values ​of a gamlssMX model?

I am working with a dataset that includes a binary target variable (0 or 1). I have built a model with the gamlssMX() function included on the "gamlss.mx" package to explain a continuous ...
0 votes
0 answers
28 views

Are gaussian mixture models for clustering robust to data sparsity?

I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
0 votes
0 answers
46 views

How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?

Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
  • 1
0 votes
0 answers
32 views

In a mixture model should I update the parameters of variance jointly or one-by-one?

Suppose that I have the following mixture model, where I know the true values of $(\pi_{1},\pi_{2},\pi_{3},\mu_{1},\mu_{2},\mu_{3})$ (I know them for the simulation that I build) $$f(x) = \pi_{1}N(x;\...
  • 2,028
0 votes
0 answers
36 views

How to conduct EM algorithm when there are some outliers in GMM Models?

I'm just confused about the problem of adding an outlier component directly to the primary form of GMM models: Suppose that the observed data contains several outliers. The mixture model could be: $$ ...
  • 1
2 votes
0 answers
35 views

High dimensional behavior of Dirichlet Process-based clustering?

I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed. Let $D$ be the dimensionality ...
2 votes
0 answers
31 views

Confusion Regarding Bayesian Mixture of Gaussians

Following is the screenshot from the paper "Variational Inference: A Review for Statisticians". I am having confusion understanding equations (7) and (8). Can anyone please let me know ...
  • 401
0 votes
0 answers
110 views

Selecting the optimal bandwidth in kernel density estimation

I have a question regarding kernel density estimation. At the moment I have a set of sample date $V$, where for each $v \in V$ I have an associated standard deviation $\sigma_v$ (some measurements ...
2 votes
1 answer
214 views

Fitting truncated normal mixtures in R

I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
  • 371
4 votes
1 answer
187 views

Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?

In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of Multimodality of the Likelihood. ...
  • 3,930
0 votes
0 answers
119 views

My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
2 votes
1 answer
262 views

Why cannot MLE be implemented for Gaussian mixture model directly?

Consider the following density, the mixture of two Gaussian distributions, \begin{align*} p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) , \end{align*} where $p(k=1)+p(k=0)=\pi_1+\...
  • 105
0 votes
1 answer
391 views

Creating a probability density function from a Gaussian Mixture Model

I have some daily timeseries (27 right now but will be over 200 when I get more data) for electricity consumption. For each hour I want to know what the probability density function looks like. What I ...

1
2 3 4 5
13