Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
21 views

Question about Gaussian Mixture model [duplicate]

suppose I have two parameters: $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and suppose I mapped them to PDF using the gaussian mixture ...
Andyale's user avatar
  • 133
0 votes
0 answers
22 views

Derivation of EM algorithm for Gaussian mixture

I am going through Expectation Maximization (EM) algorithm derivation for Gaussian Mixture models. I understand it except for a small detail. So, the general idea of EM is to maximize the expectation ...
baronett's user avatar
2 votes
0 answers
16 views

Derive ELBO for Mixture of Gaussian

I am working through "Variational Inference: A Review for Statisticians" by Blei et al. (see https://arxiv.org/abs/1601.00670) and they illustrate Variational Inference using a Bayesian ...
PSLS's user avatar
  • 41
0 votes
0 answers
57 views

Understanding of Gamma distribution as precision prior in Bayesian inference for Gaussian

Christopher M. Bishop in his book "Pattern Recognition and Machine Learning" nicely explains where does Student t-distribution $St(x|\mu,\lambda,\upsilon)$ originate into. In Chapter 2, it ...
baronett's user avatar
0 votes
1 answer
38 views

Identify outliers in testing data based on trained Gaussian mixture model

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
0 answers
67 views

Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
1 answer
46 views

Gaussian Mixture Model with Minkowski distance

Gaussian Mixture Models assume Mahalanobis distance (essentially L2). Is it possible to use Lp distance in a GMM? Intuitively, in 1-space, distance is clear. In 2-space, the relation between the two ...
olivarb's user avatar
  • 111
0 votes
0 answers
14 views

Are combinations of variables a problem in multinomial mixture models?

I am quite new to cluster analysis and I am currently using Gaussian-multinomial mixture models. I have a sample size of 10000 people with some continuous variables and about 14 categorical variables (...
Lander's user avatar
  • 3
2 votes
2 answers
109 views

MAP estimation for a Gaussian mixture using EM. Concerns with the covariance update formula

I am implementing the EM algorithm for a Gaussian mixture model with prior; that is, I am using the EM algorithm to find the MAP estimate, rather than the ML estimate. As briefly discussed in section ...
ummg's user avatar
  • 135
1 vote
0 answers
23 views

Linear Discriminant Analysis with unlabeled data

In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following: From the mixture formulation [that is, ...
Sergio's user avatar
  • 336
1 vote
1 answer
116 views

Generate marginally dependent (with predetermined covariance) but conditionally independent data from a Mixture of Gaussians

Suppose you have three variables $y\in\{0,1\}$ and $x_1\in\mathbb{R}$ and $x_2\in\mathbb{R}$. I want to produce data with the following generative process which corresponds to a Mixture of Gaussians (...
Sergio's user avatar
  • 336
0 votes
0 answers
15 views

Mixture Model: Data Consist of Continuous and Binary Features

I have a features like below id x1 x2 x3 x4 x5 id1 0.4 1.4 5.6 1 0 id2 -0.01 0.5 -3.4 0 1 where x1, x2, x3 are continuous features and x4 and x5 are binary. The goal is to find $k$ clusters using ...
AnonymousJ's user avatar
0 votes
0 answers
15 views

Supervised classifier for nested interval data and ordinal classes

I'm having trouble formalizing the following classification problem: Let $x_i$ denote univariate (scalar), continuous, real data points Let $y_i \in \mathbb{N}$ be their corresponding labels Classes ...
themodelguy's user avatar
0 votes
1 answer
48 views

Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model

I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
Kyungmin's user avatar
0 votes
1 answer
37 views

Assumptions and setting for bayesian mixture model (for RJMCMC)

I want to understand about Bayesian mixture model discussed in RJMCMC paper (Richardson and Green, 1997) (https://academic.oup.com/jrsssb/article/59/4/731/7083042) I also posted similar question ...
Kyungmin's user avatar
0 votes
0 answers
50 views

EM algorithm get new parameters by optimizing the Q function (lower bound of likelihood function) or optimizing the likelihood function

We know that in the EM (Expectation-Maximization) algorithm, the E-step determines the $Q$ function by calculating expectations, which is a lower bound of the likelihood function. In the M-step, by ...
yi ren's user avatar
  • 1
2 votes
1 answer
175 views

Why gaussian mixture model is sensitive to cluster size?

I did an experiment where I generated 3 well separated clusters with a different multivariate gaussian distribution for each cluster. One of the clusters contains 1000 points and the other two ...
striker22's user avatar
  • 323
0 votes
2 answers
56 views

Flexmix maxima are not where they are expected to be

For my dataset I have plotted the density with ggplot. As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus. ...
Lukas's user avatar
  • 111
0 votes
0 answers
20 views

Decomposing into Gaussian components using Bhattacharya from topFishR

I am working with fishery data. I have a data vector called SFL that contains the sizes of the fish caught. Here is some sample data: ...
sermomon's user avatar
  • 113
0 votes
1 answer
42 views

Confidence interval for unsymmetrical Gaussian Mixure Model PDFs?

Let Y be a vector of observations. A Gaussian Mixure Model (GMM) is fit to the dataset. The distribution can appear unsymmetrical, with different thickness of tails in both sides. What is the best way ...
Bahar's user avatar
  • 1
0 votes
1 answer
53 views

Fitting a Gaussian Mixture Model with known share of noise/outliers

A Gaussian Mixture model is fitted by the Expectation-Maximization algorithm. This fairly simple iterative algorithm consists of two steps and the initialization. Initialization (for ...
Klops's user avatar
  • 156
1 vote
1 answer
110 views

Variance of a mixture of normally distributed random variables [duplicate]

Setup Suppose I have two normally distributed random variables, $s_\mathrm{prev}$ (with mean $\mu_\mathrm{prev}$ and variance $\sigma^2_\mathrm{prev}$) and $s_\mathrm{curr}$ (with mean $\mu_\mathrm{...
abcd's user avatar
  • 113
0 votes
0 answers
13 views

Requisites in data in order to use Gaussian Mixture Models

I would like to know if I want to use GMM if the data has to follow a Gaussian distribution even if I preprocess the data and normalize it. Is it enough to normalize the data or has it have to pass ...
Jesus M.'s user avatar
0 votes
0 answers
103 views

Bayesian Gaussian mixture - is my prior correct?

I'd like to sample from the Bayesian Posterior of a Gaussian mixture model, but I am not sure about the correct Bayesian formulation of the latter. Is the following correct? I consider the 1-...
reloh100's user avatar
1 vote
0 answers
45 views

Could one use mixtures of Gaussians to turn MCMC posterior samples into a new prior?

Theoretically in Bayesian inference one could use one experiment's posterior as another experiment's prior, such that knowledge of the parameters accumulates from $p(\theta) \rightarrow p(\theta|\...
Durden's user avatar
  • 955
0 votes
0 answers
47 views

Differential Entropy of Zero-Mean Gaussian Mixtures

Introduction Consider a univariate circularly symmetric complex Gaussian (CSCG) mixture $Y$ with pdf $$p_Y(y) = \sum_i c_i p_i(y) = \sum_i c_i \frac{\exp(-\lvert y \rvert^2/\sigma_i^2)}{\pi \sigma_i^2}...
SnowzTail's user avatar
0 votes
0 answers
11 views

What assumptions are made about categorical variables in regression models? [duplicate]

This is the formalization for a continuous regression model as I understand it: Assume that your outcome $Y$ is normally distributed. Assume that your predictors, $X_1, X_2, \cdots X_n$ are normally ...
djpassey's user avatar
0 votes
0 answers
15 views

What is the difference between hidden states and mixture components in a continuous Hidden Markov Model?

I'm trying to understand how a continuous Hidden Markov Model works and I am confused by the difference of the hidden states of the discrete latent variables and the mixture components in the emission ...
Alfonso Lava's user avatar
0 votes
0 answers
52 views

Clustering correlated variables in Gaussian Mixture Models

I would like to ask it dose not require to consider the correlation of variables if using Gaussian mixture model for clustering, because covariance has already been considered in the model. So the ...
Xiaose's user avatar
  • 1
0 votes
0 answers
18 views

Similarity of two sets of scatter data

I am currently working on a project where I have a dataset of scatter-points in space. What I am doing is that I fit Gaussian Mixture Models (GMM) on the data set and use the model to sample '...
Kelvin Walenta's user avatar
0 votes
0 answers
33 views

How to sample from a given accessible PDF?

I am learning Gibbs Sampling for GMMs. Particularly, given $\boldsymbol \theta$, I must sample from the latent $\boldsymbol z$ before sampling $\boldsymbol x$. The PDF of $\boldsymbol z$ is given as$$ ...
wd violet's user avatar
  • 605
1 vote
0 answers
237 views

What does mixture refer to in LCMM and HLME in R

I am trying to use the HLME and LCMM functions to fit latent class mixed models to my data. Here are the documentations to both of them: https://www.rdocumentation.org/packages/lcmm/versions/1.8.1.1/...
TYL's user avatar
  • 203
1 vote
1 answer
164 views

Gaussian mixture model probabilities

I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
alle_meije's user avatar
1 vote
0 answers
37 views

How to solve gamma from GMM

GMM refers to the Gaussian mixture model, cf. here. Suppose we have $N$ data points (or observations in Statistics) and $K$ Gaussian models to mix. After going through Maximum Likelihood Estimation, ...
Sam Wong's user avatar
  • 177
5 votes
1 answer
412 views

Mixture of Gaussian is not log-concave

I've encountered the statement: For $p\in(0,1),$ the location mixture of standard univariate normal densities $f(x)=p\phi(x)+(1-p)\phi(x-\mu)$ is log-concave if and only if $\Vert\mu\Vert \leq 2.$ ...
12345's user avatar
  • 213
0 votes
1 answer
40 views

Mixture of Two Normals

Suppose we have a data which consists of two normals, x = rnorm(50,mean=1,sd=2) y = rnorm(50,mean=2,sd=3) z = sample( c(x,y) , size = 100, replace=FALSE ) The goal ...
Nicolas Bourbaki's user avatar
0 votes
0 answers
98 views

Model Comparison within Bayesian Gaussian Mixture Model framework

Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model. $$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
Fiodor1234's user avatar
  • 2,152
7 votes
1 answer
258 views

EM algorithm for Bivariate Normal

Consider a random sample $X_i = (U_i,V_i)$ where $i=1,2,...,n$ from a bivariate normal population with mean $(\mu_1,\mu_2)$ and variances $(\sigma_1 ^2, \sigma_2 ^2)$ and correlation $\rho$. Let's ...
Maths Freak's user avatar
0 votes
1 answer
652 views

Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]

In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
Vincent Granville's user avatar
1 vote
0 answers
25 views

What is the best way to calculate the fit of the mixture distribution to the actual data?

I have fitted the mixture of Gaussians to the natural log of the data. I know that the model is not a very good fit to the data in the tail region, however in the high density region the actual data ...
Karina's user avatar
  • 85
1 vote
1 answer
144 views

Closed form solution of multivariate Gaussian over mixture of multivariate Gaussians

Suppose I have three variables $X_{1}, X_{2}$ and $Y$, where $X_{1}, X_{2}$ are continuous and $Y$ is binary. The conditional distribution of $X_{1}, X_{2}$, given $Y$ is a multivariate Gaussian ...
Sergio's user avatar
  • 336
1 vote
1 answer
68 views

Coefficients of Gaussian mixture

This is in context of Gaussian mixtures $$p(\boldsymbol{x}) = \sum_{k=1}^K \pi_k\cal{N}(\boldsymbol{x}|\boldsymbol{\mu_k},\boldsymbol{\Sigma_k})$$ Bishop mentions on Page-111 Also, the requirement ...
muser's user avatar
  • 135
2 votes
2 answers
605 views

p(x) in Gaussian mixture model

I've recently learned about Gaussian mixture models (GMM) in school. The formula for a GMM goes something like this: $$p(x) = \sum_{k=1}^{K} \pi_i\mathcal{N}(x|\mu_i,\,\sigma^{2}_i)$$ Now, I know from ...
Dude156's user avatar
  • 123
1 vote
0 answers
17 views

When imputing missing labels Y1 == NaN during training, how do additional target vectors (Y2 != NaN) impact learning Y1==NaN?

I am training a Mixture Density Network (MDN) to map from continuous input vectors X to continuous targets Y [i.e. X -> Y]. There are missing labels on vector Yi, which I impute from the mdn (as ...
j_bayes's user avatar
  • 11
0 votes
1 answer
97 views

Could the likelihood increase monotonically in a misspecified EM algorithm?

I am dealing with the estimation of a Gaussian Hidden Markov Model with conditional distribution given the first-order Markov state $S_t = j,\ j=1,...,J$ $$ Y_t|S_t=j\sim N(0,\sigma^2_j) $$ where the ...
Mr Frog's user avatar
  • 329
0 votes
0 answers
28 views

Metric to check the number of clusters in one dimentional data

I have a set of 1D data as shown below. I want to find a metric that represents the number of clusters in data. Is there any suitable metric that matches my criteria? Example 1D data list: [68, 3, 3, ...
sammy17's user avatar
0 votes
0 answers
36 views

Is there methods for splitting a Guassian Mixture model into separate unimodal distributions?

I want to apply an algorithm which usually works on unimodal distributions on multimodal distributions, I'm considering doing this by splitting the multimodal distributions into multiple unimodal ...
MicroscooterMan's user avatar
1 vote
0 answers
14 views

How do I make a table that prints the Mean and its corresponding weight of a Gaussian Mixture model?

I am successfully generating weights and means for a GMM, but I'm trying to get the results to print in a way that shows the corresponding mean for a weight. Here's what I have now: ...
user1643103's user avatar
1 vote
0 answers
107 views

Finding Probability that data is taken from a Gaussian Mixture Model

I'm using scikit's Gaussian Mixture to obtain a 2 component mixture model for some 1-D data. Having obtained a model, I want to test how well a different data set matches this mixture model, or more ...
Hereami's user avatar
  • 33
2 votes
1 answer
680 views

Calculating weighted covariance matrix of a weighted finite mixture of multivariate normal distributions

I am trying to calculate the weighted covariance matrix for a finite mixture of multivariate normal distributions. I read this post here and this one here, but the first post is focused on uniformly ...
BeginnersMindTruly's user avatar

1
2 3 4 5
13