Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
7 views

Kurtosis of a Mixture distribution: error in formula/code?

I have been using the text Fruhwirth-Shnatter (2006) for analytic formulas on normal mixture distributions. I implemented the formula for the fourth moment but, after converting it to excess Kurtosis, ...
3 votes
0 answers
33 views

Can a Gaussian Process predict random events?

I know that we can use Gaussian processes effectively for function approximation and regression. However,suppose there is a sequence of points in time $S = \{s_1, s_2, \dots, s_n\}$, where $s_i$ can ...
3 votes
2 answers
72 views

GMM derivation for diagonal covariance matrices

I was trying to understand the derivation of M step in the EM algorithm for GMM. All the resources available consider only "full covariance" matrices. I wanted to implement GMM for "...
1 vote
0 answers
31 views

Closed Form Solution for MLE parameter defining Linear Combination of two multivariate normal distributions

I have one set of $n$ observations which can be described as a single vector sampled from a multivariate normal distribution of the following form: $$ (1-\lambda)\mathbb{I}_n + \lambda \Sigma_{n} $$ ...
1 vote
1 answer
266 views

Three component mixture model for element concentration using mixtools in R

As an update to a previously posed question, Fitting a mixture distribution for two approximately normally distributed populations using mixtools in R , I have now fit a three component mixture ...
0 votes
0 answers
12 views

What is the function space corresponding to the Gaussian-mixture parameter space?

If the parameter $\theta$ is Gaussian with mean $0$ and covariance matrix $\Sigma$, then the function $f(x;\theta)=x^T\theta$ is a Gaussian process indexed by $x$, which can be characterized by a mean ...
3 votes
3 answers
6k views

Mixed data in Gaussian Mixture Models

Is it possible to use a dataset with mixed variables such as continuous, ordered, and categorical variables and cluster the data using the Gaussian Mixed Model with EM algorithm. I cannot find ...
3 votes
1 answer
64 views

Interpretation of $\sigma$ in Gaussian mixture

I have a distribution of a variable that was normalized with plt.hist and then fitted with a sum of gaussian curves $g_M = \displaystyle\sum_i\frac{w_i}{\sigma_i \...
3 votes
1 answer
661 views

How to get number of iterations in EM-algorithm using R mclust gaussian mixture model

I am clustering data using the mclust function from the R mclust package. I am struggling to get the number of iterations the EM ...
0 votes
0 answers
12 views

Under What Conditions Does a Gaussian Mixture Model (GMM) Have Maximum Entropy?

Introduction I'm delving into Gaussian Mixture Models (GMMs) within unsupervised learning frameworks and am particularly interested in their statistical properties, with a focus on entropy. Entropy ...
0 votes
0 answers
15 views

Expected Variance of EM Estimator in GMM with Respect to Observations

Title: Variance of EM Estimator in GMM with Respect to Observations Body: I'm estimating a parameter S from observations X and <...
2 votes
2 answers
1k views

Decision boundary between two Gaussians of unequal variance

This question is concerning a similar problem as mentioned in this question. The only difference is that in my case the variances are unequal. To recap, consider a two class scenario. At the decision ...
0 votes
0 answers
8 views

Multivariate mixture models with INLA: Relating CAR random effect to the MVN covariance matrix

For a research project I am trying to implement a multivariate mixture model on areal data using Integrated Nested Laplace Approximation (INLA). Let $y_{i,d}$ be a data point, where $i = 1, 2, \cdots, ...
0 votes
1 answer
894 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
0 votes
0 answers
12 views

Online mixture inference; better alternatives than windowed EM?

I have an online Gaussian mixture estimation problem that I would appreciate some input on. To be more precise, I have a stream of scalar observations $x_1, x_2, \dotsc$ arriving over time which are ...
0 votes
0 answers
23 views

Can I assume that this is a GMM?

I'm trying to find the MLE for the parameters of the following distribution: $$f(x) = a \ \mathcal{N}(\mu_a, 1) + \beta \ \mathcal{N}(\mu_\beta, 1)$$ Taking the log likelihood of this complicates ...
1 vote
1 answer
244 views

Simulating with mixture distribution

I have fitted a gaussian mixture distribution to residuals, and now I want to simulate the residuals. However, I want the model to be independent of the time steps(have it as an input to the model). ...
2 votes
1 answer
319 views

Is there any background for constraining covariances on fitting GMM?

On clustering data using GMM model, I often see the option to constrain covariances of each clustered GMM. For example, http://scikit-learn.org/0.16/auto_examples/mixture/plot_gmm_classifier.html ...
1 vote
2 answers
4k views

Testing for Unimodality or Bimodality Data Using MATLAB

I am trying to figure out what I did wrong or what I could do to get accurate results. I have n vectors of data, and I am trying to decide whether each dataset is unimodal or bimodal. I assumed that ...
0 votes
0 answers
11 views

X and Y are correlated, errors in both X and Y but error variances unknown; How to predict X|Y or Y|X? Deming, bivariate gaussian ellipses, other?

Seeking relationships between two variable, both with random gaussian errors; ratio of error variances is unknown, no correlation of errors in X and Y, but another unknown variable Z (unmeasured) may ...
0 votes
0 answers
13 views

Computational issue for finding the JS divergence

I had two datasets with 144 points in 2 dimension then I used he sklearn library to fit the GMM and that fits well , I checked for the BIC values while fitting the model and chooses the model with low ...
0 votes
0 answers
21 views

Question about Gaussian Mixture model [duplicate]

suppose I have two parameters: $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and suppose I mapped them to PDF using the gaussian mixture ...
0 votes
0 answers
33 views

Derivation of EM algorithm for Gaussian mixture

I am going through Expectation Maximization (EM) algorithm derivation for Gaussian Mixture models. I understand it except for a small detail. So, the general idea of EM is to maximize the expectation ...
2 votes
0 answers
50 views

Derive ELBO for Mixture of Gaussian

I am working through "Variational Inference: A Review for Statisticians" by Blei et al. (see https://arxiv.org/abs/1601.00670) and they illustrate Variational Inference using a Bayesian ...
0 votes
0 answers
79 views

Understanding of Gamma distribution as precision prior in Bayesian inference for Gaussian

Christopher M. Bishop in his book "Pattern Recognition and Machine Learning" nicely explains where does Student t-distribution $St(x|\mu,\lambda,\upsilon)$ originate into. In Chapter 2, it ...
0 votes
1 answer
47 views

Identify outliers in testing data based on trained Gaussian mixture model

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
1 vote
0 answers
82 views

Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
1 vote
1 answer
71 views

Gaussian Mixture Model with Minkowski distance

Gaussian Mixture Models assume Mahalanobis distance (essentially L2). Is it possible to use Lp distance in a GMM? Intuitively, in 1-space, distance is clear. In 2-space, the relation between the two ...
0 votes
0 answers
15 views

Are combinations of variables a problem in multinomial mixture models?

I am quite new to cluster analysis and I am currently using Gaussian-multinomial mixture models. I have a sample size of 10000 people with some continuous variables and about 14 categorical variables (...
7 votes
2 answers
14k views

Standard deviation for weighted sum of normal distributions

I have 2 normally distributed random variable $H_0$ and $H_1$, which are combined to give the weighted distribution $H$ as follows: $H_0 \sim N(\mu_0, \sigma_0)$ $H_1 \sim N(\mu_1, \sigma_1)$ $$f_H ...
2 votes
2 answers
160 views

MAP estimation for a Gaussian mixture using EM. Concerns with the covariance update formula

I am implementing the EM algorithm for a Gaussian mixture model with prior; that is, I am using the EM algorithm to find the MAP estimate, rather than the ML estimate. As briefly discussed in section ...
1 vote
1 answer
329 views

Is it a known phenomenon for the variance of a component (GMM) to increase without stopping? [closed]

I know it can happen for it to decrease dramatically as it overfits on a single datapoint. But I've never read about a component "taking everything over". See the following images (circles are stddevs)...
1 vote
1 answer
122 views

Generate marginally dependent (with predetermined covariance) but conditionally independent data from a Mixture of Gaussians

Suppose you have three variables $y\in\{0,1\}$ and $x_1\in\mathbb{R}$ and $x_2\in\mathbb{R}$. I want to produce data with the following generative process which corresponds to a Mixture of Gaussians (...
1 vote
0 answers
29 views

Linear Discriminant Analysis with unlabeled data

In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following: From the mixture formulation [that is, ...
1 vote
0 answers
296 views

What does mixture refer to in LCMM and HLME in R

I am trying to use the HLME and LCMM functions to fit latent class mixed models to my data. Here are the documentations to both of them: https://www.rdocumentation.org/packages/lcmm/versions/1.8.1.1/...
7 votes
2 answers
2k views

Skewness of a mixture density

I am fitting a gaussian mixture to financial data. My mixture density is given by: $f(l)=πϕ(l;μ_1,σ^2_1)+(1−π)ϕ(l;μ_2,σ_2^2)$ I calculated the skewness of the data already. Now, I want to look at ...
0 votes
0 answers
17 views

Mixture Model: Data Consist of Continuous and Binary Features

I have a features like below id x1 x2 x3 x4 x5 id1 0.4 1.4 5.6 1 0 id2 -0.01 0.5 -3.4 0 1 where x1, x2, x3 are continuous features and x4 and x5 are binary. The goal is to find $k$ clusters using ...
0 votes
0 answers
15 views

Supervised classifier for nested interval data and ordinal classes

I'm having trouble formalizing the following classification problem: Let $x_i$ denote univariate (scalar), continuous, real data points Let $y_i \in \mathbb{N}$ be their corresponding labels Classes ...
0 votes
1 answer
907 views

Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]

In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
0 votes
1 answer
70 views

Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model

I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
0 votes
1 answer
40 views

Assumptions and setting for bayesian mixture model (for RJMCMC)

I want to understand about Bayesian mixture model discussed in RJMCMC paper (Richardson and Green, 1997) (https://academic.oup.com/jrsssb/article/59/4/731/7083042) I also posted similar question ...
3 votes
1 answer
1k views

EM algorithm with a component for outliers

I have a vector of measurements from one to three classes, which can be modeled by Gaussian distributions. There are some outliers in the data. I use the EM algorithm to learn the parameters of the ...
7 votes
1 answer
8k views

Anomaly detection with gaussian mixture models

I am new to the topic, and I am trying to understand how it is possible to perform anomaly detection by using gaussian mixture models. Could you please give me some hints about literature on the topic?...
0 votes
0 answers
67 views

EM algorithm get new parameters by optimizing the Q function (lower bound of likelihood function) or optimizing the likelihood function

We know that in the EM (Expectation-Maximization) algorithm, the E-step determines the $Q$ function by calculating expectations, which is a lower bound of the likelihood function. In the M-step, by ...
2 votes
1 answer
274 views

Why gaussian mixture model is sensitive to cluster size?

I did an experiment where I generated 3 well separated clusters with a different multivariate gaussian distribution for each cluster. One of the clusters contains 1000 points and the other two ...
0 votes
2 answers
68 views

Flexmix maxima are not where they are expected to be

For my dataset I have plotted the density with ggplot. As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus. ...
0 votes
0 answers
26 views

Decomposing into Gaussian components using Bhattacharya from topFishR

I am working with fishery data. I have a data vector called SFL that contains the sizes of the fish caught. Here is some sample data: ...
0 votes
1 answer
54 views

Confidence interval for unsymmetrical Gaussian Mixure Model PDFs?

Let Y be a vector of observations. A Gaussian Mixure Model (GMM) is fit to the dataset. The distribution can appear unsymmetrical, with different thickness of tails in both sides. What is the best way ...
12 votes
1 answer
3k views

Gaussian Mixture and Method of Moments

Given solely the first $n$ moments $m_1,\dots,m_n$ of a random variables $X\in\mathbb{R}$, I was wondering whether there exists a direct methodology to approximate $X$ with a Gaussian Mixture ?
0 votes
1 answer
97 views

Fitting a Gaussian Mixture Model with known share of noise/outliers

A Gaussian Mixture model is fitted by the Expectation-Maximization algorithm. This fairly simple iterative algorithm consists of two steps and the initialization. Initialization (for ...

1
2 3 4 5
13