Skip to main content

Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0 votes
1 answer
29 views

Bayesian MCMC and Only Updating Some Variables at a Time

I want to do Bayesian MCMC on a Gaussian Mixture Model. But, I want to update the means, weights, and covariance matrix for a single component separate from the others. Would there be the issue of ...
LifeisGood94's user avatar
0 votes
1 answer
33 views

EM-algorithm for spatial data

I am very new to Geostatistics (Modeling spatial data) and have some questions: 1- I found that in many literature, the spatial random field is divided into spatial bins. That is, suppose I am ...
Dr. Statistics's user avatar
2 votes
0 answers
34 views

Testing for Gaussian Mixtures

Suppose I observe data from a univariate distribution $\mathcal{D}$ with zero mean, and unit variance. We know that either $\mathcal{D} = \mathcal{D}_0 = \mathcal{N}(0,1)$ is standard Gaussian or $\...
Claudio Moneo's user avatar
1 vote
0 answers
42 views

Linear Regression with gaussian mixture prior

In linear regression, we assume that the output variable is Normally distributed, i.e., $p(y|\mathbf{x}, \mathbf{w}) = N (y | \mathbf{w}^T\mathbf{x}, \sigma^2_y)$. I want to assign a mixture of ...
maktukmak's user avatar
3 votes
0 answers
41 views

Can a Gaussian Process predict random events?

I know that we can use Gaussian processes effectively for function approximation and regression. However,suppose there is a sequence of points in time $S = \{s_1, s_2, \dots, s_n\}$, where $s_i$ can ...
Hassan Ali's user avatar
1 vote
0 answers
42 views

Closed Form Solution for MLE parameter defining Linear Combination of two multivariate normal distributions

I have one set of $n$ observations which can be described as a single vector sampled from a multivariate normal distribution of the following form: $$ (1-\lambda)\mathbb{I}_n + \lambda \Sigma_{n} $$ ...
A Friendly Fish's user avatar
0 votes
0 answers
12 views

What is the function space corresponding to the Gaussian-mixture parameter space?

If the parameter $\theta$ is Gaussian with mean $0$ and covariance matrix $\Sigma$, then the function $f(x;\theta)=x^T\theta$ is a Gaussian process indexed by $x$, which can be characterized by a mean ...
W. Zhu's user avatar
  • 135
3 votes
2 answers
135 views

GMM derivation for diagonal covariance matrices

I was trying to understand the derivation of M step in the EM algorithm for GMM. All the resources available consider only "full covariance" matrices. I wanted to implement GMM for "...
Equation_Charmer's user avatar
3 votes
1 answer
70 views

Interpretation of $\sigma$ in Gaussian mixture

I have a distribution of a variable that was normalized with plt.hist and then fitted with a sum of gaussian curves $g_M = \displaystyle\sum_i\frac{w_i}{\sigma_i \...
poki456's user avatar
  • 33
0 votes
0 answers
32 views

Under What Conditions Does a Gaussian Mixture Model (GMM) Have Maximum Entropy?

Introduction I'm delving into Gaussian Mixture Models (GMMs) within unsupervised learning frameworks and am particularly interested in their statistical properties, with a focus on entropy. Entropy ...
Alireza's user avatar
  • 113
0 votes
0 answers
18 views

Expected Variance of EM Estimator in GMM with Respect to Observations

Title: Variance of EM Estimator in GMM with Respect to Observations Body: I'm estimating a parameter S from observations X and <...
Alireza's user avatar
  • 113
0 votes
0 answers
15 views

Multivariate mixture models with INLA: Relating CAR random effect to the MVN covariance matrix

For a research project I am trying to implement a multivariate mixture model on areal data using Integrated Nested Laplace Approximation (INLA). Let $y_{i,d}$ be a data point, where $i = 1, 2, \cdots, ...
BasMts's user avatar
  • 23
0 votes
0 answers
13 views

Online mixture inference; better alternatives than windowed EM?

I have an online Gaussian mixture estimation problem that I would appreciate some input on. To be more precise, I have a stream of scalar observations $x_1, x_2, \dotsc$ arriving over time which are ...
ummg's user avatar
  • 145
0 votes
0 answers
24 views

Can I assume that this is a GMM?

I'm trying to find the MLE for the parameters of the following distribution: $$f(x) = a \ \mathcal{N}(\mu_a, 1) + \beta \ \mathcal{N}(\mu_\beta, 1)$$ Taking the log likelihood of this complicates ...
John Katsantas's user avatar
0 votes
0 answers
11 views

X and Y are correlated, errors in both X and Y but error variances unknown; How to predict X|Y or Y|X? Deming, bivariate gaussian ellipses, other?

Seeking relationships between two variable, both with random gaussian errors; ratio of error variances is unknown, no correlation of errors in X and Y, but another unknown variable Z (unmeasured) may ...
David's user avatar
  • 1
0 votes
0 answers
16 views

Computational issue for finding the JS divergence

I had two datasets with 144 points in 2 dimension then I used he sklearn library to fit the GMM and that fits well , I checked for the BIC values while fitting the model and chooses the model with low ...
tiger's user avatar
  • 1
0 votes
0 answers
21 views

Question about Gaussian Mixture model [duplicate]

suppose I have two parameters: $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and $(\mu_{1},\Sigma_{1},\alpha_{1},\mu_{2},\Sigma_{2})$ and suppose I mapped them to PDF using the gaussian mixture ...
Andyale's user avatar
  • 133
0 votes
0 answers
37 views

Derivation of EM algorithm for Gaussian mixture

I am going through Expectation Maximization (EM) algorithm derivation for Gaussian Mixture models. I understand it except for a small detail. So, the general idea of EM is to maximize the expectation ...
baronett's user avatar
2 votes
0 answers
115 views

Derive ELBO for Mixture of Gaussian

I am working through "Variational Inference: A Review for Statisticians" by Blei et al. (see https://arxiv.org/abs/1601.00670) and they illustrate Variational Inference using a Bayesian ...
PSLS's user avatar
  • 41
0 votes
0 answers
105 views

Understanding of Gamma distribution as precision prior in Bayesian inference for Gaussian

Christopher M. Bishop in his book "Pattern Recognition and Machine Learning" nicely explains where does Student t-distribution $St(x|\mu,\lambda,\upsilon)$ originate into. In Chapter 2, it ...
baronett's user avatar
0 votes
1 answer
53 views

Identify outliers in testing data based on trained Gaussian mixture model

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
0 answers
95 views

Evaluate CDF and outliers of multidimensional Gaussian mixture [closed]

I use Gaussian mixture model (GMM) to infer probability density of multidimensional data written as: $p(x) = \sum_{j=1}^{K}\pi_j*N(x|\bf \mu_j, \Sigma_j)$, where $K$ is a number of mixtures, $\pi_j$ ...
baronett's user avatar
1 vote
1 answer
102 views

Gaussian Mixture Model with Minkowski distance

Gaussian Mixture Models assume Mahalanobis distance (essentially L2). Is it possible to use Lp distance in a GMM? Intuitively, in 1-space, distance is clear. In 2-space, the relation between the two ...
olivarb's user avatar
  • 173
0 votes
0 answers
17 views

Are combinations of variables a problem in multinomial mixture models?

I am quite new to cluster analysis and I am currently using Gaussian-multinomial mixture models. I have a sample size of 10000 people with some continuous variables and about 14 categorical variables (...
Lander's user avatar
  • 23
2 votes
2 answers
224 views

MAP estimation for a Gaussian mixture using EM. Concerns with the covariance update formula

I am implementing the EM algorithm for a Gaussian mixture model with prior; that is, I am using the EM algorithm to find the MAP estimate, rather than the ML estimate. As briefly discussed in section ...
ummg's user avatar
  • 145
1 vote
0 answers
34 views

Linear Discriminant Analysis with unlabeled data

In section 4.4.5 "Logistic regression or LDA?" of Elements of Statistical Learning by Friedman, Tibshirani and Hastie, it is claimed the following: From the mixture formulation [that is, ...
Sergio's user avatar
  • 336
1 vote
1 answer
124 views

Generate marginally dependent (with predetermined covariance) but conditionally independent data from a Mixture of Gaussians

Suppose you have three variables $y\in\{0,1\}$ and $x_1\in\mathbb{R}$ and $x_2\in\mathbb{R}$. I want to produce data with the following generative process which corresponds to a Mixture of Gaussians (...
Sergio's user avatar
  • 336
0 votes
0 answers
19 views

Mixture Model: Data Consist of Continuous and Binary Features

I have a features like below id x1 x2 x3 x4 x5 id1 0.4 1.4 5.6 1 0 id2 -0.01 0.5 -3.4 0 1 where x1, x2, x3 are continuous features and x4 and x5 are binary. The goal is to find $k$ clusters using ...
AnonymousJ's user avatar
0 votes
0 answers
16 views

Supervised classifier for nested interval data and ordinal classes

I'm having trouble formalizing the following classification problem: Let $x_i$ denote univariate (scalar), continuous, real data points Let $y_i \in \mathbb{N}$ be their corresponding labels Classes ...
themodelguy's user avatar
0 votes
1 answer
89 views

Jacobian and proposal ratio of Birth/death step in RJMCMC of Gaussian mixture model

I am asking questions regarding RJMCMC several times in this site. Some of my questions are answered and some are unanswered. It didn't clarify all of my unclear points but I am glad that I have ...
Kyungmin's user avatar
0 votes
1 answer
44 views

Assumptions and setting for bayesian mixture model (for RJMCMC)

I want to understand about Bayesian mixture model discussed in RJMCMC paper (Richardson and Green, 1997) (https://academic.oup.com/jrsssb/article/59/4/731/7083042) I also posted similar question ...
Kyungmin's user avatar
0 votes
0 answers
93 views

EM algorithm get new parameters by optimizing the Q function (lower bound of likelihood function) or optimizing the likelihood function

We know that in the EM (Expectation-Maximization) algorithm, the E-step determines the $Q$ function by calculating expectations, which is a lower bound of the likelihood function. In the M-step, by ...
yi ren's user avatar
  • 1
2 votes
1 answer
377 views

Why gaussian mixture model is sensitive to cluster size?

I did an experiment where I generated 3 well separated clusters with a different multivariate gaussian distribution for each cluster. One of the clusters contains 1000 points and the other two ...
striker22's user avatar
  • 323
0 votes
2 answers
74 views

Flexmix maxima are not where they are expected to be

For my dataset I have plotted the density with ggplot. As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus. ...
Lukas's user avatar
  • 111
0 votes
0 answers
29 views

Decomposing into Gaussian components using Bhattacharya from topFishR

I am working with fishery data. I have a data vector called SFL that contains the sizes of the fish caught. Here is some sample data: ...
sermomon's user avatar
  • 113
0 votes
1 answer
61 views

Confidence interval for unsymmetrical Gaussian Mixure Model PDFs?

Let Y be a vector of observations. A Gaussian Mixure Model (GMM) is fit to the dataset. The distribution can appear unsymmetrical, with different thickness of tails in both sides. What is the best way ...
Bahar's user avatar
  • 1
0 votes
1 answer
113 views

Fitting a Gaussian Mixture Model with known share of noise/outliers

A Gaussian Mixture model is fitted by the Expectation-Maximization algorithm. This fairly simple iterative algorithm consists of two steps and the initialization. Initialization (for ...
Klops's user avatar
  • 156
1 vote
1 answer
193 views

Variance of a mixture of normally distributed random variables [duplicate]

Setup Suppose I have two normally distributed random variables, $s_\mathrm{prev}$ (with mean $\mu_\mathrm{prev}$ and variance $\sigma^2_\mathrm{prev}$) and $s_\mathrm{curr}$ (with mean $\mu_\mathrm{...
abcd's user avatar
  • 113
0 votes
0 answers
192 views

Bayesian Gaussian mixture - is my prior correct?

I'd like to sample from the Bayesian Posterior of a Gaussian mixture model, but I am not sure about the correct Bayesian formulation of the latter. Is the following correct? I consider the 1-...
reloh100's user avatar
1 vote
0 answers
71 views

Could one use mixtures of Gaussians to turn MCMC posterior samples into a new prior?

Theoretically in Bayesian inference one could use one experiment's posterior as another experiment's prior, such that knowledge of the parameters accumulates from $p(\theta) \rightarrow p(\theta|\...
Durden's user avatar
  • 1,352
0 votes
0 answers
13 views

What assumptions are made about categorical variables in regression models? [duplicate]

This is the formalization for a continuous regression model as I understand it: Assume that your outcome $Y$ is normally distributed. Assume that your predictors, $X_1, X_2, \cdots X_n$ are normally ...
djpassey's user avatar
0 votes
0 answers
63 views

Clustering correlated variables in Gaussian Mixture Models

I would like to ask it dose not require to consider the correlation of variables if using Gaussian mixture model for clustering, because covariance has already been considered in the model. So the ...
Xiaose's user avatar
  • 1
0 votes
0 answers
22 views

Similarity of two sets of scatter data

I am currently working on a project where I have a dataset of scatter-points in space. What I am doing is that I fit Gaussian Mixture Models (GMM) on the data set and use the model to sample '...
Kelvin Walenta's user avatar
0 votes
0 answers
35 views

How to sample from a given accessible PDF?

I am learning Gibbs Sampling for GMMs. Particularly, given $\boldsymbol \theta$, I must sample from the latent $\boldsymbol z$ before sampling $\boldsymbol x$. The PDF of $\boldsymbol z$ is given as$$ ...
wd violet's user avatar
  • 777
1 vote
0 answers
353 views

What does mixture refer to in LCMM and HLME in R

I am trying to use the HLME and LCMM functions to fit latent class mixed models to my data. Here are the documentations to both of them: https://www.rdocumentation.org/packages/lcmm/versions/1.8.1.1/...
TYL's user avatar
  • 213
2 votes
1 answer
209 views

Gaussian mixture model probabilities

I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
alle_meije's user avatar
1 vote
0 answers
40 views

How to solve gamma from GMM

GMM refers to the Gaussian mixture model, cf. here. Suppose we have $N$ data points (or observations in Statistics) and $K$ Gaussian models to mix. After going through Maximum Likelihood Estimation, ...
Sam Wong's user avatar
  • 177
5 votes
2 answers
549 views

Mixture of Gaussian is not log-concave

I've encountered the statement: For $p\in(0,1),$ the location mixture of standard univariate normal densities $f(x)=p\phi(x)+(1-p)\phi(x-\mu)$ is log-concave if and only if $\Vert\mu\Vert \leq 2.$ ...
12345's user avatar
  • 213
1 vote
1 answer
50 views

Mixture of Two Normals

Suppose we have a data which consists of two normals, x = rnorm(50,mean=1,sd=2) y = rnorm(50,mean=2,sd=3) z = sample( c(x,y) , size = 100, replace=FALSE ) The goal ...
Nicolas Bourbaki's user avatar
0 votes
0 answers
132 views

Model Comparison within Bayesian Gaussian Mixture Model framework

Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model. $$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
Fiodor1234's user avatar
  • 2,286

1
2 3 4 5
13