Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
4
votes
0answers
69 views

How to build a probability distribution from locations with accuracies?

I have a set of $n$ GPS locations $l_i$ with latitude, longitude in degrees and accuracy in meters, corresponding to $3 σ$, i.e. the probability ≈ 0.997) $(lat_i, lng_i, acc_i)$ or $(lat_i, lng_i, σ_i ...
4
votes
1answer
609 views

Derivation of M step for Gaussian mixture model

Summary So to summarize my question, how can I take \begin{align} = \sum_{i=1}^{n}W_{i1} \left(log (1-\sum_{j=2}^{K}\pi_j) -\frac{1}{2} log(|\Sigma_1|) -\frac{d}{2} log(2\pi) -\frac{1}{2}(x_i-\mu_1)^{...
4
votes
0answers
171 views

Implementation of EM algorithm confusion

Here EM algorithm manually implemented, there's a question of the implementation in R of the EM algorithm for 2 mixed gaussians. The answer has a supposedly correct implementation. However, don't the ...
4
votes
0answers
324 views

Entropy of a mixture of Gaussians

I need to estimate as fast and accurately as possible the differential entropy of a mixture of $K$ multivariate Gaussians: $$ \mathcal{H}[q] = -\sum_{k=1}^K w_k \int q_k(\textbf{x}) \log \left[\sum_{...
4
votes
0answers
4k views

Correct number of components in GMM according to BIC and AIC plots

I have applied GMM(Gaussian Mixture Model) to my data set and I have plotted the resulting BIC(Bayesian Information Criterion) and AIC(Akaike Information Criterion) for different number of components. ...
4
votes
0answers
233 views

Convergence of k-means or EM on Mixture of Gaussians

There are many algorithms for learning mixture of Gaussians but typically k-means/EM is used in practice. My question is related to the performance of k-means/EM for MoG. Recently, I came across this ...
4
votes
0answers
1k views

Finite mixture models with bounded data

I am trying to fit a finite mixture model to a dependent variable which is bounded (practically) between -0.594 and 1 (theoretically, the latent variable is bounded between -Inf - 1). The data are ...
3
votes
1answer
5k views

Fit mixture of distributions to your time-series data in R

I have time-series data containing 1440 observations and the plot of the data is I want to fit the Gaussian Mixture Models (GMM) to the above plot, and for the same I am using Mclust function of ...
3
votes
1answer
899 views

Scikit-Learn Gaussian Mixture: How can log-probabilities be positive? [closed]

I am fitting a Gaussian Mixture model: gm = GaussianMixture(n_components=K) gm.fit(features) When I do: ...
3
votes
2answers
9k views

Significant difference between time series - Can I do this?

I'd like to know whether the solution proposed below is valid/acceptable and any justification available. We have two biological conditions, and for each condition we measured 3 time series, so at ...
3
votes
2answers
538 views

Distribution of the exponential of a mixture?

Suppose that $X$ is distributed as a finite mixture of normals $$\sum_{j=1}^k w_j \phi(x;\mu_j,\sigma_j^2).$$ Is $\exp(X)$ distributed as a finite mixture of log-normal distributions?
3
votes
2answers
71 views

Why does it appear impossible to fit Gaussians to arbitrary probability density functions $p$?

I want to fit a Gaussian $q$ to a pdf $p$ by minimizing the energy $E = -\int q(x) \log p(x) dx$. This should result in a "delta function" Gaussian with $\sigma \rightarrow 0$ and $\mu \...
3
votes
1answer
508 views

Difference Between Latent Class Analysis and Mixture Models

I have been trying to look into latent class analysis and don't exactly understand what it is. Is it basically the expectation maximization using and analyzing the classes formed? The resources on the ...
3
votes
4answers
319 views

PCA for probability vectors

Is there a procedure equivalent to principal component analysis (PCA) for probability vectors? I have an n-by-m array where every column sums to one, and all entries are positive. PCA works in ...
3
votes
1answer
2k views

Derivation of M-step in EM algorithm for mixture of Gaussians

I am trying to derive the parameter estimation equations for the M-step of the expectation maximization (EM) algorithm for a mixture Gaussians when all Gaussians share the same covariance matrix $\...
3
votes
2answers
4k views

Gaussian Process Regression With Multiple Inputs?

Is it possible to use a Gaussian Process to relate multiple independent input variables (X1, X2, X3) to an output variable (Y)? More specifically, I would like to produce a regression graph like the ...
3
votes
1answer
499 views

Are growth mixture models just Gaussian mixtures applied to coefficients of polynomials fitted to time-series data?

Am I understanding correctly that growth mixture model is just Gaussian mixtures applied to coefficients of polynomials fitted to the time-series data? For example, we have 1000 cases, with 3 ...
3
votes
2answers
36 views

Sum of random variables that follow a finite normal mixture distribution

Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as $f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
3
votes
1answer
834 views

How to cluster parts of broken line made of points?

I am studying clustering techniques and i am pretty new at this topic. Here is my problem: I created a 5 lines which are made of points. This lines are supposed to be continuous and they look like ...
3
votes
1answer
5k views

Sampling from a multivariate Gaussian mixture model [duplicate]

How can I generate multi-dimensional data from a (estimated) Gaussian mixture pdf? In general, what would be ways to generate multi-dimensional data from a pdf? I read rejection sampling can be used, ...
3
votes
1answer
4k views

Differences linear discriminant analysis and Gaussian mixture model

I know that there are topics about this question but in my view, the answers are not clear enough. I don't understand the main difference between Linear Discriminant Analysis (LDA) and Gaussian ...
3
votes
1answer
940 views

conjugate prior for (multivariate) Gaussian mixtures (with known mean and covariance)?

Say I have a (multivariate) Gaussian mixture model $$p(x)=\sum_k\pi_iN(\mu_i,\Sigma_i),$$ of which the $\boldsymbol\mu$ and $\boldsymbol\Sigma$ are known, so the likelihood function of the ...
3
votes
1answer
3k views

Difference between multivariate Gaussian distribution and multivariate Gaussian mixture model

I would like to know the difference between a multivariate Gaussian distribution and multivariate Gaussian mixture model. Can someone provide an intuitive and/or detailed explanation? Thanks.
3
votes
1answer
504 views

Convert normal pdf into lognormal pdf

I have fitted a mixture of 3 normal distributions to my log-transformed data Y, as the package I'm using cannot fit a mixture of lognormal distributions. My questions is: how I can convert pdf from 3 ...
3
votes
1answer
400 views

Is there other types of mixture distribution besides the normal mixture

There are quite a lot of study on the normal mixture distributions, say, $X=Y*Z$,where $Z$ is a normal r.v. and Y is a r.v. follows other distributions and $Y$ and $Z$ are independent. Some well-known ...
3
votes
2answers
2k views

Algorithms for 1D Gaussian mixture with equal variance and noise cluster

I would like to fit a Gaussian mixture model to some data. The data is 1D and I want to constrain all the Gaussians to have equal variance. I would also like to have a uniform background noise cluster ...
3
votes
1answer
31 views

Difference of independent random variables that is not unimodal

This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
3
votes
1answer
267 views

Can a Gaussian Mixture model be fit with a continuous response variable?

Does the Gaussian Mixture model require binary and multiclass response/target variable (classification), or can the target vector consist of all real numbers (continuous variable, regression)? Why is ...
3
votes
1answer
2k views

comparing gaussian mixture models with and without regularization

I am using Gaussian mixture models for clustering a bunch of data sets. On some data sets a regularization value is often used/suggested (see the MATLAB example for Fisher's Iris data which sets ...
3
votes
1answer
2k views

How to deal with categorical feature in a Gaussian Mixture model clustering model

I am performing clustering by Gaussian Mixture model using EM algorithm in R. U use the mclust package. My data (205 observations and 25 variables) has both ...
3
votes
2answers
2k views

Manually generate random sample in Gaussian mixture model

I want to generate (manually) a random sample in the Gaussian mixture model: $$f_{\theta}(x) = \sum_{k = 1}^{K}\pi_k f_{\mathcal N(\mu_k, \sigma^2_k)}(x)$$ Here is my work: ...
3
votes
2answers
344 views

Universal Approximation Capabilities of Mixture Models

I am looking for two reference incl. proofs showing 1) that a discrete Mixture of Gaussians can asymptotically approximate any (well behaved) continuous density 2) that a discrete Mixture of ...
3
votes
1answer
680 views

Defining overlapping periods

I have a dataset containing the abundance of migrating bird species. In the figure below you can see that there are two "bell" shapes that are overlapping somewhere around September. One of the bell ...
3
votes
1answer
261 views

GMM weighting change during marginalization

Suppose one has a multivariate Gaussian Mixture Model: $$ \text{pdf}(\vec{x}) = \sum_{i=1}^N w_i \mathcal{N}(\mu^{(i)}, \Sigma^{(i)}) $$ Suppose $\vec{x} = \{\vec{a},\vec{b}\}$ and we marginalize ...
3
votes
2answers
322 views

Gaussian Mixture, sampling and interpretation

I came across the following which I have troubles understanding and would appreciate your help. Suppose we want to fit a Gaussian mixture model (GMM) to a $t$-dstribution with some degrees of ...
3
votes
1answer
1k views

Clustering data with Fourier series representation

We are analyzing temporal behavioral patterns across many users and we want to cluster users in order to understand "natural types of behavior". Our idea is to represent the data (672 bins for each ...
3
votes
2answers
931 views

Mixture of Gaussians on Log of Data

I am practicing Mixture of Gaussians and found the below dataset snoq, which is the precipitation amounts recorded at a US region, with ...
3
votes
1answer
747 views

Does a Gaussian mixture model always imply a within-class multivariate normal probability distribution?

If I use a latent profile analysis (Gaussian Mixture Model) to model my observed multivariate probability distribution as a mixture (K-classes) of conditionally-independent normal pdfs, does this ...
3
votes
1answer
765 views

EM algorithm with a component for outliers

i have a vector of measurements from one to three classes, which can be modeled by gaussian distributions. There are some outliers in the data. I use the EM algorithm to learn the parameters of the ...
3
votes
1answer
2k views

matlab gmdistribution.fit 'Regularize' - what regularization method?

I am wondering what is behind matlab 'Regularize' option for method gmdistribution.fit. If it is simply adding a 'little' value to diagonal elements of covariance matrix, so as to make covariance ...
3
votes
1answer
406 views

Correlated random variables from mixture distributions

Let I have three random variables whose density is a mixture of two Normals with these parameters: $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$ $\...
3
votes
1answer
339 views

Check on intuition behind infinite mixture models for clustering

I'm trying to better understand the intuition and practical application of infinite mixture models (Dirichlet Process) and finite mixture models. For example, say I have a data set on which I run a ...
3
votes
1answer
844 views

How to reestimate GMMs in a HMM-GMM

Context: Automatic Speech Recognition I understand the training of a pure HMM with Baum-Welch: Expectation step compute $\gamma_t(i) = P(q_t=i |O,\lambda)$ //p(passing state $i$ at frame $...
3
votes
2answers
513 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian (...
3
votes
1answer
3k views

Number of free parameters in Gaussian mixture models

When comparing GMM models with different number of components (i.e number of Gaussians) one penalizes the likelihood for the total number of free parameters in the mixture model. If the data is in $D$ ...
3
votes
0answers
31 views

Expectation-Maximisation derivations [duplicate]

I've come across a few different sources on expectation-maximisation which I can't quite match up. The CS229 lecture 8 [1] states that the function we must write down and maximise is: $$ Q_1 = \sum_{...
3
votes
0answers
88 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
3
votes
0answers
185 views

Conditional sampling from a multivariate Gaussian Mixture

I am using scikit-learn to fit a gaussian mixture on a non-parametric multivariate distribution with three variables $ \mathbf{X} = (X_1, X_2, X_3) $ I want to sample from that distribution given ...
3
votes
0answers
180 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
3
votes
3answers
1k views

GMM EM algorithm complexity per iteration

I was fitting GMM clusters with diagonal covariance on my data using EM with $n$ (=5e6) points, each having $m$ (=160) ...

1 2
3
4 5
12