Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

2
votes
1answer
30 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
1
vote
2answers
55 views

Comparing K-Means and Expectation Maximization on the dataset generated - When does K-Means perform better?

I was experimenting with K-Means and Gaussian Mixture Models (Expectation-Maximization) on the data set that I generated. Here is how the plot for two distributions looks like: Since this was ...
0
votes
0answers
13 views

How do you derive the k-means equation from the gaussian mixture model [closed]

How do you derive the k-means equation from the gaussian mixture model
0
votes
0answers
10 views

Collapsed Gibbs Sampler on Gaussian Mixture Model: testing on held-out data

I'm a newbie to stackexchange so please let me know how to improve the question. I want to compare performance of 2 inference algorithms for GMM: variational inference and collapsed Gibbs sampler. ...
0
votes
0answers
14 views

Combining GMM's in different dimensions

I have input $X$, which follows a distribution $P(X)$, which is best modeled using a mixture of Gaussians. I also have another random variable $T$, which is also best modeled by a mixture of Gaussians....
0
votes
1answer
22 views

Is it important to make a feature scaling before using Gaussian Mixture Model?

Is it important to make a feature scaling before using Gaussian Mixture Model? and why is it important while we are using probability in getting our clusters's parameters (mean and covariance matrix). ...
1
vote
0answers
23 views

Gaussian Mixture Model with labels in Python

I have data X and corresponding labels y and want to fit a Gaussian Mixture model to it. In Matlab, one has the option of specifying initial labels. I am trying to do the same in Python. This is what ...
2
votes
1answer
42 views

Correlated random variables from mixture distributions

Let I have three random variables whose density is a mixture of two Normals with these parameters: $\mu_{1,1}=6.8$, $\mu_{1,2}=6.95$, $\sigma_{1,1}=0.065$, $\sigma_{1,2}=0.055$ and $\alpha_{1}=0.4$ $\...
4
votes
2answers
245 views

Is a GMM-HMM equivalent to a no-mixture HMM enriched with more states?

I'm trying to model sequence data that has 5 hidden states. Observation data conditional to each state is gaussian except for one state for which mixture of 2 gaussians seems more appropriate. ...
2
votes
1answer
387 views

A Gaussian Mixture Model Is a Universal Approximator of Densities

When discussing the concept of mixtures of distributions in my machine learning textbook, the authors state the following: A Gaussian mixture model is a universal approximator of densities, in the ...
0
votes
1answer
31 views

Decompose/split a single multivariate gauss into random gaussian mixture

Say, there is a single $n$-dimensional multivariate Gaussian. $$Gauss_a(\mu_a,\Sigma_a) $$ $\mu_a$ is $1\times n$ vector and $\Sigma_a$ is $n\times n$ matrix. Is there any easy way to decompose/...
0
votes
1answer
24 views

Why do we assume Gaussian margins in Gaussian mixture models [duplicate]

A Gaussian mixture model is a weighted sum of Gaussian densities, i.e., $L(\theta) = \sum_{i=1}^{m} \pi_{i} f(x_i)$ where $m$ is the number of the mixture component. Hence, Gaussian mixture ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
1
vote
0answers
31 views

Conditional sampling from a multivariate Gaussian Mixture

I am using scikit-learn to fit a gaussian mixture on a non-parametric multivariate distribution with three variables $ \mathbf{X} = (X_1, X_2, X_3) $ I want to sample from that distribution given ...
1
vote
0answers
29 views

Entropy of a mixture of Gaussians

I need to estimate as fast and accurately as possible the differential entropy of a mixture of $K$ multivariate Gaussians: $$ \mathcal{H}[q] = -\sum_{k=1}^K w_k \int q_k(\textbf{x}) \log \left[\sum_{...
0
votes
0answers
13 views

Estimating parameters in a mixed distribution

From my samples I have some distributions made up a mixture of 2 to 4 different components. I have the overall density and the proportion of each component in the distribution, but I want to estimate ...
1
vote
0answers
27 views

Can the hidden states of a HMM be interpreted as number of clusters underlying the data?

Trying to understand the physical significance of the number of hidden states of a HMM. Should they be interpreted as number of clusters in the data? If not, why? Or they should be interpreted as the ...
1
vote
0answers
14 views

MIxture model in R to generate noise in data

I have a bit of code in R that adds noise to an harmonic series according to a normal distribution: ...
0
votes
1answer
71 views

Does assumption of normality of each mixture components implies that each margins is normal

I just would like to understand some information about the joint normality and the margins. I read that the normal joint distribution almost always implies that the univariate margins are all normal. ...
2
votes
1answer
37 views

Introduction to Gaussian mixture models

First of all, I am sorry if this question is not acceptable by some of the readers. However, I really read many, many sources about Gaussian mixture models, but all what I found was a short tutorial ...
0
votes
2answers
54 views

Does the distribution of each mixture components have any information of the margins of the variables? [closed]

I just start working with Gaussian mixture models and I just confused about some information, which I really would like to make sure that I understand the model very well. A Gaussian mixture model ...
2
votes
1answer
85 views

Gaussian Mixture Model

with the following code I fit a Gaussian Mixture Model to arbitrarily created data. The code is working. The only thing I encounter is that during the calculation of the multivariate_normal I ...
1
vote
0answers
19 views

Statistical test for comparing two-gaussian mixture

I have a distribution of shape sizes under two different (biological) conditions. From prior knowledge, I do expect there to be two populations. I fit each condition to a two-Gaussian mixture model. ...
1
vote
1answer
46 views

Gaussian mixture models with constrained mixing proportions

I am fitting a Gaussian mixture model to multivariate data and my application suggests constraining the mixing proportions to lie in a pre-determined sub-space. I am curious if such an approach has ...
0
votes
0answers
41 views

How to interpret a GMM and a predict_GMM output

I would like to do outlier detection in a dataset by using e Mixture Model. I'm using the R functions GMM and predict_GMM but I can't understand which is the outlier factor and how to indentify ...
1
vote
0answers
49 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
0
votes
0answers
88 views

Calculating BIC for sklearn.mixture BayesianGaussianMixture

The GaussianMixture function easily allows calculation of BIC. In the source code it is defined as: ...
0
votes
0answers
30 views

Clustering with mclust and cluster packages in R [duplicate]

I need to group my data into few buckets and wanted to use clustering with R. It is important that I cluster it right as my downstream processing largely depends on how well the data is clustered ...
0
votes
1answer
23 views

Gaussian Bayesian Networks and covariance calculation

I have difficulties in understanding the way of calculation of covariance matrix in Gaussian Bayesian Nets (from conditional to joint): The last formula is about to calculates covariance between ...
0
votes
0answers
20 views

How to select the best Baysian GMM with Dirichlet process

I am very new to GMM models and currently trying to make a unsupervised clustering on data. The data has 34 features (dimension). I am thinking of using GMM with Dirichlet process and trying to code ...
2
votes
1answer
77 views

Doesn't the non-Gaussian source assumption of ICA render it practically useless?

Gaussian distributions appear everywhere in nature, indeed this was largely the justification for most classical methods' reliance on assumption of normality. ICA assumes non-Gaussian sources, indeed ...
3
votes
0answers
19 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
2
votes
0answers
24 views

In a normal mixture model, what general happens if the number of groups to be summed far exceeds the dimensions of each normal?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_kI_d) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean vector $\mu$ and d-...
1
vote
0answers
205 views

Correct number of components in GMM according to BIC and AIC plots

I have applied GMM(Gaussian Mixture Model) to my data set and I have plotted the resulting BIC(Bayesian Information Criterion) and AIC(Akaike Information Criterion) for different number of components. ...
0
votes
1answer
89 views

General conditional distributions for multivariate Gaussian mixtures

My question is similar to this one but considers a more general situation. Suppose that $ \vec{x} = (x_1, \dots, x_d) $ and let $$ p(\vec{x}) = \sum_{k=1}^{n} \pi_k \mathcal{N}(\vec{x} | \mu_k, \...
3
votes
1answer
35 views

Sampling posterior of empty cluster in GMM and Gibbs

Consider performing inference via a standard Gibbs sampler for a standard Gaussian Mixture Model (GMM) with $k$ components that are Gaussians $$\mathcal{N}(\mu_{k}, \sigma^{2}_{k})$$ where we assume ...
1
vote
0answers
53 views

How does scikit-learn calculate a data point's probability of belong to normal distribution?

In GMM calculating, we need to calculate the probability of data point $X$ belong to the $kth$ Gaussian Distribution $\mathcal{N}(X_n|\mu_k,\Sigma_k)$. I have read How to calculate the probability of ...
0
votes
1answer
158 views

Anomaly detection on 1D data with multiple gaussian distributions

My core problem is to set a cutoff to my one dimension data between normal with abnormal. I think this is a 'anomaly detection' problem. My Data My data is one dimension, consists with below: (...
0
votes
0answers
24 views

A transformation from uniform random variable to Gaussian mixture

I am attempting to describe a prior_transform for a multivariate Gaussian mixture in order to estimate the evidence integral of that prior convolved with another likelihood distribution. This is ...
1
vote
1answer
95 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
1
vote
1answer
74 views

Conditional mean for mixture of multivariate normal distributions

If x = (x_1,x_2,...,x_n) is a vector whose components have a distribution that is a finite mixture of multivariate normals, is the expected value of x_1 still a linear function of the other components,...
1
vote
2answers
42 views

Meaning of Gaussian mixture model parameters

I came across this question from a tutorial: Suppose we have observations $x_1$ , $x_2$ , $\ldots$, $x_n$ of a continuous r.v. $X$ known to be drawn from a “mixture” of $k$ Gaussian distributions. ...
1
vote
1answer
375 views

Estimating truncation point in Gaussian mixture

I have data modeled as a mixture of two Gaussian distributions. The data is "clipped" i.e., there is data only for values greater than a threshold $t$, even though it is feasible for data to exist in ...
0
votes
0answers
31 views

Reducing the number of Gaussians in a Gaussian Mixture Model

I build a kernel density estimation (KDE) of Gaussian kernels. I have many samples, but the distribution is not too complicated. I think it should be possible to approximate the resulting KDE by a ...
2
votes
0answers
32 views

Multimodality of mixtures of more than two Normal distributions

Let $$\phi(x;\mu,\sigma) = \frac{1}{\sigma \sqrt{2\pi}} \exp \left(- \frac{(x-\mu)^2}{2\sigma^2}\right)$$ denote the Gaussian density function ($\sigma > 0$). Let $$f(x) = \sum_{i=1}^N p_i \...
0
votes
0answers
23 views

Conditional distribution from a mixture of normals?

I need help with something I am getting confused on (this is not a homework problem). If you have a mixture of 2 normal distributions, how do I use Bayes rule with another normal to get a conditional ...
1
vote
0answers
45 views

Interpretation of plots for outlier detection in healthcare

Christy et al. propose cluster-based approach to outlier detection as part of the preprocessing step. However, I don't think the plots are very interpretable. The authors use the R mclustbic function ...
0
votes
0answers
32 views

Re-scaling mixing coefficients for GMM

Suppose I have fit a 3 component mixture of Gaussians to some real data. Based on the system that produced this data, the mixing weights of 2 of the components are 8x greater than they should be. How ...
0
votes
0answers
54 views

how to model hourly wind speed data

I am trying to forecast hourly wind speed (HWS) data in Trinidad and Tobago and I have read in the literature that "Direct application of stochastic models (ARMA & ARIMA models) to HWS series is ...
0
votes
1answer
56 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...