The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

171 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
153 views

Entropy of a mixture of Gaussians

I need to estimate as fast and accurately as possible the differential entropy of a mixture of $K$ multivariate Gaussians: $$ \mathcal{H}[q] = -\sum_{k=1}^K w_k \int q_k(\textbf{x}) \log \left[\sum_{...
1
vote
0answers
32 views

Can the hidden states of a HMM be interpreted as number of clusters underlying the data?

Trying to understand the physical significance of the number of hidden states of a HMM. Should they be interpreted as number of clusters in the data? If not, why? Or they should be interpreted as the ...
1
vote
0answers
35 views

MIxture model in R to generate noise in data

I have a bit of code in R that adds noise to an harmonic series according to a normal distribution: ...
1
vote
0answers
73 views

Statistical test for comparing two-gaussian mixture

I have a distribution of shape sizes under two different (biological) conditions. From prior knowledge, I do expect there to be two populations. I fit each condition to a two-Gaussian mixture model. ...
1
vote
0answers
256 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
1
vote
1answer
548 views

Anomaly detection on 1D data with multiple gaussian distributions

My core problem is to set a cutoff to my one dimension data between normal with abnormal. I think this is a 'anomaly detection' problem. My Data My data is one dimension, consists with below: (...
1
vote
0answers
55 views

A transformation from uniform random variable to Gaussian mixture

I am attempting to describe a prior_transform for a multivariate Gaussian mixture in order to estimate the evidence integral of that prior convolved with another likelihood distribution. This is ...
1
vote
0answers
81 views

Interpretation of plots for outlier detection in healthcare

Christy et al. propose cluster-based approach to outlier detection as part of the preprocessing step. However, I don't think the plots are very interpretable. The authors use the R mclustbic function ...
1
vote
0answers
52 views

Mean and variance of a vector $W = (X', Y')'$

$(X_i,Y_i)$, $i = 1,2, ... ,n$, is a random sample from a bivariate normal distribution, with means $\mu_x$ and $\mu_y$, variances $\sigma_x$ and $\sigma_y$, and correlation $\rho$. How do I formally ...
1
vote
0answers
305 views

Using Gaussian Mixture Model for outlier detection

Can someone summarize how to use Gaussian Mixture Model for outlier detection purpose. I am more interested in a general method and not so much the mathematical aspects of GMM's
1
vote
1answer
76 views

Estimating a GCD

I have two related questions. Question 1: Let $k_1, \ldots k_n$ be positive integers, and $\alpha_1, \ldots \alpha_n \in (0,1)$ be such that $\sum_{j \leq n} \alpha_j = 1$. Suppose $\langle X_m \...
1
vote
2answers
165 views

PCA for probability vectors

Is there a procedure equivalent to principal component analysis (PCA) for probability vectors? I have an n-by-m array where every column sums to one, and all entries are positive. PCA works in ...
1
vote
0answers
79 views

How to interpret profiles / mixture components without any observations classified to them (when using MCLUST in R)

I am using the MCLUST software in R to fit normal mixture models, as part of what in my field are commonly called Latent Profile Analysis. Some of the time, ...
1
vote
1answer
46 views

Is there a way to accelerate expectation maximization?

There is a way to run a faster $k$-means by using Elkan's method, which uses the triangle inequality to avoid some calculations. I am trying to think of a way you could do a similar sort of thing for ...
1
vote
0answers
36 views

Approximating a 3D spline surface by a weighted sum of gaussians

I have spatial data(2D) with some quantity associated with each point - basically 3D data. I want to model the quantity distribution in the space and then use N clusters as a compact representation. ...
1
vote
0answers
16 views

how can I integrate probabilities from two GMMs?

Without going into the details of why exactly I must do this, I have four GMMs (two sets for two classes), and I need to integrate their predictions. Two of them are trained on two classes from ...
1
vote
0answers
2k views

Computing covariance matrix and mean in python for a Gaussian Mixture Model

I am studying Bishop's PRML book and trying to implement Gaussian Mixture Model from scratch in python. So I have prepared a synthetic dataset which is divided into 2 classes using the following ...
1
vote
0answers
162 views

Improve gaussian mixture model performance

I have a data set of 10-dimensional cell measurements for leukemia. The data points are unlabeled and the task is to find the ratio of pathological measurements w.r.t. the rest of the sample. In other ...
1
vote
0answers
161 views

Gassuian mixture model visualization

Let $x\in\mathbb{R}^n$ be a GMM-distributed vector with $K$ components. In the setting where we consider diagonal covariances, we have $P1=K\cdot (n+n)+K$, i.e. linear in $n$ and $K$, and $P_2=K(n^2/2+...
1
vote
0answers
86 views

Which parameters need to be initialized random for gaussian mixture hidden markov model?

So, if I model observation probability for a given hidden state according to a multivariate gaussian mixture model, then which parameters need to be initialized random to perform parameter re-...
1
vote
0answers
26 views

Mixture models - am I on the right path?

We have 2 chemical parameters measured for material 1 and 2. We are interested in finding out the probability of an unknown sample being the mixture of the two reference materials. Also determination ...
1
vote
0answers
657 views

Mixture of bivariate normal distribution in R

I would have to define, in R, a mixture of a number of bivariate normal distributions like that: a strategy would be to define the single pieces of the expressions, for example: ...
1
vote
0answers
58 views

can I fit a GMM over probability distributions?

We have been using a GMM to fit gaussians over a set of ~ 3M vectors. Now the input are not vectors but probability distributions (coming from a topic model like LDA). Is it still mathematically ...
1
vote
0answers
413 views

How to fit a gaussian mixture model in R with fixed parameters

I am using R to analyse experimental, two dimensional data via gaussian mixture modeling with the mclust package in order to find the mean of each component. I ...
1
vote
0answers
237 views

Detecting distribution peaks and their significance

I have (a lot of) datasets with points having 1d distributions like these: Note, that the data is periodic in nature, like time of a day, so left and right sides of the plots above correspond to the ...
1
vote
0answers
121 views

Gaussian Mixture Modeling - Determining More Than One Component

Let us follow the convention that a lower information criteria score is considered better. Suppose we have a ground-truth Gaussian mixture model (GMM) with $k$ components. Suppose also that we (1) ...
1
vote
0answers
1k views

Bayesian Networks - CPD representation and inference for non-Gaussian continuous variables

I'm trying to implement an approximate inference algorithm based on junction tree algorithm for a Bayesian Network that has continuous variables which happen to have non-linear relationships, and in ...
1
vote
0answers
54 views

How to evaluate multiple density estimations in one space?

Let's say we have several users, each represented by a set of document vectors in $\mathbb{R}^n$. We fit the generative distributions using one Gaussian mixture model for each user. The goal is to use ...
1
vote
0answers
127 views

Posterior pointwise uncertainty of multivariate normal-Wishart (variational GMM)

Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\...
1
vote
0answers
30 views

Does a check failing to compare observed and predicted data qualify as a posterior predictive check?

I consider a Gaussian mixture distribution and I want to implement posterior predictive checks for choosing the model with the correct number of mixture components. I know the true number of ...
1
vote
0answers
954 views

Why is it that a larger 'k' value fails to converge but a smaller 'k' converges?

I'm doing clustering via GMM, which is initialized first by k-means. I am using a data matrix that cannot be classified as small by any standards, they are usually of the size ...
1
vote
0answers
96 views

How to generate n theano.shared variables for Gaussian mixture regression?

I am trying to program a Gaussian mixture regression using python, and the theano package. Suppose I have n clusters, then I need to generate n regression weight matrices for each cluster (also, n ...
1
vote
0answers
148 views

How to generate samples of Gaussian scale mixture?

If I want to generate multivariate Gaussian scale mixture samples with given location and covariance (characteristic matrix) of sample how will I generate that with any mixture distribution? For ...
1
vote
0answers
31 views

mixture of 2 Gaussians and using a priori information about one of the Gaussians

I am working on a large dataset of 2 populations, one is healthy controls and other is considered to be dysfunctional My variables interests suggest a good fit for a unitary Gaussian distribution for ...
1
vote
0answers
157 views

Problem on Clustering Discrete Input using GMM

I want to do clustering using Gaussian Mixture modeling (GMM) on a set of data which is a 5-dimension vector of real values $(x_1,x_2,x_3,x_4,x_5)$. However the clustering result were pretty bad, ...
1
vote
0answers
141 views

Composing a likelihood function for a convolved normal- and lognormal- dataset

I am trying to use some implementation of MLE to solve for the parameters of a mixed normal and lognormal population, and have had no end of trouble. A colleague and the internet has me convinced ...
1
vote
0answers
52 views

Criterion to determine the number of kernels in Expectation-Maximization algorithm

Suppose we have a random variable $X$, whose distribution is a Gaussian mixture $$f_K =\sum_{i = 1}^K \alpha_i N(\mu_i, \sigma^2_i)$$ where $\alpha_i> 0$, $\sum_i \alpha_i = 1$ and $N(\mu,\sigma^2)$...
1
vote
0answers
44 views

How to find the probability distribution that one variable equals another in a Gaussian Mixture Model

I have a multi-dimensional Gaussian, and I want to find the distribution for the case that some variables are a linear function of the others. For example, in the case of a two-dimensional Gaussian, ...
1
vote
0answers
419 views

Probabilistic score vs $L^2$ norm to evaluate Gaussian Mixture Models

There seems to be (at least) two ways that one could evaluate the fit of a Gaussian Mixture Model (GMM) to a data set. First, a probabilistic score, is the log likelihood of a set of points $D$ ...
1
vote
0answers
52 views

Source separation of a mixture of mixtures

I am quite new to ML / DSP so I apologise if the answer is obvious to others. I am comfortable with complex software implementation, and have a degree focussed on statistics, but little experience ...
1
vote
0answers
146 views

How to calculate the posterior probabilty of Gaussian Mixture Component

If the mean vector and the Covariance matrix of a Gaussian Mixture model are known, how could I calculate the posterior probability of each of the Gaussian Component in the mixture.
1
vote
0answers
140 views

clustering vs fitting with a distribution

I have a question about using a clustering method vs fitting the same data with a distribution. Assuming that I have a dataset with 2 features (feat_A and feat_B) and let's assume that I use a ...
1
vote
0answers
452 views

Fitting multiple normal distributions to sample data

I have a data set of (time, action)-tuples. Actions are typically performed at approximately the same times every day, and depending on what the action is it may be done multiple times per day. If I ...
1
vote
0answers
29 views

best theory on fitting mixture of gaussians

What are the current best results on fitting mixtures of Gaussians with any algorithm (EM or something fancier)? Specifically, if I know only the number of components, what are the sharpest sample ...
1
vote
0answers
146 views

Can i use a mixture model for when I have an omitted variable?

I plan to fit a GAM or GAMM. There is one categorical variable which I think is important for explaining Y (or Y*), but it is not in my dataset - it is measurable but has not been measured. Can I use ...
1
vote
0answers
93 views

What criteria are used to compare feature-based classification techniques?

When comparing feature-based classification techniques, what characteristics about the different processes should be considered? I'm comparing different classification techniques to try to figure ...
1
vote
0answers
347 views

Exact derivation for finding k-means from Gaussian Mixtures

I am having difficulty in deriving k-means from Mixture of Gaussians. I am following the notation from Bishop (2006), Section 9.3.2: Suppose we have : $$ p(\mathbf{x}| \boldsymbol{\mu}_k, \...
1
vote
0answers
119 views

Confusion about the EM algorithm

I am reading through the EM-Algorithm but on the slide 39, I don't get how $$P(D) = P(D|A)P(A) + P(D|B)P(B)$$ I am trying to understand it in order to get my head around modelling with mixture of ...
1
vote
0answers
241 views

Comparing two samples in mixtools

I have 16 experiments performed under different conditions and I would like to compare them between each other. Each experiment has 5-6 samples (cells). Each cell produces hundreds of data points each ...
0
votes
0answers
15 views

how to use categorical variable with continuous variables in a EM mixture model

I'm trying to use the mclust and flexmix packages in R to do unsupervised clustering of my data which has both continuous variables and categorical variables. I'm having a hard time understanding how ...