Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

1
vote
0answers
15 views

how can I integrate probabilities from two GMMs?

Without going into the details of why exactly I must do this, I have four GMMs (two sets for two classes), and I need to integrate their predictions. Two of them are trained on two classes from ...
1
vote
0answers
2k views

Computing covariance matrix and mean in python for a Gaussian Mixture Model

I am studying Bishop's PRML book and trying to implement Gaussian Mixture Model from scratch in python. So I have prepared a synthetic dataset which is divided into 2 classes using the following ...
1
vote
0answers
154 views

Improve gaussian mixture model performance

I have a data set of 10-dimensional cell measurements for leukemia. The data points are unlabeled and the task is to find the ratio of pathological measurements w.r.t. the rest of the sample. In other ...
1
vote
0answers
155 views

Gassuian mixture model visualization

Let $x\in\mathbb{R}^n$ be a GMM-distributed vector with $K$ components. In the setting where we consider diagonal covariances, we have $P1=K\cdot (n+n)+K$, i.e. linear in $n$ and $K$, and $P_2=K(n^2/2+...
1
vote
0answers
78 views

Which parameters need to be initialized random for gaussian mixture hidden markov model?

So, if I model observation probability for a given hidden state according to a multivariate gaussian mixture model, then which parameters need to be initialized random to perform parameter re-...
1
vote
0answers
26 views

Mixture models - am I on the right path?

We have 2 chemical parameters measured for material 1 and 2. We are interested in finding out the probability of an unknown sample being the mixture of the two reference materials. Also determination ...
1
vote
0answers
612 views

Mixture of bivariate normal distribution in R

I would have to define, in R, a mixture of a number of bivariate normal distributions like that: a strategy would be to define the single pieces of the expressions, for example: ...
1
vote
0answers
55 views

can I fit a GMM over probability distributions?

We have been using a GMM to fit gaussians over a set of ~ 3M vectors. Now the input are not vectors but probability distributions (coming from a topic model like LDA). Is it still mathematically ...
1
vote
0answers
391 views

How to fit a gaussian mixture model in R with fixed parameters

I am using R to analyse experimental, two dimensional data via gaussian mixture modeling with the mclust package in order to find the mean of each component. I ...
1
vote
0answers
204 views

Detecting distribution peaks and their significance

I have (a lot of) datasets with points having 1d distributions like these: Note, that the data is periodic in nature, like time of a day, so left and right sides of the plots above correspond to the ...
1
vote
0answers
115 views

Gaussian Mixture Modeling - Determining More Than One Component

Let us follow the convention that a lower information criteria score is considered better. Suppose we have a ground-truth Gaussian mixture model (GMM) with $k$ components. Suppose also that we (1) ...
1
vote
0answers
991 views

Bayesian Networks - CPD representation and inference for non-Gaussian continuous variables

I'm trying to implement an approximate inference algorithm based on junction tree algorithm for a Bayesian Network that has continuous variables which happen to have non-linear relationships, and in ...
1
vote
0answers
53 views

How to evaluate multiple density estimations in one space?

Let's say we have several users, each represented by a set of document vectors in $\mathbb{R}^n$. We fit the generative distributions using one Gaussian mixture model for each user. The goal is to use ...
1
vote
0answers
124 views

Posterior pointwise uncertainty of multivariate normal-Wishart (variational GMM)

Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\...
1
vote
0answers
29 views

Does a check failing to compare observed and predicted data qualify as a posterior predictive check?

I consider a Gaussian mixture distribution and I want to implement posterior predictive checks for choosing the model with the correct number of mixture components. I know the true number of ...
1
vote
0answers
900 views

Why is it that a larger 'k' value fails to converge but a smaller 'k' converges?

I'm doing clustering via GMM, which is initialized first by k-means. I am using a data matrix that cannot be classified as small by any standards, they are usually of the size ...
1
vote
0answers
95 views

How to generate n theano.shared variables for Gaussian mixture regression?

I am trying to program a Gaussian mixture regression using python, and the theano package. Suppose I have n clusters, then I need to generate n regression weight matrices for each cluster (also, n ...
1
vote
0answers
144 views

How to generate samples of Gaussian scale mixture?

If I want to generate multivariate Gaussian scale mixture samples with given location and covariance (characteristic matrix) of sample how will I generate that with any mixture distribution? For ...
1
vote
0answers
29 views

mixture of 2 Gaussians and using a priori information about one of the Gaussians

I am working on a large dataset of 2 populations, one is healthy controls and other is considered to be dysfunctional My variables interests suggest a good fit for a unitary Gaussian distribution for ...
1
vote
0answers
149 views

Problem on Clustering Discrete Input using GMM

I want to do clustering using Gaussian Mixture modeling (GMM) on a set of data which is a 5-dimension vector of real values $(x_1,x_2,x_3,x_4,x_5)$. However the clustering result were pretty bad, ...
1
vote
0answers
137 views

Composing a likelihood function for a convolved normal- and lognormal- dataset

I am trying to use some implementation of MLE to solve for the parameters of a mixed normal and lognormal population, and have had no end of trouble. A colleague and the internet has me convinced ...
1
vote
0answers
52 views

Criterion to determine the number of kernels in Expectation-Maximization algorithm

Suppose we have a random variable $X$, whose distribution is a Gaussian mixture $$f_K =\sum_{i = 1}^K \alpha_i N(\mu_i, \sigma^2_i)$$ where $\alpha_i> 0$, $\sum_i \alpha_i = 1$ and $N(\mu,\sigma^2)$...
1
vote
0answers
43 views

How to find the probability distribution that one variable equals another in a Gaussian Mixture Model

I have a multi-dimensional Gaussian, and I want to find the distribution for the case that some variables are a linear function of the others. For example, in the case of a two-dimensional Gaussian, ...
1
vote
0answers
408 views

Probabilistic score vs $L^2$ norm to evaluate Gaussian Mixture Models

There seems to be (at least) two ways that one could evaluate the fit of a Gaussian Mixture Model (GMM) to a data set. First, a probabilistic score, is the log likelihood of a set of points $D$ ...
1
vote
0answers
51 views

Source separation of a mixture of mixtures

I am quite new to ML / DSP so I apologise if the answer is obvious to others. I am comfortable with complex software implementation, and have a degree focussed on statistics, but little experience ...
1
vote
0answers
139 views

How to calculate the posterior probabilty of Gaussian Mixture Component

If the mean vector and the Covariance matrix of a Gaussian Mixture model are known, how could I calculate the posterior probability of each of the Gaussian Component in the mixture.
1
vote
0answers
134 views

clustering vs fitting with a distribution

I have a question about using a clustering method vs fitting the same data with a distribution. Assuming that I have a dataset with 2 features (feat_A and feat_B) and let's assume that I use a ...
1
vote
0answers
443 views

Fitting multiple normal distributions to sample data

I have a data set of (time, action)-tuples. Actions are typically performed at approximately the same times every day, and depending on what the action is it may be done multiple times per day. If I ...
1
vote
0answers
29 views

best theory on fitting mixture of gaussians

What are the current best results on fitting mixtures of Gaussians with any algorithm (EM or something fancier)? Specifically, if I know only the number of components, what are the sharpest sample ...
1
vote
0answers
145 views

Can i use a mixture model for when I have an omitted variable?

I plan to fit a GAM or GAMM. There is one categorical variable which I think is important for explaining Y (or Y*), but it is not in my dataset - it is measurable but has not been measured. Can I use ...
1
vote
0answers
93 views

What criteria are used to compare feature-based classification techniques?

When comparing feature-based classification techniques, what characteristics about the different processes should be considered? I'm comparing different classification techniques to try to figure ...
1
vote
0answers
333 views

Exact derivation for finding k-means from Gaussian Mixtures

I am having difficulty in deriving k-means from Mixture of Gaussians. I am following the notation from Bishop (2006), Section 9.3.2: Suppose we have : $$ p(\mathbf{x}| \boldsymbol{\mu}_k, \...
1
vote
0answers
118 views

Confusion about the EM algorithm

I am reading through the EM-Algorithm but on the slide 39, I don't get how $$P(D) = P(D|A)P(A) + P(D|B)P(B)$$ I am trying to understand it in order to get my head around modelling with mixture of ...
1
vote
0answers
233 views

Comparing two samples in mixtools

I have 16 experiments performed under different conditions and I would like to compare them between each other. Each experiment has 5-6 samples (cells). Each cell produces hundreds of data points each ...
0
votes
0answers
5 views

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture Problem description I am trying to normalize multivariate time series data. The ...
0
votes
0answers
23 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
0
votes
0answers
23 views

Confidence regions after fitting a 2 parameter gaussian mixture model?

Suppose I have a gaussian mixture model with 2 parameters $(u,v)$ and 2 parts. The model is $P({x_i}|u,v)=uN(x_i|\mu_1^{i} = x_i^2/v,\sigma_1^{i}) + (1-u)N(x_i|\mu_2^{i} = x_i^3/2v^2,\sigma_2^i)$. ...
0
votes
0answers
10 views

Training Hidden Markov model with GMM, nan appears after some iterations

Problem During the training process of my continuous observation sequence data using HMM with GMM mixtures, the cost function reduces gradually and it becomes NaN ...
0
votes
0answers
27 views

Maximum likelihood convergence in mixture gaussian

Suppose there are two datasets $D_{1}$ and $D_{2}$ with same structure, which means the cluster and cluster proportion is the same. The only difference between them is that the size of $D_{1}$ is $n_{...
0
votes
0answers
12 views

How is jaccard similarity used to find the similarity between Bootstrap samples when measuring stability of EM?

Im reading the answer on "how to determine number of clusters in EM algorithm". How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results? One of the ...
0
votes
0answers
6 views

Biased viterbi training result

I try to use GMM-HMM model to infer the topic of sentences in a short paragraph. While instead of using normal Baum-Welch optimization, I use viterbi training as follows. I use average of word ...
0
votes
0answers
16 views

Expectation of normal RV conditional on normal mixture

Let $v\sim\text{Normal}\left(\mu,\sigma_v^2\right)$ a random variable with $\mu>0$ and $u\sim\text{Normal}\left(0,\sigma_u^2\right)$ Let $k\sim\text{Binomial}\left(N,p\right)$ a random variable ...
0
votes
0answers
10 views

Interpretation of NLP pipeline for topic discovery using gaussian mixture model clustering

I built a pipeline that does the following to discover topics out of a (very big: 50k docs per ~350 terms) Term Document Matrix: Compute the TfIdf score for each Term x Document pair; Rescale each ...
0
votes
0answers
15 views

How to forecast a time-series with a dynamic time unit?

I'm working on a forecasting problem, and I'm not sure if the data requires any transformation because the unit of time is dynamic. I'm working with an education data set where I have data on ...
0
votes
0answers
14 views

How to 'normalize' the product of two variables from Gaussian distribution?

I have two variables, x1 and x2, which are sampled from two Gaussian distributions respectively. I created an iteraction term x3 which is x1 multiply x2. Not surprisingly, x3 has very fat tail, ie., ...
0
votes
0answers
13 views

Imposing constraints on observation model in a HMM

I have $N$ observations ($x_1, x_2,.. ,x_N$) from a HMM with $K$ latent states. The M step for computing the observation model $\mu_k$ involves maximizing the expression: $$ L = \sum_{n=1}^{N}{ln \...
0
votes
0answers
9 views

Can we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?

I learned from this answer that: A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A gaussian-mixture is the ...
0
votes
0answers
19 views

Redefining latent variables as observed data

This was just a thought that occurred to me, but technically, is it possible to redefine what I treat as latent variables and what I treat as data? For example, lets assume I have a set of latent ...
0
votes
0answers
47 views

Standard deviation in multimodal data

I have a dataset, 90% of observations are unimodal normal (with couple of outliers per feature), 10% are mixture of normals, components have the same standard deviations. Data contains outliers => ...
0
votes
0answers
14 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...