The Stack Overflow podcast is back! Listen to an interview with our new CEO.

Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

171 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
0
votes
0answers
35 views

Determining Link for GLM

I am having a great deal of difficulty understanding how to use the Generalized linear model for my data set. The response variable of interest is hatch success of sea turtles, which is a %. The ...
0
votes
0answers
10 views

Probabilistic Models, what do they infer?

As per my understanding, Mixture Models such as GMM, Probabilistic Models such as Variation Autoencoder, they explain the latent space behind the features. But how they turn from learning latent space ...
0
votes
1answer
25 views

What is the difference between the latent variable and the cluster weights in mixture models?

$p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the difference between the the $w$ and the ...
0
votes
0answers
41 views

Gibbs Sampler for GMM

In Rasmussen's paper it is introduced a Gibbs sampler to make inference about a standard Gaussian Mixture Model. To simplify, assume the 1-d case with basic hierarchical structure, that is: $x_i|...
0
votes
0answers
28 views

what is wrong with my Gaussian Mixture density estimation fitting (Python)?

I have a data set (1D) link: dataset, which has values ranging from 21,000 to 8,000,000. When i plot histogram of the log values, i can see there are two peaks, roughly. I tried to fit Gaussian ...
0
votes
2answers
41 views

What is the interpretation of the weights in the GMM?

GMM is $p(x|\theta) = w_1 \mathcal{N}(x|\mu_1,\,\sigma_1^{2})\ + w_2 \mathcal{N}(x|\mu_2,\,\sigma_2^{2}) + w_3 \mathcal{N}(x|\mu_3,\,\sigma_3^{2})\,$ What is the interpretation of the weights here? Do ...
0
votes
0answers
12 views

How to deduct the complete likelihood of mixture normal in EM algorithm

We have the well known complete likelihood of mixture normal in EM algorithm: Here $Z$ is a random variable that it has probability $\pi_k$ to choose k-th normal variable $X_k:N(\mu_k,\sigma_k).$ We ...
0
votes
0answers
6 views

Decompose 2D matrices into mixture of Gaussians

I have a 2D array that roughly represents a probability distribution in the 2D space. That is, all values in this 2D matrix sum up to 1. I want to decompose this 2D matrix into a sum of Gaussians. ...
0
votes
0answers
8 views

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture Problem description I am trying to normalize multivariate time series data. The ...
0
votes
0answers
40 views

latent variables in EM algorithm are assumed to be i.i.d from multinomial distribution, from what they are idependent

In EM algorithm we introduce a latent variables, say $z_i$, $i=1,...n$, $n$ is the number of the mixture component. These variables ($z_i$) are assumed to be independent and identically distributed ...
0
votes
0answers
29 views

Confidence regions after fitting a 2 parameter gaussian mixture model?

Suppose I have a gaussian mixture model with 2 parameters $(u,v)$ and 2 parts. The model is $P({x_i}|u,v)=uN(x_i|\mu_1^{i} = x_i^2/v,\sigma_1^{i}) + (1-u)N(x_i|\mu_2^{i} = x_i^3/2v^2,\sigma_2^i)$. ...
0
votes
0answers
23 views

Training Hidden Markov model with GMM, nan appears after some iterations

Problem During the training process of my continuous observation sequence data using HMM with GMM mixtures, the cost function reduces gradually and it becomes NaN ...
0
votes
0answers
40 views

Maximum likelihood convergence in mixture gaussian

Suppose there are two datasets $D_{1}$ and $D_{2}$ with same structure, which means the cluster and cluster proportion is the same. The only difference between them is that the size of $D_{1}$ is $n_{...
0
votes
0answers
19 views

How is jaccard similarity used to find the similarity between Bootstrap samples when measuring stability of EM?

Im reading the answer on "how to determine number of clusters in EM algorithm". How to tell if data is "clustered" enough for clustering algorithms to produce meaningful results? One of the ...
0
votes
0answers
13 views

Biased viterbi training result

I try to use GMM-HMM model to infer the topic of sentences in a short paragraph. While instead of using normal Baum-Welch optimization, I use viterbi training as follows. I use average of word ...
0
votes
0answers
18 views

Expectation of normal RV conditional on normal mixture

Let $v\sim\text{Normal}\left(\mu,\sigma_v^2\right)$ a random variable with $\mu>0$ and $u\sim\text{Normal}\left(0,\sigma_u^2\right)$ Let $k\sim\text{Binomial}\left(N,p\right)$ a random variable ...
0
votes
0answers
21 views

Interpretation of NLP pipeline for topic discovery using gaussian mixture model clustering

I built a pipeline that does the following to discover topics out of a (very big: 50k docs per ~350 terms) Term Document Matrix: Compute the TfIdf score for each Term x Document pair; Rescale each ...
0
votes
0answers
18 views

How to forecast a time-series with a dynamic time unit?

I'm working on a forecasting problem, and I'm not sure if the data requires any transformation because the unit of time is dynamic. I'm working with an education data set where I have data on ...
0
votes
0answers
18 views

How to 'normalize' the product of two variables from Gaussian distribution?

I have two variables, x1 and x2, which are sampled from two Gaussian distributions respectively. I created an iteraction term x3 which is x1 multiply x2. Not surprisingly, x3 has very fat tail, ie., ...
0
votes
1answer
19 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
0
votes
0answers
15 views

Imposing constraints on observation model in a HMM

I have $N$ observations ($x_1, x_2,.. ,x_N$) from a HMM with $K$ latent states. The M step for computing the observation model $\mu_k$ involves maximizing the expression: $$ L = \sum_{n=1}^{N}{ln \...
0
votes
0answers
11 views

Can we apply Gaussian mixture model to all kinds of scensrios where some variables are unobserved?

I learned from this answer that: A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A gaussian-mixture is the ...
0
votes
0answers
19 views

Redefining latent variables as observed data

This was just a thought that occurred to me, but technically, is it possible to redefine what I treat as latent variables and what I treat as data? For example, lets assume I have a set of latent ...
0
votes
0answers
66 views

Standard deviation in multimodal data

I have a dataset, 90% of observations are unimodal normal (with couple of outliers per feature), 10% are mixture of normals, components have the same standard deviations. Data contains outliers => ...
0
votes
0answers
15 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
0
votes
0answers
27 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
0
votes
1answer
217 views

Inferring GMM parameters with Gibbs Sampling

On my book, "Machine Learning A Probabilistic Approach". It's stated that is straightforward to derive a Gibbs sampling algorithm to fit a mixture model, especially if we use conjugate priors. So ...
0
votes
0answers
33 views

GMM model of the joint distribution from multivariate marginals

I have two multivariate Gaussian variables $X \sim \mathcal{N}(\boldsymbol {\mu}_X \in \mathcal R^d,\boldsymbol {\Sigma}_X \in \mathcal R^{d \times d})$ and $Y \sim \mathcal{N}(\boldsymbol {\mu}_Y \...
0
votes
1answer
33 views

Concise way to visualize / compare many Gaussian mixtures

I have 5,000 samples drawn from each of approximately 50,000 distributions. I have good reason to expect most of them to be normally distributed, and I expect some of them to be multi-modal (mixture ...
0
votes
0answers
25 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
0
votes
0answers
7 views

Guassian Mixture Classification: interpretating component x variable means matrix

The Guassian Mixture model output by mclust::Mclust() function has a $parameters$mean element which is a matrix with dimensions ...
0
votes
0answers
130 views

Output Size of Mixture Density Networks

I am working on a neural network in which the final output layer will be a Mixture Density Network (MDN), but am confused about the shape of the values that final layer should return. In the paper in ...
0
votes
1answer
114 views

Decompose/split a single multivariate gauss into random gaussian mixture

Say, there is a single $n$-dimensional multivariate Gaussian. $$Gauss_a(\mu_a,\Sigma_a) $$ $\mu_a$ is $1\times n$ vector and $\Sigma_a$ is $n\times n$ matrix. Is there any easy way to decompose/...
0
votes
0answers
67 views

Reducing the number of Gaussians in a Gaussian Mixture Model

I build a kernel density estimation (KDE) of Gaussian kernels. I have many samples, but the distribution is not too complicated. I think it should be possible to approximate the resulting KDE by a ...
0
votes
0answers
84 views

how to model hourly wind speed data

I am trying to forecast hourly wind speed (HWS) data in Trinidad and Tobago and I have read in the literature that "Direct application of stochastic models (ARMA & ARIMA models) to HWS series is ...
0
votes
1answer
237 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
0
votes
1answer
209 views

JAGS mixture models with exogenous regressors

This is my first post,I hope this is the right forum for such a question and I formulate it correctly. I am working with a time series data set where the response y seems to follow a mixture of two ...
0
votes
1answer
88 views

Finding a Dominant Cluster

Super-basic question here: I'm looking for a way to find the dominant cluster of a set of clusters (as in the first image): This is not what I get when I run a Gaussian Mixture model with one ...
0
votes
1answer
100 views

Three component mixture model for element concentration using mixtools in R

As an update to a previously posed question, Fitting a mixture distribution for two approximately normally distributed populations using mixtools in R , I have now fit a three component mixture ...
0
votes
0answers
33 views

Any trick to swap order of determinant and matrix inverse operation?

Been thinking through fitting a kind of Gaussian mixture model in more of a neural network style (kind of similar to RNADE or RMADE by Larochelle, without going into details) and see that this could ...
0
votes
1answer
29 views

How to match sample points to a probabilistic model?

This might be a trivial question I have a probabilistic model, say a Gaussian mixture model of known parameters, and with that I have a set of defined sample points. I would like to know how likely ...
0
votes
1answer
103 views

Is it a known phenomenon for the variance of a component (GMM) to increase without stopping?

I know it can happen for it to decrease dramatically as it overfits on a single datapoint. But I've never read about a component "taking everything over". See the following images (circles are stddevs)...
0
votes
0answers
225 views

Mixture modelling with skewed distributions

I am trying to find a R library that splits a distribution into a symmetric and asymmetric components. I have a distribution that I want to split into two components, one skewed and the other most ...
0
votes
0answers
98 views

Mixture model with covariance matrix with varying variances and equal covariances

In some illustrations of mixture models, the covariance matrix is structured to have varying variances and equal covariances between mixture components. For example, Pastor et al. (2007) (I'm sorry ...
0
votes
1answer
21 views

How to choose closer gaussian?

Suppose, I have two random number generators with normal distribution, which generated a number. How to decide, to which one it is most probably belong? Should I calculate probability density of each ...
0
votes
0answers
118 views

Two variables in Multivariate Gaussian pdf

I have problem with computing multivariate gaussian probability density function(pdf) value. As I found the equation is, $$p(\textbf{x}|\mu, \Sigma) = \frac{1}{(2\pi )^{\frac{n}{2}} |\Sigma|^{\frac{1}...
0
votes
0answers
137 views

upper bound on number of components of GMM

I want to fit GMM to a data set with N data points in D dimensions. I am using the full GMM in MATLAB, i.e., each component has a complete covariance matrix (as opposed to diagonal covaraices which ...
0
votes
0answers
36 views

System of Gaussian equations

Let $N(x\ |\ \mu,\sigma^2)$ be the pdf of a normal random variable with mean $\mu$ and variance $\sigma^2$. Question: Given $n$ data points $(x_i,y_i)_{i=1,\dots,n}$, compute $\{w_i\}_i$,$\{\sigma_i^2\...
0
votes
0answers
268 views

Why is gradient used in Fisher Vectors?

My understanding of Fisher vectors can be described in the following manner: A GMM is trained on all data, which gives $p(X,\theta)$, then, for each image/video ($X_i$), the gradient of $p(X,\theta)$ ...