Questions tagged [gaussian-mixture]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

1
vote
0answers
55 views

can I fit a GMM over probability distributions?

We have been using a GMM to fit gaussians over a set of ~ 3M vectors. Now the input are not vectors but probability distributions (coming from a topic model like LDA). Is it still mathematically ...
1
vote
0answers
391 views

How to fit a gaussian mixture model in R with fixed parameters

I am using R to analyse experimental, two dimensional data via gaussian mixture modeling with the mclust package in order to find the mean of each component. I ...
-1
votes
1answer
101 views

Gaussian Mixture description

Looking at this link on Gaussian Mixtures and EM: http://www.ics.uci.edu/~smyth/courses/cs274/notes/EMnotes.pdf from the link: Given a data set D = {$x_1, x_N$} where $x_i$ is a d-dimesional vector. ...
1
vote
1answer
11k views

Generate sample data from Gaussian mixture model [duplicate]

I am given the values for mean, co-variance, initial_weights for a mixture of Gaussian Models. Now how can I generate samples given those: In brief, I need a function like ...
1
vote
1answer
1k views

Derivation of M-step in EM algorithm for mixture of Gaussians

I am trying to derive the parameter estimation equations for the M-step of the expectation maximization (EM) algorithm for a mixture Gaussians when all Gaussians share the same covariance matrix $\...
0
votes
0answers
259 views

Why is gradient used in Fisher Vectors?

My understanding of Fisher vectors can be described in the following manner: A GMM is trained on all data, which gives $p(X,\theta)$, then, for each image/video ($X_i$), the gradient of $p(X,\theta)$ ...
1
vote
0answers
204 views

Detecting distribution peaks and their significance

I have (a lot of) datasets with points having 1d distributions like these: Note, that the data is periodic in nature, like time of a day, so left and right sides of the plots above correspond to the ...
4
votes
2answers
4k views

Gaussian Mixture Models: Maximum Likelihood Estimation or Expectation Maximization?

As far as I know the usual method for estimating the parameters in GMM is EM. However, it is also possible to use maximum likelihood. What are the differences between these two methods? Why would one ...
2
votes
0answers
53 views

Variance of mix of normals

Suppose we have $n$ random variables distributed normally with the same mean and different variances. Suppose we know these variances. Which will be the variance of the marginal distribution induced ...
1
vote
1answer
38 views

what is the difference in training and testing for Gaussian and Mixture of Gaussians

what is the difference in training and testing between the Gaussian and Mixture of Gaussians? Are they the same except one is unimodal and one is multimodal?
-1
votes
3answers
688 views

Area under peaks considering mixture of normal distributions [closed]

I have a question regarding fitting mixture of normal distribution. I have the following data: I identified three peaks and using mixtools package, I fitted ...
3
votes
1answer
764 views

Median of mixture of two Gaussian distributions with equal weights

I am given a population $P$ that is equally divided into subsets $A$ and $B$. I know that a property $H$ of the population $P$ is normally distributed with mean $\mu_1$ and variance $\sigma_1^2$ for ...
0
votes
0answers
134 views

Mixture Proposal Distributions

I have a target distribution $\mu$ which I would like to investigate using, for instance Metropolis-Hastings-Green (MHG). So, given a Gaussian prior, $\pi$, and a likelihood $L$ such that $\mu(dx) \...
2
votes
0answers
394 views

Can a Gaussian mixture model be specified using a regression equation?

From: https://stats.stackexchange.com/a/236297/22199, I quote A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A ...
1
vote
1answer
69 views

How to use Bayes' Theorem to detect an event in a noisy signal

I'm trying to use Bayes' Theorem to solve a question that's come up in work, but I don't know if I've done it correctly, because the result seems a bit strange. The problem involves a stochastic ...
10
votes
1answer
6k views

Mclust model selection

The R package mclust uses BIC as a criteria for cluster model selection. From my understanding, a model with the lowest BIC should be selected over other models (if ...
0
votes
2answers
80 views

doubt regarding the gaussian mixture model definition

With reference to the following definition of GMM (see snapshot from Reynolds (1)), I have two doubts: In the definition of probability density, the covariance matrix (denoted by sigma) is ...
3
votes
1answer
4k views

anomaly detection with gaussian mixture models

I am new to the topic, and I am trying to understand how it is possible to perform anomaly detection by using gaussian mixture models. Could you please give me some hints about literature on the topic?...
8
votes
2answers
3k views

What is “mixture” in a gaussian mixture model

We often study Gaussian Mixture model as a useful model in machine learning and its applications. What is the physical significance of this "Mixture"? Is it used because a Gaussian Mixture Model ...
1
vote
0answers
114 views

Gaussian Mixture Modeling - Determining More Than One Component

Let us follow the convention that a lower information criteria score is considered better. Suppose we have a ground-truth Gaussian mixture model (GMM) with $k$ components. Suppose also that we (1) ...
4
votes
2answers
2k views

How to derive the MLE of a Gaussian mixture distribution

In my self-study, I consider a Gaussian mixture distribution: $$p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0)$$ where $p(k=1)+p(k=0)=\pi_1+\pi_0=1$. I am now asked to do three ...
2
votes
0answers
150 views

Which cluster analysis for ordinal temporal data?

I would like to perform a cluster analysis but I’m not sure which is the best algorithm to apply to my data. My dataset is made of 200 cases (but the sample size can be enlarged). For each case, I ...
3
votes
2answers
3k views

How to use Kullback-leibler divergence if mean and standard deviation of of two Gaussian Distribution is provided?

With Apache Spark MLLib library I am trying to find Clusters within a dataset by using Gaussian Mixture Model (number cluster =3) . Now it returns 3 different values of mean and standard deviation. I ...
2
votes
0answers
799 views

Entropy of a set of categorical variables

In the context of Expectation-Maximization, I would like to compute te entropy factor in order to get the value of the lower bound when the algorithm converged. This lower bound can be expressed as: ...
1
vote
1answer
412 views

Decomposition of multimodal distributions

I have decomposed a multimodal distribution into the constituent single distributions for for further analysis. I have spent some time researching various approaches and I have not found one that that ...
1
vote
0answers
990 views

Bayesian Networks - CPD representation and inference for non-Gaussian continuous variables

I'm trying to implement an approximate inference algorithm based on junction tree algorithm for a Bayesian Network that has continuous variables which happen to have non-linear relationships, and in ...
3
votes
0answers
202 views

Convergence of k-means or EM on Mixture of Gaussians

There are many algorithms for learning mixture of Gaussians but typically k-means/EM is used in practice. My question is related to the performance of k-means/EM for MoG. Recently, I came across this ...
1
vote
1answer
3k views

The number of parameters in Gaussian mixture model

I have D-dimensional data with K components. How many parameters if I use a model with full covariance matrices? and How many if I use diaogonal covariance matrices?
1
vote
1answer
147 views

Mixture Model Distributions

I wonder, if there could be a Pareto Mixture Model, just like the Gaussian Mixture Model (GMM). How am I supposed to build a Pareto Mixture Model (PMM)?
1
vote
1answer
471 views

incremental gaussian mixture model [closed]

I have trained GMM on small train data set, I would like to update the GMM parameters on the fly when new samples arrive. Please direct on how to do that? Please inform if some existing implementation ...
0
votes
0answers
63 views

Given a pdf which is a mixture of Gaussians, how do I infer the position (mean), variance, and number of Gaussians?

I have the following data, which when plotted as a histogram, are a mixture of Gaussians: I would like to write an algorithm that would infer: (1) the number of "peaks" or normal distributions in ...
0
votes
1answer
221 views

upper limit on number of clusters in GMM

I am using Gaussian Mixture Models (GMM) to fit a small data set with ~60 observations and 4 dimensions. This data was generated from the raw data with 14 dimensions after retaining principal ...
0
votes
2answers
436 views

When using a Gaussian Mixture Model GMM, is it possible at all to infer the number of clusters to use?

When using a gaussian mixture model, you usually need to specify the number the number of clusters in the data. However, are there methods whereby you could infer the number of clusters to use, given ...
3
votes
0answers
815 views

Fit Gaussian Mixture model directly to the mixture density

The core of the question is: Can I estimate the parameters of a gaussian mixture model (with EM or Dirichlet Process) from a mixture density directly, that is, without using data drawn from such ...
1
vote
1answer
189 views

QDA vs EM with Gaussian likelihoods

QDA (quadratic discriminant analysis) assumes that the K different classes are generated by K different multivariate Gaussians, each with potentially different mean vector and covariance matrix. If ...
3
votes
0answers
454 views

How to calculate BIC for multidimensional problem

Sorry for this question, but I am really not sure how to calculate BIC for my situation. My models are mixtures of normals with different number of components. Variances are equal for all components ...
-1
votes
1answer
32 views

Mix of n normals with known locations

I have data points that are generated with the $n$ normal distributions with the same $\sigma$ and different means. I do not know $n$, but I know that $1 \leq n \leq 4$. I know the possible set of ...
2
votes
1answer
556 views

Defining overlapping periods

I have a dataset containing the abundance of migrating bird species. In the figure below you can see that there are two "bell" shapes that are overlapping somewhere around September. One of the bell ...
2
votes
0answers
113 views

Expectation Maximization Gaussian Mixture Example

I am a biologist trying to understand expectation maximization for a mixture of two Gaussian distributions. I think I understand how to deal with the means of the two distributions, but I don't know ...
1
vote
0answers
53 views

How to evaluate multiple density estimations in one space?

Let's say we have several users, each represented by a set of document vectors in $\mathbb{R}^n$. We fit the generative distributions using one Gaussian mixture model for each user. The goal is to use ...
8
votes
2answers
415 views

How to prove this Gaussian Mixture inequality? (Fitting/Overfitting)

Let f[x] be a Gaussian Mixture pdf with n terms of uniform weight, means $\{\mu_{1},...,\mu_{n}\}$, and corresponding variances $\{\sigma_{1},...,\sigma_{n}\} $: $$f(x)\equiv\frac{1}{n}\sum_{i=1}^{n}\...
12
votes
5answers
5k views

Singularity issues in Gaussian mixture model

In chapter 9 of the book Pattern recognition and machine learning, there is this part about Gaussian mixture model: To be honest I don't really understand why this would create a singularity. Can ...
1
vote
0answers
123 views

Posterior pointwise uncertainty of multivariate normal-Wishart (variational GMM)

Given a variational mixture of Gaussians (as per, e.g., Chapter 10 of Bishop, 2006), we can compute the posterior predictive pdf: $$ \left\langle p(x|\alpha,\beta,\nu,\mu,V) \right\rangle $$ where $\...
2
votes
2answers
260 views

AICc is picking overly complex models - something stricter?

I'd like to know if there are stricter alternatives to automated model selection than AICc / AIC / BIC. We have approximately ten thousand curves, and for each we'd like to find the most parsimonious ...
1
vote
1answer
892 views

How to know if my Gaussian mixture model has enough training data?

A somewhat soft question - I'm training a Gaussian mixture model (with the EM algorithm) on data of size $N$ ($N$ is typically between 4 and 64). How much samples do I need? obviously it depends on ...
1
vote
1answer
354 views

what is marginal density based on this picture?

I am studying Gaussian mixtures from the book Pattern Recognition and Machine Learning by Chris Bishop. Figure (a) are contours of the mixture components (Gaussians) with the corresponding weights. ...
1
vote
1answer
136 views

How to project data onto a model (specifically, GMM)?

I'm using data to train a Gaussian mixture model (GMM). I then take a sample and would like to see its projection on the GMM 'space'. I can think of an optimization problem such as this: consider $y$ ...
3
votes
2answers
3k views

Significant difference between time series - Can I do this?

I'd like to know whether the solution proposed below is valid/acceptable and any justification available. We have two biological conditions, and for each condition we measured 3 time series, so at ...
0
votes
0answers
630 views

Predicting intensity of Poisson process, given event data

I have a dataset of events: each row is an event, and each column is a feature. There are millions of events and several dozen features. The features are mostly numerical (a few are categorical and I ...
1
vote
1answer
435 views

Understanding the E step of EM for GMM

I'm reading this chapter about EM (9.3.1) of the book "Pattern Recognition and Machine Learning". I understand the basic EM algorithm for GMM, but I'm having some problems understanding the ...