# Tag Info

Accepted

### Different covariance types for Gaussian Mixture Models

A Gaussian distribution is completely determined by its covariance matrix and its mean (a location in space). The covariance matrix of a Gaussian distribution determines the directions and lengths of ...
• 291k

### In cluster analysis, how does Gaussian mixture model differ from K Means when we know the clusters are spherical?

Ok, we need to start off by talking about models and estimators and algorithms. A model is a set of probability distributions, usually chosen because you think the data came from a distribution like ...
• 24.6k
Accepted

### Finding category with maximum likelihood method

This is a classic unsupervised learning problem that has a simple maximum likelihood solution. The solution is a motivating example for the expectation maximization algorithm. The process is: ...
• 53.3k
Accepted

### Growing number of Gaussians in a mixture

If your goal is to find the maximum-likelihood mixture of size $n+1$, then you can use the existing solution as an initialization, once you have enlarged it to have one more Gaussian. To enlarge it, ...
• 6,650
Accepted

### Singularity issues in Gaussian mixture model

If we want to fit a Gaussian to a single data point using maximum likelihood, we will get a very spiky Gaussian that "collapses" to that point. The variance is zero when there's only one point, which ...
• 14.1k
Accepted

### Why Expectation Maximization is important for mixture models?

In principle, both EM and standard optimization approaches can work for fitting mixture distributions. Like EM, convex optimization solvers will converge to a local optimum. But, a variety of ...
• 29.8k
Accepted

### Why do we use Gaussian distributions in Variational Autoencoder?

Normal distribution is not the only distribution used for latent variables in VAEs. There are also works using von Mises-Fisher distribution (Hypershperical VAEs [1]), and there are VAEs using ...
• 10.3k

### Simulate from a truncated mixture normal distribution

Simulation from a truncated normal is easily done if you have access to a proper normal quantile function. For instance, in R, simulating $$\mathcal{N}_a^b(\mu,\sigma^2)$$where $a$ and $b$ denote the ...
• 92.7k

• 2,464
Accepted

• 925
Accepted

### Finding the point of maximum probability in a mixture of gaussians

If you have a mixture of $n$ one-dimensional Gaussians, you can have anything between one and $n$ local maxima: ...
• 97.4k
Accepted

### Fit mixture of distributions to your time-series data in R

There is a misunderstanding in your question that needs a correction. Time-series model is not univariate since you have two variables: actual values and time. To provide an example let's take a time-...
• 115k
Accepted

### AICc is picking overly complex models - something stricter?

You could try cross-validation, fitting all five models for each fold and picking the model that has the lowest MSE (or whatever other error measure you are interested in) on the holdout folds. That ...
• 97.4k
Accepted

### A Gaussian Mixture Model Is a Universal Approximator of Densities

The idea is that an arbitrary density on $\mathbb{R}$, $f(\cdot)$, can be approximated by a Gaussian mixture model g_k(\cdot;\boldsymbol{\omega,\mu,\sigma})=\sum_{i=1}^k \omega_i \varphi(\cdot;\...
• 92.7k

### Why do we use Gaussian distributions in Variational Autoencoder?

We use normal distribution because it is easily reparameterized. Also a sufficiently powerful decoder can map the normal distribution to any other distribution, so from a theoretical viewpoint, the ...
• 23.2k

### Why is optimizing a mixture of Gaussian directly computationally hard?

In addition to juampa's points, let me signal those difficulties: The function $l(\theta|S_n)$ is unbounded, so the true maximum is $+\infty$ and corresponds to $\hat\mu^{(i)}=x_1$ (for instance) and ...
• 92.7k

### If k-means clustering is a form of Gaussian mixture modeling, can it be used when the data are not normal?

GMM uses overlapping hills that stretch to infinity (but practically only count for 3 sigma). Each point gets all the hills' probability scores. Also, the hills are "egg-shaped" [okay, they're ...
• 231
Miguel Carrera-Perpinan has a webpage on this topic with associated software. This does not directly solve your question, but indicates that unidimensional Gaussian mixtures with $k$ components have ...