Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
106 views

Conjugate prior bayesian inference on multivariate GMM

I am trying to understand how the posterior looks like when running Bayesian inference on a multivariate Gaussian-mixture model. $p(\mathbf{x}) \propto \sum_{i=1}^M w_iN(\mathbf{x}|\mu_i,\Sigma_i)$. ...
user avatar
  • 1
0 votes
0 answers
19 views

Is there a formula for the optimal weights for a Gaussian mixture model when the components are considered "correct"?

For simplicity consider two models predicting $Y=Z_0 + \sigma Z_1$, where $Z_i$ are independent $\mathcal{N}(0, 1)$ and each model only take one of the $Z_i$ as a factor. So we have $Y|Z_0$ and $Y|Z_1$...
user avatar
  • 240
1 vote
1 answer
206 views

Assumption of Gaussian mixture model [closed]

I am still confused when I read about the Gaussian Mixture Model and how does it work. One thing that I do not understand is that "GMM assumes the data is a mixture of Gaussian or i.i.d (...
user avatar
  • 173
0 votes
0 answers
13 views

Are the subsets of Mixture Models necessarily parametrized?

Im just learning about mixture models and they are described as a "mix of parametric and non parametric models". My question is how a non-parametric subset would look like? How would e.g. EM-...
user avatar
  • 1
1 vote
1 answer
125 views

Conditional distribution of the weight of a mixture gaussian with data augmentation using gibbs sampling

This question is relate to Differenciate between two distributions using gibbs sampling . For $t=1,\,\dots,\,n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl} \...
user avatar
0 votes
1 answer
54 views

Differenciate between two distributions using gibbs sampling [closed]

This question is relate to the post : " Conditional distribution for Gibbs sampling for Gaussian mixture " but is a little bit different. My objective is to know why the algorithm (which is ...
user avatar
0 votes
0 answers
43 views

Interpretation of test set negative log likelihood in neural density estimation applications

I have seen people splitting a dataset into a train and test sets and learning the parameters of a mixture density network using the negative log likelihood cost function on the train set and ...
user avatar
1 vote
1 answer
424 views

Kernel design for Gaussian processes with multiple inputs

How can I design a kernel function when there are multiple input variables and their degree of influence on the covariance with the target variable is different from each other? For example, if the ...
user avatar
  • 155
0 votes
0 answers
53 views

Mixture Density Network on Unbalanced Dataset

I want to use a mixture density network to make predictions with NLL loss (which is working) but my dataset is unbalanced so my network tends to make estimations that look like the distribution of ...
user avatar
  • 11
4 votes
1 answer
152 views

Gaussian distribution with poisson variance

Let's $r_t\sim \mathcal{N}(0,V_t)$ where $V_t = (1+m)^{J_t}$, $m\geq0$ and $J_t\sim \mathcal{P}oi(\lambda)$. How can we compute the distribution of $r_t$ when the parameters are $m$ and $\lambda$. In ...
user avatar
0 votes
0 answers
225 views

What is truncated gaussian mixture model?

I am interested in the Gaussian mixture model. I read about it and I think I am good with it. However, found that there is something called truncated Gaussian mixture model, which I do not understand. ...
user avatar
  • 1,160
1 vote
2 answers
315 views

Decision boundary between two Gaussians of unequal variance

This question is concerning a similar problem as mentioned in this question. The only difference is that in my case the variances are unequal. To recap, consider a two class scenario. At the decision ...
user avatar
1 vote
1 answer
32 views

Can we "reject" that a distribution is a finite mixture of normals?

Consider a one-dimensional distribution function $f(x)$. Suppose this distribution has all the nice properties, such as continuity, smoothness, etc. We observe $f(x)$. Suppose that we believe that $f(...
user avatar
  • 403
2 votes
0 answers
99 views

What is the "lower bound average gain" metric used in GMM stopping criterion used in Scikit learn?

In Scikit Learn's GMM class, it says that GMM training algorithm stops according to the "lower bound average gain" https://scikit-learn.org/stable/modules/generated/sklearn.mixture....
user avatar
  • 1,141
0 votes
1 answer
540 views

Covariance in multivariate Gaussian

In a single dimension Gaussian, the variance $\sigma$ denotes the expected value of the squared deviation from the mean $\mu$. I am trying to understand why in the multivariate case of modeling ...
user avatar
0 votes
1 answer
326 views

Gaussian Mixture Model fails on a simple distribution with a fixed number of components

I'm trying to fit a 2-component 2D Gaussian Mixture Model to some data. I know that there are only two components. The distribution can be seen below in the left plot: The brain can effortlessly pick ...
user avatar
  • 3,174
2 votes
1 answer
66 views

Is this equivalent to a Gaussian mixture model?

Similar to how you can derive the normal distribution as the distribution where the probability near a point exponentially decays with the square of the number of standard deviations you are away from ...
user avatar
1 vote
0 answers
31 views

What is the pdf of the square of the product of two correlated normal distributions?

Let $x$ and $y$ denote a bivariate normal random vector with zero means, unit variances and correlation coefficient $\rho$. Then, the pdf of $z=xy$ is known to be \begin{equation} f(z) = \frac{1}{\pi\...
user avatar
  • 414
1 vote
1 answer
176 views

Applying outlier adjustment using student's t distribution in a state-space model

I'm exploring performing outlier adjustment in a state-space model by using student's $t$ distribution. The gist of the problem is formulated as follows: $$ \begin{align*} y_t^* &= u_t + o_t - o_{...
user avatar
4 votes
2 answers
85 views

Why does it appear impossible to fit Gaussians to arbitrary probability density functions $p$?

I want to fit a Gaussian $q$ to a pdf $p$ by minimizing the energy $E = -\int q(x) \log p(x) dx$. This should result in a "delta function" Gaussian with $\sigma \rightarrow 0$ and $\mu \...
user avatar
  • 147
3 votes
2 answers
117 views

Sum of random variables that follow a finite normal mixture distribution

Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as $f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
user avatar
  • 1,193
6 votes
1 answer
220 views

A proof that the median is a nonlinear statistical functional

This question is with reference to the top answer (by @StephanKolassa) to this question. Let $F$ and $G$ be CDFs and define $$H(x)=aF(x)+(1-a)G(x)$$ with $a\in [0, 1]$. Now suppose $F$ and $G$ are ...
user avatar
  • 603
2 votes
0 answers
31 views

Mixture regression

I am wondering how to analyze and interpret something like the data shown in the figure below (the color is the log-density of the data points, which are a few hundred thousands in number). My ...
user avatar
  • 1,660
3 votes
1 answer
43 views

Difference of independent random variables that is not unimodal

This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
user avatar
  • 526
4 votes
0 answers
78 views

How to build a probability distribution from locations with accuracies?

I have a set of $n$ GPS locations $l_i$ with latitude, longitude in degrees and accuracy in meters, corresponding to $3 σ$, i.e. the probability ≈ 0.997) $(lat_i, lng_i, acc_i)$ or $(lat_i, lng_i, σ_i ...
user avatar
1 vote
0 answers
257 views

Clustering with gaussian mixtures: choice of hyperparameters

Question: I am interest in general in understanding how to choose the hyperparameters if we are interested in clustering bivariate vectors assuming a mixture of Gaussian mixture with conjugate Normal-...
user avatar
  • 402
1 vote
1 answer
1k views

How to evaluate the loss on a Gaussian Mixture Model?

I successfully modeled my data using a Gaussian Mixture Model in scikit-learn but I can't figure out how I should say "how good" the model is by calculating the loss. My first thought was to ...
user avatar
  • 1,300
2 votes
1 answer
403 views

Scikit learn - GMM log-likelihood. Why use Cholesky's precision matrix instead of covariance matrix?

This is my first post, please let me know if I am not being clear. I am trying to understand the sklearn.mixture.GaussianMixture.score(X). As I understand that the ...
user avatar
  • 23
2 votes
1 answer
931 views

Understanding the log-likelihood calculation of sklearn Gaussian mixture model

I am trying to understand how the Scipy is calculating the score of a sample in the Gaussian Mixture model(log-likelihood). Below is the equation I got for log-likelihood from the book C.M. Bishop, ...
user avatar
1 vote
0 answers
20 views

Finding maximum likelihood solution for mean when data is given which share the same mean but have different variance

I have some 'X sample points say (x1,x2,x3 ...) each of the samples form a Gaussian distribution with mean 'm' and variance v1,v2, ... All the distributions have the same mean but differ in variance. ...
user avatar
2 votes
1 answer
60 views

Upgrading weight parameters to random variable in Gaussian mixtures

In a Gaussian mixture model we model a density like: $p(\mathbf{x}|\pi,\mu,\sigma)=\sum \pi_i N(\mathbf{x}|\mu_i,\sigma_i)$ [1] where $\pi,\mu$ and $\sigma$ are parameters. I would like to know if the ...
user avatar
  • 693
0 votes
1 answer
126 views

About the derivation of EM for mixture of Gaussians

I'm reading Andrew Ng's note about Mixtures of Gaussians and the EM algorithm He writes the likelihood of data as where random variables $z^{(i)}$'s indicate which of the $k$ Gaussians each $x^{(i)}$ ...
user avatar
  • 973
0 votes
1 answer
267 views

Fitting Gaussian mixture model with constraints (eg. mu1<mu2) in Python

My question is similar to this one, but while the OP there has constrains such as mu1 being <=0 and mu2 being >=0, my constraints are following: It's a three component mixture model. mu1 < ...
user avatar
0 votes
1 answer
35 views

Mixture model when K=1

Assume that I want to estimate the parameters of a distribution like for example the Gaussian distribution, but I have the code only for the estimation of the parameters of a mixture of Gaussian ...
user avatar
  • 57
1 vote
0 answers
32 views

Likelihood parameters estimation in mixed data type

How to estimate parameters of a Gaussian mixture model with a mix of categorical and continuous data using log-likelihood? Indeed, I have a set of data consisting of categorical and continuous data. I ...
user avatar
  • 121
0 votes
0 answers
44 views

Closed form posterior for a mixtures of two univariate Gaussians

Giving a univariate Gaussian mixture model $$\pi_1N(x|\mu_1,\sigma_1)+(1-\pi_1)N(x|\mu_2,\sigma_2),$$ are there any priors for $\pi_1$, $\mu_1$, $\sigma_1$, $\mu_2$, $\sigma_2$ which gives a closed ...
user avatar
1 vote
0 answers
40 views

Do Gaussian Mixture Models monotonically decrease the sum of squared distances when number of clusters increases?

I am comparing the clustering performance of two closely related machine learning methods: K-means and Gaussian Mixture Models (GMM). Part of this research is selecting the best number of clusters K. ...
user avatar
2 votes
1 answer
307 views

Gaussian Mixture Model based clustering for unimodal, time series data

Problem: I have a simulated data set which is comprised of multiple sub-populations (or samples), each sub-population is drawn from, and described by, its own Gaussian distribution (although by chance,...
user avatar
  • 278
0 votes
0 answers
97 views

Cutoff for a poisson-gaussian mixture model

I have count data that is bounded on one side at zero (see image). It is bimodal and I think it results from two different processes. I would like to fit a poisson distribution around the hump around ...
user avatar
  • 1
2 votes
2 answers
150 views

Estimating weights of known component distributions in a mixture distribution

Given $n$ probability density functions ($p_1$, ..., $p_n$) with known distributions, what are the ways of estimating the weights ($w_1$, ..., $w_n$) of these component distributions given a sample ...
user avatar
1 vote
0 answers
354 views

what are the main differences between parametric and non-parametric machine learning algorithms?

I am interested in parametric and non-parametric machine learning algorithms, their advantages and disadvantages and also their main differences regarding computational complexities. In particular I ...
user avatar
0 votes
1 answer
28 views

Vatiational inference in GMM

I am learning about VI and am implementing a GMM model for clustering using variational inference. However, my implementation is not fitting the data at all, even when initializing the cluster means ...
user avatar
  • 273
0 votes
0 answers
2k views

Fitting mixture model of Gaussians and uniform distributions to real data

I have times series of wind direction and velocity. For now, I leave aside the velocity and focus on the distribution of wind directions. Over there, there is usually three main wind directions, and ...
user avatar
  • 101
4 votes
1 answer
700 views

What is the marginal posterior distribution?

Based on this question: How to build a Bayesian regression model of a response that is a Gaussian mixture Consider the mixture of normal, $$y_j\sim (N(0,\sigma_1))^{\pi}(N(0, \sigma_2))^{1-\pi}, j=1,...
user avatar
2 votes
0 answers
48 views

Confusion about two Gaussian distributions

From here, it says that, linear combination of two Gaussian distribution, are always Gaussians. However, Let 𝑋 be standard normal and 𝜀=±1 with probability 1/2 each, independently of 𝑋. Let 𝑌=𝜀𝑋...
user avatar
2 votes
0 answers
210 views

How does maximising ELBO for a Gaussian mixture model fit the model to data?

I am following along in Bishop's Pattern Recognition and ML chapters 9 and 10, and I understand that the EM algorithm works by iteratively updating model parameters using equations derived from ...
user avatar
3 votes
1 answer
231 views

Bayesian mixture model joint posterior

I am just starting to learn about bayesian mixture models. There is a few clarifications that I want to make which I am not sure myself. The graphical model below describes a gaussian mixture model ...
user avatar
  • 966
1 vote
0 answers
64 views

What distribution best describes multiple, sequential normal distributions: What is the sum of more than two normal distributions?

I am curious as to what describes the following distribution. If we were to record some data which are all from a normal distribution, but the standard-deviation changes for blocks of points recorded. ...
user avatar
  • 278
4 votes
1 answer
95 views

Given two normal populations,, classifying a given data point

I have two normal populations S1 and S2, where S1 ~ N (μ1, σ1) and S2 ~ N (μ2, σ2) respectively. The populations are independent of each other and a data point X has to be either from S1 or from S2. ...
user avatar
  • 65
1 vote
1 answer
398 views

Derive cauchy distribution as a scale mixture of normal distributions

I doing Bayesian modelling these days. I found that cauchy distribution can be written as a scale mixture of normal based on following source. Link So I started to derive this. Somehow, I am not ...
user avatar

1
2
3 4 5
12