Questions tagged [gaussian-mixture-distribution]

A type of mixed distribution or model which assumes subpopulations follow Gaussian distributions.

Filter by
Sorted by
Tagged with
0 votes
0 answers
16 views

MLE for Two component mixture model

Chapter 8 section 8.5.1 of the Elements of Statistical Learning book describes a simple mixture model for density estimation and the associated EM algorithm for carrying out maximum likelihood ...
user avatar
0 votes
0 answers
3 views

How to obtain the p-values ​of a gamlssMX model?

I am working with a dataset that includes a binary target variable (0 or 1). I have built a model with the gamlssMX() function included on the "gamlss.mx" package to explain a continuous ...
user avatar
0 votes
0 answers
8 views

Are gaussian mixture models for clustering robust to data sparsity?

I would like to cluster customers based on their product usage data (20-40 products/dimensions) on the same scale. Overall, the data is reasonably log-normally distributed for all products (the ...
user avatar
0 votes
0 answers
26 views

How many components of a gaussian mixtures do I need to match moments up to the $r$-th order?

Suppose I have a ($k$-dimensional) random variable $X \sim D$ and I want to find a Gaussian Mixture $GM \sim \sum_{i=1}^C \pi_i \mathcal{N}(\mu_i, \Sigma_i)$ such that the moments of order $r'$, for $...
user avatar
  • 1
0 votes
0 answers
16 views

In a mixture model should I update the parameters of variance jointly or one-by-one?

Suppose that I have the following mixture model, where I know the true values of $(\pi_{1},\pi_{2},\pi_{3},\mu_{1},\mu_{2},\mu_{3})$ (I know them for the simulation that I build) $$f(x) = \pi_{1}N(x;\...
user avatar
  • 1,679
0 votes
0 answers
15 views

How to conduct EM algorithm when there are some outliers in GMM Models?

I'm just confused about the problem of adding an outlier component directly to the primary form of GMM models: Suppose that the observed data contains several outliers. The mixture model could be: $$ ...
user avatar
  • 1
0 votes
0 answers
7 views

True and Estimated Posterior of a Mixture of Gaussians in Bayesian Learning via SGLD

I am trying to recreate one of the experiments in this paper, (Bayesian Learning via Stochastic Gradient Langevin Dynamics). To be exact experiment 5.1. I am pretty sure, I am missing something here ...
user avatar
2 votes
0 answers
25 views

High dimensional behavior of Dirichlet Process-based clustering?

I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed. Let $D$ be the dimensionality ...
user avatar
2 votes
0 answers
21 views

Confusion Regarding Bayesian Mixture of Gaussians

Following is the screenshot from the paper "Variational Inference: A Review for Statisticians". I am having confusion understanding equations (7) and (8). Can anyone please let me know ...
user avatar
  • 61
0 votes
0 answers
12 views

Selecting the optimal bandwidth in kernel density estimation

I have a question regarding kernel density estimation. At the moment I have a set of sample date $V$, where for each $v \in V$ I have an associated standard deviation $\sigma_v$ (some measurements ...
user avatar
1 vote
1 answer
104 views

Fitting truncated normal mixtures in R

I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
user avatar
  • 189
0 votes
0 answers
10 views

Conditional distribution of GMM with given distribution?

I'm currently looking for the propagation of Gaussian mixture noise in a linear system. I have a question regards how to calculate the following equations: Suppose there is an GMM $f_1(x,y)$ = $\sum_{...
user avatar
3 votes
1 answer
92 views

Is the likelihood for Gaussian mixture models still multimodal when Y is partially observed?

In discussing Gaussian mixture models (GMMs), https://normaldeviate.wordpress.com/2012/08/04/mixture-models-the-twilight-zone-of-statistics/ highlights the issue of Multimodality of the Likelihood. ...
user avatar
  • 3,768
0 votes
0 answers
26 views

My data can be approximated with normal distribution (multimodal). How can I find the reasons and explain this behaviour?

I use DeLonge method to compare two ROC AUCS. The result of it is Z-score. Both ROC AUCs obtained from LDA (linear discriminant analysis) from sklearn package. The ...
user avatar
1 vote
1 answer
50 views

Why cannot MLE be implemented for Gaussian mixture model directly?

Consider the following density, the mixture of two Gaussian distributions, \begin{align*} p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0) , \end{align*} where $p(k=1)+p(k=0)=\pi_1+\...
user avatar
  • 95
0 votes
1 answer
75 views

Creating a probability density function from a Gaussian Mixture Model

I have some daily timeseries (27 right now but will be over 200 when I get more data) for electricity consumption. For each hour I want to know what the probability density function looks like. What I ...
user avatar
0 votes
0 answers
10 views

What is the number of free parameters in an n-component GMM?

I am trying to calculate BIC = -2logL + log(N)d where d is the number of free parameters or degrees of freedom. If I am fitting guassian mixture model to the data, ...
user avatar
  • 1
0 votes
2 answers
68 views

Is using a fixed random seed in production okay?

I have a dataset and am trying to use GMM to cluster it. The algorithm works well but when I run it multiple times I get different results. While the clusters produced in each run are valid my users ...
user avatar
  • 11
0 votes
0 answers
46 views

How can I fit a 1d, two-component Gaussian mixture with very uneven weights?

Say I have data generated by, with probability $p$, sampling from one normal distribution, and with probability $1-p$, from a different normal distribution. I would like to estimate the means of the ...
user avatar
  • 389
0 votes
0 answers
73 views

PCA as Pre-Processing before Clustering through GMM

Suppose I start with a data matrix $X \in \mathbb{R}^{N \times D}$, where each row $x_i$ is $D$-dimensional sample. I would like to cluster this data through a Gaussian Mixture Model (GMM). If I pre-...
user avatar
0 votes
0 answers
11 views

mixture of finite regressions without a response variable

In finite mixture modelling, in particular mixture of regressions modelling, we are interested in finding latent trajectories against a response variable. But what if there is no known response ...
user avatar
0 votes
0 answers
19 views

Generating a simulated dataset from a mixture of two-trait Gaussian distributions

How can I generate a random variable which follows the mixture Gaussian distributions: I found the answer and tried to simulate it, but I think it is not the right dataset I wanted. Here is my code: <...
user avatar
  • 23
0 votes
1 answer
56 views

Why Gibbs Sampling for mixture models?

I am studying MCMC and in the book I'm reading there is this example on Gibbs algorithm for inferring the posterior of a gaussian mixture. I understand how the algorithm works and the fact that its ...
user avatar
0 votes
1 answer
29 views

For univariate data, why do we need the normalmixEM function in R instead of just computing the mean and variance with the basic methods?

I can understand why if from your univariate data (1 column?) you plot a histogram which seems to have 2+ peaks ie a mix of more than 1 gaussian. But what if you plot a histogram and there looks to be ...
user avatar
1 vote
0 answers
48 views

Problems with convergence of the EM algorithm for a gaussian mixture regression

I have been implementing a EM-algorithm for a latent-class regression model, where every individual has a vector of observations. Currenly, I have the problem that the model does not converge. The log ...
user avatar
2 votes
0 answers
54 views

What is the distribution of the Poisson Sum of gaussians?

I know that the sum of two independent normal random variables is normal. Particulary, when one is copy of other, i.e., if $X_1, X_2 \sim \mathcal{N}(0,\sigma^2)$, independent, we have: $$X_1 + X_2 \...
user avatar
  • 223
2 votes
2 answers
56 views

what is the difference between mixture of two normal distributions and sum of two independent variables

The following denotes a mixture of a standard normal with a normal with the same mean but 100 times the variance: $0.95 \mathrm{~N}(0,1)+ 0.05 \mathrm{~N}(0,100)$ Let Y = 0.95 X + 0.05 Z with X,Z are ...
user avatar
1 vote
0 answers
57 views

MAD & Median of weighted GMM

What is the median and median-absolute-deviation of a weighted GMM in terms of component mean and variance? For example, three normal distributions $A$, $B$, $C$ with means $\mu_a,\mu_b,\mu_c$, ...
user avatar
1 vote
1 answer
159 views

Derivation of the M-step in EM algorithm for a three-dimensional panel mixture model

I have a question regarding the estimation of a latent-class gaussian mixture model, where the model is for three dimensional panel data set with individuals $i$, in country $j$ in time $t$. I want ...
user avatar
1 vote
0 answers
37 views

A mixture of discrete and continuous components

Suppose that random variable $X$ is sampled from Bernoulli with probability $\pi$. Let $Y\sim N(\mu, \sigma^2)$ and denote $Z=XY$. Then, when $X=0$, $Z$ equals 0 (and this happens with probability $1-\...
user avatar
  • 789
0 votes
0 answers
64 views

Is there any way to perform MLE on a 1d gaussian mixture model comprised of two gaussians?

The MLE of the mean of a single gaussian distribution is the mean of the sample. But is there any way to do this when you have two gaussians such as in the figure attached? I know how to estimate the ...
user avatar
15 votes
2 answers
667 views

Finding category with maximum likelihood method

Let's say that we had an information for men and women heights. R code: ...
user avatar
1 vote
0 answers
21 views

What determines performance in recoverying K in Gaussian Mixture Model?

My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm. For simplicity, let's consider the case in ...
user avatar
  • 653
1 vote
1 answer
33 views

Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions?

Is Mixture Modelling the Standard Regression Technique for Dealing with Irregular Distributions? Recently, I came across the use of Gaussian Mixture Distributions being used to model the response ...
user avatar
  • 5,754
2 votes
0 answers
83 views

How to calculate the covariance matrix in (Xu and Knight 2010)?

Setup I'm reading (Xu and Knight 2010), which is a paper about estimating finite Gaussian mixture models using the CECF (Continuous Empirical Characteristic Function) method. The basic idea is to ...
user avatar
  • 148
4 votes
1 answer
444 views

K Means as a special case of GMM (using EM Algorithm)

I am looking for a tutorial/gentle introduction (preferably with mathematics/proofs) on K-means as a special case of Gaussian Mixture Model using the EM Algorithm. I have found this: https://www....
user avatar
  • 7,585
0 votes
0 answers
109 views

Compute log-likelihood in Bernoulli Gaussian Mixture

I'm working on this exercise about Gaussian Mixtures: Here's part of the solution: I don't understand how they came up with this equation for the log-likelihood (red arrow). From what I know, the ...
user avatar
0 votes
0 answers
25 views

Help students understand Gaussian Kernel in mean shift segmentation

I am creating a image mean shift segmentation algorithm for a class. So far what I've done is basically Iterate through each point/pixel on the image and with a given bandwidth(radius) calculate the ...
user avatar
1 vote
2 answers
197 views

Are linear combinations (i.e. "sums") of gaussian distributions also gaussian?

I have always assumed that this is fact : but are there any mathematical theorems that state: finite sums of gaussian distributions are gaussian themselves? In the above picture, does p(x) have a ...
user avatar
  • 5,754
0 votes
0 answers
75 views

Conjugate prior bayesian inference on multivariate GMM

I am trying to understand how the posterior looks like when running Bayesian inference on a multivariate Gaussian-mixture model. $p(\mathbf{x}) \propto \sum_{i=1}^M w_iN(\mathbf{x}|\mu_i,\Sigma_i)$. ...
user avatar
  • 1
0 votes
0 answers
11 views

Is there a formula for the optimal weights for a Gaussian mixture model when the components are considered "correct"?

For simplicity consider two models predicting $Y=Z_0 + \sigma Z_1$, where $Z_i$ are independent $\mathcal{N}(0, 1)$ and each model only take one of the $Z_i$ as a factor. So we have $Y|Z_0$ and $Y|Z_1$...
user avatar
  • 238
1 vote
1 answer
123 views

Assumption of Gaussian mixture model [closed]

I am still confused when I read about the Gaussian Mixture Model and how does it work. One thing that I do not understand is that "GMM assumes the data is a mixture of Gaussian or i.i.d (...
user avatar
  • 173
0 votes
0 answers
12 views

Are the subsets of Mixture Models necessarily parametrized?

Im just learning about mixture models and they are described as a "mix of parametric and non parametric models". My question is how a non-parametric subset would look like? How would e.g. EM-...
user avatar
  • 1
1 vote
1 answer
109 views

Conditional distribution of the weight of a mixture gaussian with data augmentation using gibbs sampling

This question is relate to Differenciate between two distributions using gibbs sampling . For $t=1,\,\dots,\,n$, let's $r_t\sim\mathcal{N}(0,\,\sigma_t^2)$ and $$\sigma_t^2=\left\{\begin{array}{lcl} \...
user avatar
0 votes
1 answer
54 views

Differenciate between two distributions using gibbs sampling [closed]

This question is relate to the post : " Conditional distribution for Gibbs sampling for Gaussian mixture " but is a little bit different. My objective is to know why the algorithm (which is ...
user avatar
0 votes
0 answers
33 views

Interpretation of test set negative log likelihood in neural density estimation applications

I have seen people splitting a dataset into a train and test sets and learning the parameters of a mixture density network using the negative log likelihood cost function on the train set and ...
user avatar
1 vote
1 answer
299 views

Kernel design for Gaussian processes with multiple inputs

How can I design a kernel function when there are multiple input variables and their degree of influence on the covariance with the target variable is different from each other? For example, if the ...
user avatar
  • 155
0 votes
0 answers
35 views

Mixture Density Network on Unbalanced Dataset

I want to use a mixture density network to make predictions with NLL loss (which is working) but my dataset is unbalanced so my network tends to make estimations that look like the distribution of ...
user avatar
  • 11
4 votes
1 answer
141 views

Gaussian distribution with poisson variance

Let's $r_t\sim \mathcal{N}(0,V_t)$ where $V_t = (1+m)^{J_t}$, $m\geq0$ and $J_t\sim \mathcal{P}oi(\lambda)$. How can we compute the distribution of $r_t$ when the parameters are $m$ and $\lambda$. In ...
user avatar
0 votes
0 answers
11 views

Serie calculation - Normal poisson distribution

I would like to compute that serie if it is possible : $\sum_{j\geq 0}\frac{(\lambda m^{-1/2})^j}{j!} \exp{\left(-B\,m^{-j}\right)}$ where $B \geq 0,\, \lambda > 0,\, m>1$. I know it converge as ...
user avatar

1
2 3 4 5
12