Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

0
votes
0answers
6 views

Piecewise Pareto Distribution

What are the best practices for Piece-wise Pareto Distribution or maybe Pareto Mixture Model(?). Example: $x\in [0, 1) \Rightarrow \alpha=0.1$ $x\in [1, 10) \Rightarrow \alpha=0.5$ $x\in [10, 100) ...
0
votes
0answers
27 views

finite mixture models with countreg and flexmix

I am testing making a finite mixture model for poisson and negative binomial on some artificial data using flexmix and countreg packages. Here is the core part of my small script: ...
0
votes
0answers
8 views

Joint “density” of data and indicators in Bayesian mixture model

I'm currently working through the chapter on finite mixture models in BDA3 and came across the following model setup (with the usual slight abuse of notation): Let $\lambda=(\lambda_1,\dots,\lambda_H)...
0
votes
0answers
20 views

Seeking guidance on determining (predicting) probability (distribution) using Mixture Density (Neural) Networks

I work for an education PAAS (platform as a service) company helping colleges & training institutes manage their course evaluation and recruitment. Students can re-take tests multiple times, tests ...
0
votes
0answers
22 views

Using mixture models as a prediction model

I read many papers that used mixture model to predict, for example, The diagnosis of a disease. For example, assume that there are a measurements on different variables for healthy patient and ...
0
votes
1answer
16 views

What does it means for “fit a less parsimonious model” in a clustering algorithm?

I'm now trying to implement the algorithm presented in https://www.stat.washington.edu/raftery/Research/PDF/fraley2005.pdf. The algorithm is the following one: First I get a mixture model for ...
0
votes
0answers
8 views

Identification of mixture models with location shift

I am studying identification of mixture models of the type $$ F(x)=\sum_{j=1}^J \lambda_j G(x-\mu_j) $$ with $F,G$ denoting univariate CDFs, $G$ symmetric about $0$, $\mu_1<\mu_2<...<\mu_J$, $...
1
vote
1answer
27 views

Practical considerations on a mixture of Multivariate Normals, with many terms

Let's say the density of $Y$ is given by $p(y)=\frac{1}{L}\sum^L_{i=1}N(y\mid \mu_i, \Sigma_i)$, where $N(y \mid \mu_i, \Sigma_i)$ is the multivariate normal density evaluated at $y$, with known $L,\...
3
votes
1answer
22 views

Is it acceptable in mathematical saying that the E-step is equal to posterior probability

I am studying EM-algorithm for mixture data. I read that some authors said that, the E-step is equivalent to the calculation of the posterior probability (I think this come from Bayesian rule). So, my ...
0
votes
1answer
20 views

Equivalent way of rewriting a two-component mixture

I'm confused on the following equivalent way of rewriting a two-component mixture. Consider the two-component conditional mixture $$ F(z|x)=\lambda F_1(z|x)+(1-\lambda)F_2(z|x) $$ where all the $F$'...
1
vote
0answers
26 views

A 2 component mixture is symmetric if and only if $\lambda\in \{0,1,\frac{1}{2}\}$

Consider the following mixture of two densities $$ f(x)=\lambda g(x-\mu_1)+(1-\lambda)g(x-\mu_2) $$ with $\lambda\in [0,1]$, $g(\cdot)$ symmetric around zero, $\mu_1<\mu_2$. Claim: the mixture is ...
0
votes
0answers
14 views

What is the different between the set of all model parameters and the parameter vector of the nth component

I read many articles about mixture models. I read that the author called the model parameters as "a set of all model parameters", while they said "parameter vector for the n-th component". I wonder ...
0
votes
0answers
25 views

Mixture of $K$ components

Consider a random vector $$ X\equiv \begin{pmatrix} X_1\\ X_2\\ X_3 \end{pmatrix} $$ with pdf $$f(x)=\overbrace{\sum_{k=1}^ K \frac{1}{K} f_k(x)}^{\text{finite mixture}}$$ and $\forall k=1,...,K$ $...
0
votes
1answer
40 views

Mixture of two components

Consider a univariate probability density function $p(x)$ that is a mixture of $2$ probability density functions with weights $\eta, 1-\eta$ and $\eta\in (0,1)$: $$ p(x)=(1-\eta)g(x)+\eta f(x) \...
2
votes
1answer
38 views

Clustering circles with different radii with Gaussian Mixture Models

I am interested in clustering $N$ circles in the plane with varying radii using a Gaussian mixture model. The radius of each circle is an integer number $R_i\in\mathbb{N}$ determined from observation. ...
2
votes
1answer
195 views

Posterior distribution of mixture models

In the context of mixture models in Bayesian inference, one can assume that the general form of the joint posterior for a mixture model of $k$ components is $$ \begin{equation} p( \boldsymbol{\...
0
votes
0answers
9 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
3
votes
2answers
51 views

Confusion in modelling finite mixture model

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated: The usual representation (of a finite ...
3
votes
1answer
59 views

Generate a finite mixture distribution by sampling mixture component parameters

Assume I have a some mixture distribution, $H$, with mean $\mu$ and variance $\sigma^2$. $H$ is a mixture of $n$ component distributions where all component weights are equal. Let $\mu_i$ be the mean ...
0
votes
0answers
19 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
78 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
0
votes
1answer
159 views

PyMC3: Mixture Model with Latent Variables

I have a rather basic knowledge of Bayesian inference and I'm somewhat new to MCMC and PyMC3. Can I model data that looks like this? ...
2
votes
1answer
47 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
0
votes
0answers
153 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...
1
vote
0answers
25 views

Model Selection and inference for mixture of logisitc regressions (or GLM) with heterogenous covariates by component

I am facing a problem which should be quite common IMO but for which I don't find relevant contribution. So the situation is this. Let's say that a binary response $Y$ is generated by a mixture of $K$ ...
4
votes
1answer
61 views

Marginal Distribution of Exponential Mixture Model

I am currently trying to marginalize over the scale parameter in a mixture distribution of exponential pdfs, but I do not trust my result. Let me show you my steps: Probability Density Function The ...
2
votes
2answers
358 views

Question about the latent variable in EM algorithm

In mixture models, Expectation maximization algorithm (EM) is a commonly used method to estimate the model parameters. Suppose that I have bivariate mixture model with two mixture components, with ...
1
vote
0answers
21 views

What is the appropriate analysis for this type of repeated measures multi-binary data?

There is a popular theory within psychology that certain emotions will trigger "prototypical" facial expressions defined by the simultaneous contraction of specific facial muscles. For example, if a ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
-1
votes
2answers
41 views

Variance of a mixture of Normals with same $\sigma^2_i$

Let $Y\sim \sum^N_{i=1}\omega_iN(m_i,h^2 V)$. The text I'm reading states that $Var(Y)=(1+h^2)V$, when $m_i=\theta_i$, where $\theta_i$ are draws taken from $P(\theta|D)$, and $V=Var(\theta|D)$ I ...
1
vote
0answers
195 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
0
votes
0answers
23 views

simulating finite gaussian mixture models

I'm doing the project for my data science exam, I'm not actually a statistician so I had some doubts. This is my data set:y=c(9.259778,9.178891,9.262575,9.212859,9.241963,9.290917,9.178455,9.183814,9....
3
votes
0answers
27 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
2
votes
1answer
46 views

Fitting a finite mixture: choice of the distribution and model selection for the number of components

This is a question about finite mixture models (FMM). We want to fit a dataset $D$ but we are not 100% sure of which distributions we should use to create the mixture; we do not how many clusters ...
1
vote
1answer
187 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
0
votes
1answer
17 views

Why does the mixture dependencies exist between variables?

Sorry if my question is clear to most of you. As a mathematics background, I really just start working with the mixture and would like to understand it in a clear way. Mixture dependencies are when ...
0
votes
1answer
178 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
0
votes
1answer
30 views

Very steep decrease in information criteria for mixture models with more components

I am analyzing data using mixture modeling. When I plot the information criteria (the BIC) for a series of models (with different model specifications and different number of mixture components), I ...
3
votes
1answer
689 views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
0
votes
0answers
168 views

Mixture modelling with skewed distributions

I am trying to find a R library that splits a distribution into a symmetric and asymmetric components. I have a distribution that I want to split into two components, one skewed and the other most ...
3
votes
0answers
26 views

Explicitly Show Conditional Independence In a Mixture Model

Suppose we have the following mixture model: $$\pi \sim Dirichlet(\alpha)$$ $$\theta_1 ,\ldots ,\theta_K \overset{iid}{\sim} N(0,1)$$ $$Z_1 , \ldots , Z_n \mid \pi \overset{iid}{\sim} Categorical(\...
0
votes
1answer
50 views

Why the sample method of mixture distribution works?

For example this thread: Generating random variables from a mixture of Normal distributions First choose a distribution according to the weights. Then sample from the chosen distribution. How to ...
6
votes
1answer
125 views

Observations for a bivariate Gaussian mixture

Consider two random vectors $X\equiv(X_1, X_2),Y\equiv(Y_1, Y_2)$ distributed as below 1) $X\sim N(\begin{pmatrix} \mu_{X,1}\\ \mu_{X,2}\\ \end{pmatrix}, \begin{pmatrix} v_{X,1} & 0\\ 0 & v_{...
0
votes
2answers
1k views

Mixture models vs Mixed models

I was wondering what is the difference between Mixture models and Mixed models in Statistics? Explaining with Any practical example would be appreciated.
3
votes
0answers
132 views

Dirichlet Process vs. Mixture Models with Many Mixtures

The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
0
votes
0answers
30 views

DPMM asymptotics for finite mixtures?

My understanding is that using a Dirichlet process mixture model for a mixture with finitely many components will result in a misspecified model. Are there any asymptotic results or bounds on the ...
3
votes
1answer
51 views

85% of the samples come from an unknown distribution, the rest come from the same distribution with a larger variance.How to recognize them?

Assume I have a data, say columns are the samples, and rows are the features, the problem is that around 85% of the samples come from an unknown distribution but the rest come from the same type of ...
2
votes
1answer
193 views

Does the sum of updated mixture weights in EM algorithm equal 1 in M step, why?

I am usign EM algorithm to estimate my model's parameters. As you know, the mixture weight must be sum to one. $\sum \pi_n = 1$ where $n$ is the number of mixture component. In M step we can find the ...
0
votes
1answer
230 views

Does a mixture model need to sum or integrate to $1$?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean $\mu$ and variance $\...
1
vote
1answer
149 views

What is the relationship between gaussian mixture models and covariate regression?

Consider a histogram of observations $y$ which look a bit like this, Imagine that the two bell-curve shaped histograms (ignore the actual fitted curves) correspond to observations from females and ...