Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [finite-mixture-model]

Finite mixture model represents the presence of subpopulations within an overall population and describes the data in terms of mixture distribution. Finite mixture models are commonly used for model-based clustering, but they can be used also for other problems, like cluster-wise regression, mixture of generalized linear models and other mixtures. Finite mixture models for binary and categorical data are known under the name of latent class analysis.

0
votes
0answers
67 views

Posterior distribution of mixture models

In the context of mixture models in bayesian inference, one can assume that the general form of the joint posterior for a mixture model of $k$ components is $$ \begin{equation} p( \boldsymbol{\...
2
votes
1answer
78 views

univariate mixture of Gaussians using EM and Gibbs sampling

Mixture models are quite fun to work with. For example, we have the following three algorithms for univariate mixtures of Gaussians: EM, A gibbs sampler using independent priors and a gibbs sampler ...
0
votes
0answers
8 views

Typo in the definition of Finite Mixed Model in Machine Learning a probabilistic Perspective

In subsection 25.2.1 it's stated, regarding finite mixture model: The usual representation (of a finite mixture model) is as follows: $p(x_i|z_i = k, \boldsymbol\theta) = p(x_i|\boldsymbol\...
2
votes
2answers
43 views

Confusion in modelling finite mixture model

From the book "Machine Learning a probabilistic Perspective", I'm reading about finite/infinite mixture models. Particularly at paragraph 25.2.1 it's stated: The usual representation (of a finite ...
3
votes
1answer
46 views

Generate a finite mixture distribution by sampling mixture component parameters

Assume I have a some mixture distribution, $H$, with mean $\mu$ and variance $\sigma^2$. $H$ is a mixture of $n$ component distributions where all component weights are equal. Let $\mu_i$ be the mean ...
0
votes
0answers
11 views

Imposing independence constraints in mixture modeling of correlated data?

For 1-D signals (spectra) or 2-D signals (images), is there a way to impose the constraint that the data within a group is uncorrelated? I am iteratively applying background correction model fitted to ...
1
vote
1answer
41 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
0
votes
1answer
60 views

PyMC3: Mixture Model with Latent Variables

I have a rather basic knowledge of Bayesian inference and I'm somewhat new to MCMC and PyMC3. Can I model data that looks like this? ...
2
votes
1answer
34 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
0
votes
0answers
41 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...
1
vote
0answers
22 views

Model Selection and inference for mixture of logisitc regressions (or GLM) with heterogenous covariates by component

I am facing a problem which should be quite common IMO but for which I don't find relevant contribution. So the situation is this. Let's say that a binary response $Y$ is generated by a mixture of $K$ ...
4
votes
1answer
52 views

Marginal Distribution of Exponential Mixture Model

I am currently trying to marginalize over the scale parameter in a mixture distribution of exponential pdfs, but I do not trust my result. Let me show you my steps: Probability Density Function The ...
2
votes
2answers
246 views

Question about the latent variable in EM algorithm

In mixture models, Expectation maximization algorithm (EM) is a commonly used method to estimate the model parameters. Suppose that I have bivariate mixture model with two mixture components, with ...
1
vote
0answers
17 views

What is the appropriate analysis for this type of repeated measures multi-binary data?

There is a popular theory within psychology that certain emotions will trigger "prototypical" facial expressions defined by the simultaneous contraction of specific facial muscles. For example, if a ...
7
votes
5answers
1k views

Random variable defined as A with 50% chance and B with 50% chance

Note: this is a homework problem so please don't give me the whole answer! I have two variables, A and B, with normal distributions (means and variances are known). Suppose C is defined as A with 50% ...
-1
votes
2answers
37 views

Variance of a mixture of Normals with same $\sigma^2_i$

Let $Y\sim \sum^N_{i=1}\omega_iN(m_i,h^2 V)$. The text I'm reading states that $Var(Y)=(1+h^2)V$, when $m_i=\theta_i$, where $\theta_i$ are draws taken from $P(\theta|D)$, and $V=Var(\theta|D)$ I ...
0
votes
0answers
32 views

Interpretation and presentation of coefficients for continuous variables in poLCA

I am analysing data on symptoms, signs, and autopsy findings (a set of binary Y/N variables), viral serology, for several viruses, and a few other covariates (Age, gender, site) in pigs. After some ...
1
vote
0answers
97 views

fitting curve to my data and calculating fwhm

Hello and thank you in advance for your inputs. I am trying to find a model in R that will give me curves that fit my data. I am aiming for 2 peaks (i am thinking of normal distributions but might be ...
0
votes
0answers
14 views

simulating finite gaussian mixture models

I'm doing the project for my data science exam, I'm not actually a statistician so I had some doubts. This is my data set:y=c(9.259778,9.178891,9.262575,9.212859,9.241963,9.290917,9.178455,9.183814,9....
3
votes
0answers
21 views

Possible statistical tests to separate two distributions within a dataset

I have a dataset that contains a range of values. I have created a frequency distribution of the values, and have included the plot below. To my untrained eye, it appears that the frequency ...
2
votes
1answer
40 views

Fitting a finite mixture: choice of the distribution and model selection for the number of components

This is a question about finite mixture models (FMM). We want to fit a dataset $D$ but we are not 100% sure of which distributions we should use to create the mixture; we do not how many clusters ...
1
vote
1answer
151 views

Prior for covariance matrices in Gaussian Mixtures Model

I am looking to choose a prior that helps me avoid singularities (as mentioned in this answer) in the covariance matrices of a GMM model. The Jeffrey prior (or a simple improper prior) would be very ...
0
votes
0answers
52 views

gating function for mixture of tree bayes nets

I am implementing a mixture of tree bayesian networks using bagging by generating k sets of boostrap samples and am currently using a simple uniform weight as the gating function, but am looking for ...
0
votes
1answer
16 views

Why does the mixture dependencies exist between variables?

Sorry if my question is clear to most of you. As a mathematics background, I really just start working with the mixture and would like to understand it in a clear way. Mixture dependencies are when ...
0
votes
0answers
10 views

Can describe a dependencies as weak dependency?

Mixture dependencies structure can be very strong. For example, 60% of the data come from the first mixture component. However, sometime we may have a weak mixture ...
0
votes
0answers
17 views

LRT for betabinomial mixture model?

Is the LRT appropriate to compare two mixtures of betabinomials (or gaussians), one with k components, the other with k+1? I've read in some place that it is appropriate, but after that found papers ...
0
votes
1answer
97 views

Number of parameters mixture model

In order to do a LRT between two mixture models with different numbers of components, I need to know the number of parameters. I would like to know the answer both for: a) Gaussian mixture model b) ...
0
votes
0answers
69 views

Calculate moments of log-normal mixture in Matlab

I have given the parameters of a 3-component log-normal mixture distribution, i.e. the weights $w_1, w_2, w_3 := 1 - w_1 - w_2$, as well as the component's mean and variance, $\mu_i, \sigma^2_i, i = 1,...
0
votes
0answers
20 views

Mixed Model IRT (MMIRT) versus Factor Mixture Modeling (FMM)

I wonder if my following description about MMIRT and FMM is correct. This will help me check my understanding of the two statistical techniques: MMIRT is a hybrid between latent class analysis (LCA) ...
0
votes
1answer
29 views

Very steep decrease in information criteria for mixture models with more components

I am analyzing data using mixture modeling. When I plot the information criteria (the BIC) for a series of models (with different model specifications and different number of mixture components), I ...
3
votes
1answer
559 views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
0
votes
0answers
120 views

Mixture modelling with skewed distributions

I am trying to find a R library that splits a distribution into a symmetric and asymmetric components. I have a distribution that I want to split into two components, one skewed and the other most ...
3
votes
0answers
26 views

Explicitly Show Conditional Independence In a Mixture Model

Suppose we have the following mixture model: $$\pi \sim Dirichlet(\alpha)$$ $$\theta_1 ,\ldots ,\theta_K \overset{iid}{\sim} N(0,1)$$ $$Z_1 , \ldots , Z_n \mid \pi \overset{iid}{\sim} Categorical(\...
0
votes
1answer
28 views

Why the sample method of mixture distribution works?

For example this thread: Generating random variables from a mixture of Normal distributions First choose a distribution according to the weights. Then sample from the chosen distribution. How to ...
6
votes
1answer
118 views

Observations for a bivariate Gaussian mixture

Consider two random vectors $X\equiv(X_1, X_2),Y\equiv(Y_1, Y_2)$ distributed as below 1) $X\sim N(\begin{pmatrix} \mu_{X,1}\\ \mu_{X,2}\\ \end{pmatrix}, \begin{pmatrix} v_{X,1} & 0\\ 0 & v_{...
0
votes
2answers
940 views

Mixture models vs Mixed models

I was wondering what is the difference between Mixture models and Mixed models in Statistics? Explaining with Any practical example would be appreciated.
0
votes
1answer
523 views

Latent class analysis (LCA) with mixed data

first of all I am sorry if a similar question has already been asked, but I have not found a thread dealing with this topic. I want to use latent class analysis on 4 variables as an approach to ...
3
votes
0answers
103 views

Dirichlet Process vs. Mixture Models with Many Mixtures

The Dirichlet Process prior is a Bayesian non-parametric prior to model your data as coming from an infinite mixture of distributions. Since your data is finite, only a finite number of these mixture ...
0
votes
0answers
30 views

DPMM asymptotics for finite mixtures?

My understanding is that using a Dirichlet process mixture model for a mixture with finitely many components will result in a misspecified model. Are there any asymptotic results or bounds on the ...
3
votes
1answer
51 views

85% of the samples come from an unknown distribution, the rest come from the same distribution with a larger variance.How to recognize them?

Assume I have a data, say columns are the samples, and rows are the features, the problem is that around 85% of the samples come from an unknown distribution but the rest come from the same type of ...
1
vote
1answer
115 views

Does the sum of updated mixture weights in EM algorithm equal 1 in M step, why?

I am usign EM algorithm to estimate my model's parameters. As you know, the mixture weight must be sum to one. $\sum \pi_n = 1$ where $n$ is the number of mixture component. In M step we can find the ...
0
votes
1answer
165 views

Does a mixture model need to sum or integrate to $1$?

Suppose that we have a mixture model: $$ p_\theta(y) = \sum_{k = 1}^{K}w_k \phi(y;\mu_k, \sigma^2_k) $$ where $\phi(y;\mu_k, \sigma^2_k)$ is the normal density at $y$ with mean $\mu$ and variance $\...
1
vote
1answer
123 views

What is the relationship between gaussian mixture models and covariate regression?

Consider a histogram of observations $y$ which look a bit like this, Imagine that the two bell-curve shaped histograms (ignore the actual fitted curves) correspond to observations from females and ...
1
vote
0answers
142 views

Inference with Mixture of Linear Regression

I have used an EM algorithm to fit a finite mixture of linear regression to my data, and cluster them into $k$ clusters. Now that I have my clusters with the estimated parameters $\beta_k$ and $\...
2
votes
0answers
135 views

How to find the covariance of a Bernoulli Mixture distribution?

I am reading the Pattern Recognition and Machine Learning book and on page 445, it states that the covariance of the Bernoulli mixture distribution is $$Cov(\mathbf{x}) = \sum^K_{k=1} \pi_k (\Sigma_k ...
1
vote
0answers
31 views

calculate percent of sample in overlapping distributions

I'm looking at 80k samples representing results of a test after a fixed period of time. I think it looks like a combination of normal curves. I see a fair amount of documentation about combining ...
1
vote
1answer
87 views

Decompose mixture of gamma and gaussian distributions

I have a data, which looks like mixture of $\gamma$ and Gausian distributions: Could you help me to find parameters of these distributions?
2
votes
0answers
435 views

Programming a mixture of a Gamma with a Normal distribution using R

I have some data x in R which seems to be a mixture of a Gamma and Normal distribution. Therefore I'd like to model this as a mixture model consisting of said distributions, but I don't know how to ...
2
votes
0answers
153 views

Fitting mixture model of different distributions in R [closed]

I have some data $X$ which seems to consist of two different subpopulations, one following a Gamma and one following a Normal distribution. Therefore I'd like to model this as a mixture model ...
2
votes
0answers
55 views

Does a latent class model become unstable with large samples and dimensions?

I have a dataset of approx 2million individuals with 40 dichotomous variables signifying presence or not of diseases (e.g heart disease, asthma, etc.) along with some sociodemographic info. I have ...