Skip to main content

Questions tagged [finite-mixture-model]

a model that represents the presence of subpopulations within an overall population and describes the data in terms of a mixture distribution.

Filter by
Sorted by
Tagged with
0 votes
0 answers
5 views

Is omitting the mixture components with small weight enough to select the number of mixture components

I had a discussion with one of my colleagues and he told me that if we fit k- mixture components and some of them are very small, then we can remove them and hence we select the number of the mixture ...
Dr. Statistics's user avatar
0 votes
1 answer
33 views

EM-algorithm for spatial data

I am very new to Geostatistics (Modeling spatial data) and have some questions: 1- I found that in many literature, the spatial random field is divided into spatial bins. That is, suppose I am ...
Dr. Statistics's user avatar
1 vote
1 answer
56 views

Hypothesis Test Finite Sample Spatial Gaussian Mixture Model

I have $n$ observations of pairs $(x, y)$ and three different models I would like to compare. Model0 is nested within Model1. Model0 is also nested within Model2. I would like to do hypothesis ...
A Friendly Fish's user avatar
0 votes
1 answer
39 views

Example of nonidentification mixture

Consider a continuous r.v. $X$ with pdf $f$ obeying the following finite mixture model for each $x\in \mathbb{R}$: $$ f(x)=\sum_{k=1}^K \lambda_k f_k(x) \quad \lambda_k\geq 0, \sum_k\lambda_k=1 $$ ...
Star's user avatar
  • 889
3 votes
2 answers
198 views

Mixture distributions: an intuition on why we cannot infer the number of mixture components by visual inspection

I am studying mixture models and I would like your help with this question: Consider the distribution $\Gamma$ and assume it is a finite mixture distribution, i.e., $\Gamma=\sum_{k=1}^K \Gamma_k \...
Star's user avatar
  • 889
0 votes
0 answers
10 views

MMD to estimate coefficients of a finite mixture is not quadratic?

I am trying to find a relatively fast density estimation by matching RKHS embeddings. I am somehow surprised by my findings and would like a sanity check: I have some observations $y_1, ..., y_n \in [...
Honorine G's user avatar
1 vote
0 answers
26 views

Proposal parameterization accuracy for Importance Sampling

Suppose I am fitting a Bayesian mixture model that's structured as follows: $$ Y_i | (z_i = k) \sim \mathcal{N}(\mu_k, \sigma_k^2), \quad k = 1, \cdots, K $$ $$ z_i \sim \text{Mult}(1; w_{i1}, \cdots, ...
BasMts's user avatar
  • 23
0 votes
0 answers
24 views

Is it possible to fit a latent class regression where the latent class is grouped by 1 variable, but there is a random intercept by another variable?

The data generating process I am interested in is the following: $$y_i=\beta_g x_i+\epsilon_t+\epsilon_i$$ $$S(i)=s, G(s)=g$$ What this means is that that the $i$th observation, which is made of ...
Chechy Levas's user avatar
  • 1,275
0 votes
0 answers
76 views

Is there a way to make a latent class a predictor of a latent growth model?

I was wondering if it is possible to run a second-order latent growth model, where latent classes can be added as a covariate? I have three variables of which I have 7 repeated measures, say, variable ...
naz's user avatar
  • 23
1 vote
0 answers
76 views

How to calculate standard errors in the finite mixture model

I have a panel dataset, and I am estimating a self-defined likelihood function using finite mixture model. $$ L_i(\theta)=p\prod^T_tL(y_{it}(\theta)|type1) + (1-p)\prod^T_tL(y_{it}(\theta)|type2) $$ $...
jasmine's user avatar
  • 367
3 votes
0 answers
20 views

Feature importance in expectation maximization

The context is using EM algorithm for a mixture model - more precisely Dirichlet Multinomial Mixture, as discussed in Dirichlet Multinomial Mixtures: Generative Models for Microbial Metagenomics. One ...
Roger V.'s user avatar
  • 4,429
0 votes
0 answers
44 views

Inference of a mixture of logistic regression from simulation data in R

Here is the setup and the code that allows to simulate the mixture of 15 component of logistic regression; here each component has 5 common variables that it shares with the other component (with ...
TheBridge's user avatar
  • 207
1 vote
0 answers
78 views

Is it possible to implement Growth Mixture Models/Latent Class Mixed Models in Python? If so, how? R has packages such as lcmm and flexmix for this [closed]

R has lots of support for Finite Mixture Models, as well as specialized packages for more specific Mixture Models for approaches such as Latent Class Mixed Models (lcmm package) and Growth Mixture ...
JElder's user avatar
  • 1,037
1 vote
1 answer
387 views

When to use Mixture Models (e.g., Latent Class Analysis) vs. Cluster Analysis (e.g., K-Means) for segmenting subpopulations?

I have watched a video describing the differences between Cluster Analysis and Mixture Models. https://www.youtube.com/watch?v=HwsMZwhO7wU&t=2s Clustering determines compact clusters and assigns ...
JElder's user avatar
  • 1,037
1 vote
0 answers
58 views

Generate unbalanced sample [closed]

Suppose we have $N$ individuals consisting of two different groups, A and B. Each group contains $N/2$ people. The label of each individual $Z$ follows a binary distribution with probability $P(Z=A)=P(...
Fangzhi Luo's user avatar
0 votes
0 answers
16 views

Supervised classifier for nested interval data and ordinal classes

I'm having trouble formalizing the following classification problem: Let $x_i$ denote univariate (scalar), continuous, real data points Let $y_i \in \mathbb{N}$ be their corresponding labels Classes ...
themodelguy's user avatar
0 votes
0 answers
22 views

Calculate acceptance ratio of Jacobian of split-merge RJMCMC

I am keep studying the RJMCMC and want to ask question regarding the acceptance ratio of split/merge step of RJMCMC The split/merge step, suggested by Richardson and Green (1997) is following for w_j, ...
Kyungmin's user avatar
0 votes
1 answer
43 views

Assumptions and setting for bayesian mixture model (for RJMCMC)

I want to understand about Bayesian mixture model discussed in RJMCMC paper (Richardson and Green, 1997) (https://academic.oup.com/jrsssb/article/59/4/731/7083042) I also posted similar question ...
Kyungmin's user avatar
0 votes
0 answers
91 views

EM algorithm get new parameters by optimizing the Q function (lower bound of likelihood function) or optimizing the likelihood function

We know that in the EM (Expectation-Maximization) algorithm, the E-step determines the $Q$ function by calculating expectations, which is a lower bound of the likelihood function. In the M-step, by ...
yi ren's user avatar
  • 1
0 votes
0 answers
118 views

How to compute the covariance matrix for a mixture model estimated by the EM algorithm

I am trying to compute the observed Fisher information matrix for a mixture model estimated by the EM algorithm. My original thought is to simply compute the second derivative of a mixture density. ...
Lydia2kkx's user avatar
0 votes
1 answer
74 views

Bayesian mixture model with Random Effects in Linear Predictor

My master's thesis involves modelling fMRI data. Each of $M$ participants has a total of $N$ voxels being measured. All these measurements represent an activation amplitude. Aside from this, I have ...
BasMts's user avatar
  • 23
1 vote
0 answers
151 views

Implementing bias-adjustion for step3 latent profile analysis in R [closed]

I am identifying a latent profile model with the Mclust package in R. After identifying an optimal number of cluster I would like to identify possible covariates and distal outcomes via logistic/...
David Janda's user avatar
1 vote
0 answers
80 views

Are there any fast algorithm to estimate a Bernoulli mixture model?

In a problem I need to estimate a Bernoulli mixture model with 3 mixing components. More specifically, we have a random vector $\mathbf{D}=(D_1,D_2,D_3,D_4,D_5)$. $D_1,D_2,D_3,D_4,D_5$ are drawn ...
ExcitedSnail's user avatar
  • 2,966
1 vote
0 answers
83 views

How to choose between a random-effects and a finite mixture model?

Suppose my theory tells me that some outcome $y^{}_{it}$ for individual $i$ in time $t$ is generated by the following process: \begin{align} y_{it} =& \ \alpha^{}_{i} + \beta^{}_{i}x^{}_{it}+\...
lasoon's user avatar
  • 103
0 votes
0 answers
29 views

Metric to check the number of clusters in one dimentional data

I have a set of 1D data as shown below. I want to find a metric that represents the number of clusters in data. Is there any suitable metric that matches my criteria? Example 1D data list: [68, 3, 3, ...
sammy17's user avatar
1 vote
0 answers
67 views

Conditional entropy of finite mixture model

I am trying to understand the conditional entropy of the finite mixture model given in this paper about regularized EM algorithm. On page 3 of the paper: in the finite mixture model, we are given $m$ ...
Jayyu's user avatar
  • 173
2 votes
1 answer
96 views

Are discrete mixtures Gauss quadrature-like integral approximations?

I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution. Let $p_{comp}(x)$ be the PDF of a compound ...
ForceBru's user avatar
  • 330
4 votes
1 answer
54 views

Linear regression model with a distribution over regression equations

Suppose that the observations $(y_t, x_t, k_t)_{t=1}^N$ satisfy the linear regression equation: \begin{equation} \begin{split} y_t = \begin{cases} x_t \beta + e_t & w.p. \; \theta \\ k_t \gamma + ...
Luca Gi's user avatar
  • 252
3 votes
1 answer
162 views

Draw random numbers from finite mixture model

Setup Let X follow a fininte mixture model with density \begin{equation} f=\lambda f_1+(1-\lambda) f_2 \end{equation} Where $f_1$ and $f_2$ are both log-normal densities with parameters $(\mu_1, \...
FicusBenji's user avatar
1 vote
1 answer
246 views

How to apply EM algorithm in case of mixture distribution?

I am familiar with regression linear models, and EM algorithms. However, I do not get the idea of fitting the mixture of regression linear models using the EM algorithm. So, what I think about it is ...
Dr. Statistics's user avatar
2 votes
0 answers
78 views

Why separating the data in training and test sets is not feasible in unsupervised learning problem?

Based on my understanding: Unsupervised learning problems are modeling data with no labels. Hence, we try to cluster a given data into clusters. Supervised learning problems are modeling data with ...
Maryam's user avatar
  • 1,680
2 votes
1 answer
97 views

Is there a relationship between the number of the mixture components and the overfiting of the model?

I read the following: To prevent overfitting we would like to work with as few components as possible". How does the number of the mixture component affect the fit of the model? Is that because ...
Maryam's user avatar
  • 1,680
0 votes
1 answer
80 views

What is the average bias in MLE of 2-component univariate Gaussian mixture model?

Imagine that you have a standard 2-component univariate Gaussian mixture model: $$p(x_i∣θ)=(1-λ)N(x_i|μ_1,σ_1^2 )+λN(x_i|μ_2,σ_2^2 )$$ $$θ=\{μ_1,μ_2,σ_1,σ_2,λ\}$$ $$L(θ;x)=∏_{i=1}^N p(x_i |θ)$$ The ...
Andrey Chetverikov's user avatar
1 vote
0 answers
55 views

Universal Approximation Capabilities of Mixture of Weibulls

Can a mixture of $N$ Weibull distributions approximate any continuous density with non-negative support, if $N$ is sufficiently large? (If so, a reference to the proof would be greatly appreciated). (...
zen_of_python's user avatar
0 votes
0 answers
232 views

How to identify a mixture of poisson distribution and Gaussian distribution from the data?

Here is the distribution of the data. It seeme to me that it is a mixture of a poisson distribution at the begining of zero value and a Gaussian distribution. I also used the ...
lesdormis's user avatar
2 votes
1 answer
384 views

Fitting truncated normal mixtures in R

I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
gregorp's user avatar
  • 371
0 votes
0 answers
33 views

Can we use Dirichlet process to simultaneously estimate the number of mixtures and component distribution of a Bernoulli mixture?

Suppose I have a random sample on a Bernoulli random variable $\{X_i\}_{i=1}^N$ generated from model $p=\sum_{k=1}^K\pi_kp_k$,where $p\equiv Pr(X=1)$ and $p_k\equiv Pr(X=1|k)$, and $\pi_k$ are the ...
ExcitedSnail's user avatar
  • 2,966
5 votes
0 answers
796 views

Hamiltonian Monte Carlo vs. "Metropolis-Hastings with a Hamiltonian step"

In Hamiltonian Monte Carlo the proposal is accepted with probability: $$ \alpha\left(\mathbf{x}_n(0),\mathbf{x}_n(L\Delta t)\right) = \min\left(1, \frac{\exp\left[-H\left(\mathbf{x}_n(L\Delta t),\...
Roger V.'s user avatar
  • 4,429
2 votes
1 answer
426 views

Distribution of sum of $n$ random variables with mixture of two exponential distributions

Suppose that the random variable $Y$ follows a mixture of two exponential distributions, that is \begin{equation} f_Y(y) = \sum_{i=1}^{2}\pi_i f(y| \lambda_i) \end{equation} where $\pi$ stands for ...
Statistics 's user avatar
2 votes
1 answer
145 views

Simultaneous Bayes Estimation

Given $\theta_i$, $0 < \theta_i < 1$, a sequence of independent Bernoulli ($\theta_i$) random variables from i subpopulations, that are also independent across subpopulations. Suppose i=2 (2 ...
Guest's user avatar
  • 23
0 votes
0 answers
82 views

Anomaly detection with strong prior assumptions about data generating process

I have data that can be described using the model $$ y \sim \mathcal{N}\big(f(x; \Theta), \, \sigma^2) $$ where $f$ is some function with known functional form, but unknown parameters. I also can make ...
Tim's user avatar
  • 140k
1 vote
0 answers
308 views

Expectation maximization: does the likelihood always increase monotonically?

When working with (gaussian) mixture models, I always took it for a mathematical fact that the marginal likelihood increases with every iteration step. If it were not the case, it always meant an ...
Roger V.'s user avatar
  • 4,429
1 vote
1 answer
64 views

Using regression to both estimate and attribute a single value to a subset of established categories

I am using Stata 15.1 I have a dataset with some 12,000 observations with a continuous dependent variable and 4 continuous independent variables. Each observation is also prior assigned to one of ...
JamieH's user avatar
  • 11
2 votes
1 answer
69 views

Mixture of a realization of uniform variable and noise

Suppose that $X \sim U[0,1]$. After $X = x$ has realized, we don't observe $x$, but we instead observe a noisy signal of $x$, defined as $S = \tau x + (1 - \tau) U$, where $\tau \sim Ber(p)$ and $U \...
keepfrog's user avatar
  • 215
1 vote
0 answers
28 views

What determines performance in recoverying K in Gaussian Mixture Model?

My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm. For simplicity, let's consider the case in ...
jmb's user avatar
  • 715
0 votes
1 answer
62 views

EM algorithm when there are too many components to calculate the function Q

Assuming a regression model as follow: $$\mathbf{y} = \mathbf{x}\beta + \mathbf{\varepsilon}$$ where $\mathbf{y}=(y_1,...,y_n)^T\in\mathbb{R}^{n\times 1}$, $\mathbf{x}=(x_1,...,x_n)^T\in\mathbb{R}^{n\...
Deku's user avatar
  • 21
1 vote
0 answers
76 views

Maximum likelihood estimation when the model is misspecified (and the true data generating process is a mixture model)

I'm interested in the properties of maximum likelihood estimators under a particular form of model misspecification: We observe data $\left\{X_i\right\}$ generated from a finite mixture model Let $\...
Adrian's user avatar
  • 4,374
1 vote
1 answer
34 views

Parameter Identification in a (Simple) Mixture Model

I have a very mundane clarification question from some old lecture notes. The notes: Consider the model $$Y_i=(1+D_i)\varepsilon_i$$ where $(D_i,\varepsilon_i)\overset{iid}{\sim}Bernoulli(p)\times N(...
equanimity's user avatar
0 votes
0 answers
953 views

What is truncated gaussian mixture model?

I am interested in the Gaussian mixture model. I read about it and I think I am good with it. However, found that there is something called truncated Gaussian mixture model, which I do not understand. ...
Maryam's user avatar
  • 1,680
1 vote
1 answer
527 views

How to implement a mixture model for Dirac Delta and Normal distributions?

How could I fit data with observations from one Dirac delta component and $n$ normal distributed components? Where $n$ usually is between 1 and 5. My prior knowledge is that one component really is a ...
Jona Engel's user avatar