Questions tagged [finite-mixture-model]

a model that represents the presence of subpopulations within an overall population and describes the data in terms of a mixture distribution.

Filter by
Sorted by
Tagged with
1 vote
0 answers
36 views

Universal Approximation Capabilities of Mixture of Weibulls

Can a mixture of $N$ Weibull distributions approximate any continuous density with non-negative support, if $N$ is sufficiently large? (If so, a reference to the proof would be greatly appreciated). (...
user avatar
0 votes
0 answers
8 views

Is there a good package to implement latent profile analysis in R, which allows use of FIML for missing data

We are looking to conduct latent profile analysis in R, using a large number of continuous variables (physical activity level at different times of day). We have objective measurement of physical ...
user avatar
0 votes
0 answers
20 views

How to identify a mixture of poisson distribution and Gaussian distribution from the data?

Here is the distribution of the data. It seeme to me that it is a mixture of a poisson distribution at the begining of zero value and a Gaussian distribution. I also used the ...
user avatar
0 votes
0 answers
9 views

From GMM to Wishart MM

Given a random vector $X$ distributed as a vector Gaussian mixture model in $\mathbb R^n$ mixing $K$ centered multivariate Gaussian distributions as follows $$ \textstyle X \sim \sum_{i=k}^K \alpha_k \...
user avatar
0 votes
0 answers
20 views

GMM clustering with binary and multicollinear data

I am using GMM clustering on bank data. The data have both categorical and numerical attributes. The categorical data were converted to numerical using binary encoding. I have a couple of questions: ...
user avatar
1 vote
1 answer
101 views

Fitting truncated normal mixtures in R

I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
user avatar
  • 189
0 votes
0 answers
13 views

Can we use Dirichlet process to simultaneously estimate the number of mixtures and component distribution of a Bernoulli mixture?

Suppose I have a random sample on a Bernoulli random variable $\{X_i\}_{i=1}^N$ generated from model $p=\sum_{k=1}^K\pi_kp_k$,where $p\equiv Pr(X=1)$ and $p_k\equiv Pr(X=1|k)$, and $\pi_k$ are the ...
user avatar
  • 1,682
1 vote
0 answers
42 views

Hamiltonian Monte Carlo vs. "Metropolis-Hastings with a Hamiltonian step"

In Hamiltonian Monte Carlo the proposal is accepted with probability: $$ \alpha\left(\mathbf{x}_n(0),\mathbf{x}_n(L\Delta t)\right) = \min\left(1, \frac{\exp\left[-H\left(\mathbf{x}_n(L\Delta t),\...
user avatar
  • 1,580
2 votes
1 answer
150 views

Distribution of sum of $n$ random variables with mixture of two exponential distributions

Suppose that the random variable $Y$ follows a mixture of two exponential distributions, that is \begin{equation} f_Y(y) = \sum_{i=1}^{2}\pi_i f(y| \lambda_i) \end{equation} where $\pi$ stands for ...
user avatar
0 votes
0 answers
15 views

Monte Carlo for Dirichlet Multinomial Model

Problem I am trying to implement Markov Chain Monte Carlo for the Dirichlet Multinomial mixture, described in this reference (where one used the expectation maximization algorithm). The model is as ...
user avatar
  • 1,580
0 votes
0 answers
11 views

mixture of finite regressions without a response variable

In finite mixture modelling, in particular mixture of regressions modelling, we are interested in finding latent trajectories against a response variable. But what if there is no known response ...
user avatar
2 votes
1 answer
97 views

Simultaneous Bayes Estimation

Given $\theta_i$, $0 < \theta_i < 1$, a sequence of independent Bernoulli ($\theta_i$) random variables from i subpopulations, that are also independent across subpopulations. Suppose i=2 (2 ...
user avatar
  • 23
0 votes
0 answers
67 views

Anomaly detection with strong prior assumptions about data generating process

I have data that can be described using the model $$ y \sim \mathcal{N}\big(f(x; \Theta), \, \sigma^2) $$ where $f$ is some function with known functional form, but unknown parameters. I also can make ...
user avatar
  • 113k
0 votes
0 answers
9 views

Model choice with expectation maximization: which likelihood?

When deciding about the number of mixture components using Akaike or bayesian information criteria, should one use the full likelihood or the likelihood marginalized over the latent variables? Both ...
user avatar
  • 1,580
0 votes
0 answers
42 views

Expectation maximization: does the likelihood always increase monotonically?

When working with (gaussian) mixture models, I always took it for a mathematical fact that the marginal likelihood increases with every iteration step. If it were not the case, it always meant an ...
user avatar
  • 1,580
1 vote
1 answer
50 views

Using regression to both estimate and attribute a single value to a subset of established categories

I am using Stata 15.1 I have a dataset with some 12,000 observations with a continuous dependent variable and 4 continuous independent variables. Each observation is also prior assigned to one of ...
user avatar
  • 11
1 vote
1 answer
48 views

Mixture of a realization of uniform variable and noise

Suppose that $X \sim U[0,1]$. After $X = x$ has realized, we don't observe $x$, but we instead observe a noisy signal of $x$, defined as $S = \tau x + (1 - \tau) U$, where $\tau \sim Ber(p)$ and $U \...
user avatar
1 vote
0 answers
21 views

What determines performance in recoverying K in Gaussian Mixture Model?

My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm. For simplicity, let's consider the case in ...
user avatar
  • 653
0 votes
1 answer
34 views

EM algorithm when there are too many components to calculate the function Q

Assuming a regression model as follow: $$\mathbf{y} = \mathbf{x}\beta + \mathbf{\varepsilon}$$ where $\mathbf{y}=(y_1,...,y_n)^T\in\mathbb{R}^{n\times 1}$, $\mathbf{x}=(x_1,...,x_n)^T\in\mathbb{R}^{n\...
user avatar
  • 1
1 vote
0 answers
27 views

Maximum likelihood estimation when the model is misspecified (and the true data generating process is a mixture model)

I'm interested in the properties of maximum likelihood estimators under a particular form of model misspecification: We observe data $\left\{X_i\right\}$ generated from a finite mixture model Let $\...
user avatar
  • 3,768
0 votes
0 answers
10 views

Is there a formula for the optimal weights for a Gaussian mixture model when the components are considered "correct"?

For simplicity consider two models predicting $Y=Z_0 + \sigma Z_1$, where $Z_i$ are independent $\mathcal{N}(0, 1)$ and each model only take one of the $Z_i$ as a factor. So we have $Y|Z_0$ and $Y|Z_1$...
user avatar
  • 238
1 vote
1 answer
23 views

Parameter Identification in a (Simple) Mixture Model

I have a very mundane clarification question from some old lecture notes. The notes: Consider the model $$Y_i=(1+D_i)\varepsilon_i$$ where $(D_i,\varepsilon_i)\overset{iid}{\sim}Bernoulli(p)\times N(...
user avatar
0 votes
0 answers
153 views

What is truncated gaussian mixture model?

I am interested in the Gaussian mixture model. I read about it and I think I am good with it. However, found that there is something called truncated Gaussian mixture model, which I do not understand. ...
user avatar
  • 1,032
1 vote
1 answer
89 views

How to implement a mixture model for Dirac Delta and Normal distributions?

How could I fit data with observations from one Dirac delta component and $n$ normal distributed components? Where $n$ usually is between 1 and 5. My prior knowledge is that one component really is a ...
user avatar
0 votes
0 answers
32 views

How do you differentiate between time-varying covariate versus nontime-varying covariate in growth mixture models/latent class growth analysis in r?

I am attempting to run a LCGA / GMM analysis in R but I am not sure how to control for time-varying covariates. I understand that R only has the capability to do the 1-step approach. Does this mean ...
user avatar
  • 1
0 votes
1 answer
85 views

R mix function from DepmixS4 package provide different results with exactly same codes

I am testing the mix model from the Depmix4 package using simulating data. In the model, I provide the starting values to all parameters to be estimated. However, when I run the same code twice, I get ...
user avatar
2 votes
0 answers
59 views

Approximate a distribution as mixture of $K$ other (known, fixed) distributions

I'd like to draw samples from some "target" probability density function $f(x)$. However, I don't have a way to do that -- instead I just have access to $N$ samples, each drawn from one of $...
user avatar
  • 121
3 votes
2 answers
108 views

Sum of random variables that follow a finite normal mixture distribution

Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as $f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
user avatar
  • 1,183
1 vote
1 answer
86 views

Does variational inference solve the label switching problem in Bayesian mixture estimation?

This paper (p. 1751) claims that variational methods do not suffer the label switching problem inherent to Bayesian estimation of mixture models. However, I struggle to find additional references and ...
user avatar
  • 388
1 vote
1 answer
103 views

Goodness of fit for mixture model? [closed]

I have a problem with my vector, I thought that it was a mixture of 2 skew T and I intent to use the ks.test: ...
user avatar
1 vote
0 answers
48 views

Log-likelihood of a finite mixture distribution (PDF overflowing)

I'm trying to use a finite mixture of Dirichlet distributions in a project, but am encountering problems with the PDF becoming so large for input values close to 0 that it overflows to infinity (as ...
user avatar
2 votes
0 answers
28 views

Mixture regression

I am wondering how to analyze and interpret something like the data shown in the figure below (the color is the log-density of the data points, which are a few hundred thousands in number). My ...
user avatar
  • 1,580
0 votes
0 answers
90 views

Accurately estimating the parameters for mixture of geometric distributions

Say we have an i.i.d. sample from a mixture of Geometric distributions: $$ \begin{cases} Geo(p_1) &w.p. \pi_1\\ Geo(p_2) &w.p. 1- \pi_1 \end{cases} $$ Call the parameter set ${\theta}=(\pi_1,...
user avatar
  • 206
0 votes
1 answer
62 views

Regression or mutual information when we have mix of discrete and continuous variables

I am trying to identify relevant features for a problem. The features are discrete and continuous in [0,1]. The target variable is [0,1]. I have tried linear regression by standardizing(subtract mean ...
user avatar
  • 1
2 votes
0 answers
82 views

Choosing the Dirichlet prior in a mixture model

Consider the following mixture model with $K < \infty$ components, $$ f\left(x \mid \theta_{1}, \ldots, \theta_{K}, \pi_{1}, \ldots, \pi_{K}\right)=\sum_{k=1}^K \pi_{k} \varphi\left(x \mid \theta_{...
user avatar
  • 388
3 votes
1 answer
25 views

Compute likelihood of mixture distribution while avoiding floating point problems

I have a mixture distribution with likelihood function $$ L(\theta) = \prod_{i=1}^N \sum_{k=1}^K f(X_i|\theta_k) \lambda_k $$ where $N$ is the sample size, $K$ is the number of component, $\theta_k$ ...
user avatar
  • 653
0 votes
1 answer
53 views

Testing a sample is drawn from a mixture

Suppose you have data $(Y_1, \dots, Y_N)$ drawn from a finite mixture population with $K$ components and you estimate the model parameters $(\theta_1,\dots, \theta_K)$ and the mixture weights $(\phi_1,...
user avatar
  • 388
2 votes
1 answer
106 views

Finite Binomial mixture model

I have a finite Binomial mixture model coded up in stan as below: ...
user avatar
2 votes
2 answers
129 views

Estimating weights of known component distributions in a mixture distribution

Given $n$ probability density functions ($p_1$, ..., $p_n$) with known distributions, what are the ways of estimating the weights ($w_1$, ..., $w_n$) of these component distributions given a sample ...
user avatar
2 votes
0 answers
94 views

nominal regression, kinda. What is the right terminology for this

tl;dr This looks cool, relates possible to discrete regression, but I don't know the term for what it is to do this. I want to learn more. It looks interesting and useful. Can you point me to ...
user avatar
  • 8,280
1 vote
0 answers
94 views

Mixture model using mixtools package [closed]

I am trying to learn mixture models, here is the paper for the mixtools package. I tried to implement the univariate normal mixture model by myself. Below is the code: ...
user avatar
  • 4,527
1 vote
1 answer
203 views

Defining the overlapping area of two log-normal distributions with different means, same variance, and different scaling factors that add up to 1

Define $$ \begin{cases} X_1\sim Lognormal(ln(\mu_1), \sigma^2) \\ X_2\sim Lognormal(ln(\mu_2), \sigma^2) \end{cases} $$ where $\mu_2>\mu_1>0$ and that there is a definite proportion, $\eta\in(0,...
user avatar
4 votes
1 answer
418 views

Finite Beta mixture model in stan -- mixture components not identified

I'm trying to model data $0 < Y_i < 1$ with a finite mixture of Beta components. To do this, I've adapted the code given in section 5.3 of the Stan manual. Instead of (log)normal priors, I am ...
user avatar
2 votes
0 answers
37 views

How to fit a statistical distribution data to a mixure of normal and lognormal distributions

I have a data set that is a mixture of one normal and one lognormal distribution. How can I fit the data to get the fit parameters? Can somebody help me with this.
user avatar
  • 21
3 votes
1 answer
48 views

Reversible Jump for normal mixtures in R^d

I'm reading the article "Multivariate mixtures of normals with unknown number of components" (Dellaportas and Papageorgiou 2006). In this article they describe in great details how to ...
user avatar
  • 425
0 votes
1 answer
24 views

What is (are) scenerios and practical settings that can possibly lead to the weibull-log-Logistic mixture distribution?

In my paper I studied Weibull-loglogistic mixture distributions in reliability and life testing, some structural properties of the model are presented including moments, reliability, hazard rate ...
user avatar
0 votes
0 answers
29 views

How to make inference on cluster-specific parameters in a Bayesian mixture model

Suppose I have a mixture model, for example of the kind $$ y_i \mid w, \{\theta_h\}, H \sim \sum_{h=1}^H w_h f(y_i \mid \theta_h) \\ P(H=h) = q_h \\ w \mid H \sim Dirichlet(\alpha) \\ \theta_1, \ldots,...
user avatar
  • 425
0 votes
0 answers
230 views

Mixture model for decomposing bimodal or multimodal distributions

The gaussian mixture model (GMM) is fed mixture components or features whose time series each have differing means and variances from one another, but are unimodal (have one mode) with each component ...
user avatar
  • 3,047
6 votes
1 answer
506 views

Are Neural Networks Mixture Models?

To my understanding, Gaussian Mixture models are a set of parameterized gaussian distributions that collectively describe an entire, aggregate distribution. ^ from McGonagle et al Also to my ...
user avatar
  • 252
0 votes
1 answer
122 views

M step EM algorithm in Mixture Models. Expected value of the indicator variable under the posterior [closed]

I am not able to solve the following expectation. In the EM algorithm, the first step in the M step is to compute the expected value of $\log p(x,z)$ where $x$ are observations and $z$ indicator ...
user avatar
  • 97