Questions tagged [mixture]

A mixture distribution is one that is written as a convex combination of other distributions.

Filter by
Sorted by
Tagged with
1
vote
1answer
214 views

Why is variational Bayesian mixture model an alternative to MCMC? What are the similarities?

Why do people say that a variational Bayesian mixture model could be an alternative to MCMC for clustering? For example see the details here: https://en.wikipedia.org/wiki/Variational_Bayesian_method. ...
0
votes
2answers
84 views

Deriving a distribution whose pdf has the shape of a square + a triangle (a right trapezoid)

I want to the derive the PDF which looks like the sum of a triangular and uniform distribution which looks like this: To do this I have simply added the PDFs for the rectangular and triangular parts, ...
1
vote
0answers
214 views

How to implement Exponential Power distribution in JAGS

I would like to fit a simulated data to Exponential Power likelihood using uniform mixture with gamma mixing presented in "Scale Mixtures Distributions In Statistical Modelling" by Choy and Chan: $EP(...
1
vote
1answer
224 views

Why does not the weighted sum of gamma distribution come from weighted gamma variables?

If $Z\sim 0.3\Gamma(\alpha _1,\beta _1)+0.7\Gamma (\alpha _2,\beta_2)$, why isn't $Z=0.3X_1+0.7X_2$? $X_1\sim\Gamma(\alpha _1,\beta _1)$ and $X_2\sim\Gamma(\alpha _2,\beta _2)$?
1
vote
0answers
24 views

Does mixture of sigmoids make sense given the theories about mixture of bernoullis?

Mixture of bernoullis is the combination of bernoulli distributions, which can be illustrated by the sampling process of K bags of D coins, here is a quick tutorial about it https://cedar.buffalo.edu/~...
2
votes
1answer
108 views

Maximum likelihood estimator for a mixture of 2 distributions

Let $X_1, ..., X_n$ be iid with one of two PDFs. If $\theta = 0$, then $f(x; \theta) = 1, \ 0 < x < 1$. if $\theta = 1$, then $f(x; \theta) = \frac{1}{2\sqrt{x}}, \ 0 < x < 1$. What ...
1
vote
1answer
136 views

Derive the joint probability density function of differences of Gaussian Mixtures

Consider a 3-variate random vector $(\epsilon_0, \epsilon_1, \epsilon_2)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(\epsilon_0, \epsilon_1, \epsilon_2)=\underbrace{...
3
votes
0answers
66 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
0
votes
0answers
36 views

What is the degree of freedom of semiparametric method for mixture distribution

In the semi-parametric method for density analysis, I want to compare one component semi-parametric mixture distribution and two components mixture distribution. Semi-parametric here means the shape ...
1
vote
0answers
76 views

Why always AIC and BIC are used in mixture model than Vuong test

I am working with mixture models. I fitted more than one model to the data and then try to select the most appropriate model using different selection criteria, for example, AIC. My supervisors asked ...
2
votes
1answer
43 views

How to specify uniform distribution with same properties as normal distribution?

What I mean is, is it possible to specify a uniform random variable $U$ with random parameters $a,b$, where $a=-b$, and are generated from some other distribution, such that the marginal pdf of $U(a,b)...
2
votes
1answer
66 views

Parameterizing finite mixture distribution

Let's consider a finite mixture: $$f(x) = \sum_{i=1}^{N}w_{i}p_{i}\left(x\right)$$ where: $N$ is the number of mixed distributions $\left\{p_{1},\dots, p_{N}\right\}$ is a finite set of one-...
0
votes
0answers
80 views

How to estimate Breslow type Baseline Hazard for mixture cure model?

I have done the following bit myself: ...
0
votes
0answers
283 views

EM algorithm and AIC criteria

I am using EM algorithm to estimate the model parameters. EM-algorithm iterates until the loglikelihood is converged. After that, I need to compute AIC criteria. As known, AIC is a loglikelihood ...
4
votes
2answers
693 views

Question about the latent variable in EM algorithm

In mixture models, Expectation maximization algorithm (EM) is a commonly used method to estimate the model parameters. Suppose that I have bivariate mixture model with two mixture components, with ...
2
votes
1answer
65 views

Mixing of liquids where concentration follows normal distribution

We have several liquids where the concentration of a certain element follows a normal distribution, and we take a weighted combination of the elements. The concentrations are: $$C_i \sim \text{IID N}...
2
votes
1answer
389 views

What is a positively skewed distribution that can include zero?

I'm modelling data from a behavioural task. Participants do a few hundred trials. On each trial, they see a sequence of letters at a point on the screen and one of these letters appears surrounded by ...
2
votes
0answers
36 views

Multivariate mixture models

I am new to mixture modeling and have successfully used bernoulli mixture models to cluster datasets of binary data. My real purpose, though is to cluster datasets with mixed data types: normal, ...
3
votes
2answers
2k views

Simulating a bimodal distribution in the range of [1;5] in R

I want to simulate a continuous data set/variable with lower/upper bounds of [1;5], while at the same time ensure that the drawn distribution can be considered as bimodal. Searching for my problem, I ...
2
votes
1answer
324 views

Convert a normal to a mixture of two normal distribution with variance equal to that of the normal

Consider a variable, say x following the normal distribution N(0,5.99). I want to translate this into a symmetric mixture of two normal distributions such that the mean of x is zero and the variance ...
1
vote
0answers
134 views

How to fit a (gaussian) mixture model to a dataset with (right) censored data in R?

I am trying to fit a mixture distribution to a dataset in R. Exploiting the R package mixtools, this goes pretty well. However, up to 20% of the data point in the dataset are right censored, therefore ...
3
votes
2answers
263 views

Sample space for mixture of two Normal distributions

I am going through the book Introduction to Probability by Joseph Blitzstein, Jessica Hwang, and I found the following problem on mixture of two Normal distributions: A certain stock has low ...
1
vote
0answers
42 views

Testing mean assumption for mixture distribution

I have claim data structured by age groups and I am trying to test the assumed claim means for each age group against the actual data. It is a mixture distribution where around 75% of the sample ...
1
vote
1answer
980 views

General conditional distributions for multivariate Gaussian mixtures

My question is similar to this one but considers a more general situation. Suppose that $ \vec{x} = (x_1, \dots, x_d) $ and let $$ p(\vec{x}) = \sum_{k=1}^{n} \pi_k \mathcal{N}(\vec{x} | \mu_k, \...
5
votes
2answers
151 views

Can “cross-validation” be used to choose a prior?

To be clear, I doubt I am using the term "cross-validation" correctly here; what I am suggesting also seems similar to "boot-strapping" and "hyperparameter tuning". Terminology is not my strength. ...
2
votes
1answer
374 views

Assigning a scale mixture of Normal distributions for the data

it's well known that the scale mixture of normal distributions is equivalent to a Student t model, that is $$ t_{(v)}(x|\mu,\sigma^2)=\int_0^\infty N(x|\mu,\sigma^2/\lambda)\times G(\lambda|v/2,v/2)d\...
2
votes
0answers
68 views

Variance of “an observation” and problem 5.9.8 in BDA by Gelman, et. al

I would like to understand problem 5.9.8 in Bayesian Data Analysis by Gelman, et al.. In particular the problem asks: ... create a bimodal prior density for a normal mean, that is thought to be ...
0
votes
0answers
52 views

Is multivariate linear regression correct in this case?

I have 20 items rated on 25 different dimensions. These items can be classified in two ways. They belong to Group A or B; also, orthogonally, they belong to Groups W, X, Y or Z. Items were rated by ~ ...
0
votes
1answer
17 views

Why does the mixture dependencies exist between variables?

Sorry if my question is clear to most of you. As a mathematics background, I really just start working with the mixture and would like to understand it in a clear way. Mixture dependencies are when ...
1
vote
0answers
429 views

Gradient Descent vs EM

The EM algorithm is usually motivated because maximizing the log likelihood is described as being "complicated" or "difficult" due to having to take the log of the weighted sum of the likelihood; ie $...
35
votes
10answers
37k views

Why is the sum of two random variables a convolution?

For long time I did not understand why the "sum" of two random variables is their convolution, whereas a mixture density function sum of $f(x)$ and $g(x)$ is $p\,f(x)+(1-p)g(x)$; the arithmetic sum ...
2
votes
1answer
34 views

Two Correct Ways to Sample from a Two Component Mixture but Different Variance Results…What's happening?

Let $X_1 \sim \operatorname{Poisson}(7)$ and $X_2 \sim \operatorname{Poisson}(3)$ be two independent random variables. Let $\{0.7,0.3\}$ be our collection of mixing weights. The following R code is ...
3
votes
1answer
106 views

Finding an estimator for parameter $n$ for a mix of two Binomially distributed populations

I have a PDF defined by: $$Pr(X=k)=p \left({n\choose k}a^k(1-a)^{n-k}\right)+(1-p) \left({n\choose k}b^k(1-b)^{n-k}\right)$$ I am not given any information about the parameters $n, p, a, b$. I am ...
1
vote
1answer
45 views

How can I cluster data drawn from distributions with known symmetries?

Consider a set of data which is a mixture of samples drawn from different distributions. It is known from the underlying phenomena generating the mixture that for every distribution in the mixture ...
1
vote
3answers
744 views

Gaussian Mixture Model (GMM) has a maximum likelihood when two clusters one includes the other

What do people do when you perform GMM to cluster and the result comes to out to be a bimodal distribution. However, one cluster is broad enough that it includes the other cluster and this results in ...
10
votes
2answers
176 views

Exact Sampling from Improper Mixtures

Suppose I want to sample from a continuous distribution $p(x)$. If I have an expression of $p$ in the form $$p(x) = \sum_{i=1}^\infty a_i f_i(x)$$ where $a_i \geqslant 0, \sum_i a_i= 1$, and $f_i$ ...
0
votes
1answer
41 views

Bound on cross-covariance given linear mixture of random vectors

I have a data model whereby random vector $c = a + b$ with unknown distributions and zero means. I can empirically obtain an estimate of $E[cc^T]$ and $E[bb^T]$. For posterity, $E[cc^T] = E[(a+b)(a+b)^...
0
votes
1answer
32 views

Very steep decrease in information criteria for mixture models with more components

I am analyzing data using mixture modeling. When I plot the information criteria (the BIC) for a series of models (with different model specifications and different number of mixture components), I ...
6
votes
1answer
167 views

What is a mixture of finite mixtures?

A mixture of finite mixture models seem to be an interesting Bayesian (?) approach to solving clustering with an unknown $k$ number of components. It seems though, unlike the mixture model with a ...
2
votes
1answer
108 views

Modeling bimodal time-to-event

Here is a plot of death registration frequencies by age for the UK in 1974. I see distributions like this quite often: there is some event (e.g. death) which happens either close to birth, or ...
1
vote
0answers
96 views

Mixture of power-law distribution

I have two large sets A and B of (integer) numbers, both obtained with two different (and unknown) probability distributions $\rho_A$ and $\rho_B$. A (also large) set C contains a proportion $p$ of ...
0
votes
1answer
318 views

Can the plot of the log-likelihood told something about the good starting values in EM

As known, EM algorithm is sensitive to the starting values. One method to select the starting values is to run EM several times using different starting values each time. Then, the select the one that ...
1
vote
1answer
571 views

Can log likelihood funcion be positive [duplicate]

I have a mixture data. I used EM to estimate the model parameters. When I calculate the log likelihood function, I found that the values is positive. So, is that ok. Can the log likelihood function ...
4
votes
1answer
925 views

Number of components for Gaussian mixture model?

I have a vector of numeric values. My hypothesis is that this vector is a mixture drawn from two Gaussian distributions (ie k = 2). However, it is possible that there is only one Gaussian underlying ...
1
vote
0answers
42 views

Can the estimation of parameters in EM be decreases and increases during the estimation process

I am working on EM algorithm (manually implemented in R). I saw that the estimation of the parameters vary from iteration to another. That is, it decreases at some iteration and then start increasing ...
2
votes
0answers
429 views

loglikelihood decrease very slightly in EM algorithm

I am working with a very large and complicated function. I am using EM algorithm to estimate the model parameters. The EM works very well. However, after 27 iteration I see that the values of the ...
0
votes
0answers
114 views

Mixture model with covariance matrix with varying variances and equal covariances

In some illustrations of mixture models, the covariance matrix is structured to have varying variances and equal covariances between mixture components. For example, Pastor et al. (2007) (I'm sorry ...
4
votes
1answer
134 views

What is the distribution of a Poisson variable, where the Poisson rate is Normal (or Binomial)?

What is the distribution of $X$ if $$ X \sim \text{Poisson}(\lambda), \quad \text{where }\lambda \sim N(\mu,\sigma^2)$$ or $$ X \sim \text{Poisson} (\lambda), \quad \text{where }\lambda \sim ...
1
vote
0answers
90 views

How to interpret profiles / mixture components without any observations classified to them (when using MCLUST in R)

I am using the MCLUST software in R to fit normal mixture models, as part of what in my field are commonly called Latent Profile Analysis. Some of the time, ...
1
vote
0answers
965 views

How to model a dependent variable with a U-shaped dependent variable?

Note: I should have flipped x and y in this example to be in line with typical notation. Sorry for any confusion. Consider the ...

1
2
3 4 5
8