Skip to main content

Questions tagged [mixture-distribution]

A mixture distribution is one that is written as a convex combination of other distributions. Use the "compound-distributions" tag for "concatenations" of distributions (where a parameter of a distribution is itself a random variable).

Filter by
Sorted by
Tagged with
0 votes
1 answer
39 views

Example of nonidentification mixture

Consider a continuous r.v. $X$ with pdf $f$ obeying the following finite mixture model for each $x\in \mathbb{R}$: $$ f(x)=\sum_{k=1}^K \lambda_k f_k(x) \quad \lambda_k\geq 0, \sum_k\lambda_k=1 $$ ...
Star's user avatar
  • 889
3 votes
2 answers
198 views

Mixture distributions: an intuition on why we cannot infer the number of mixture components by visual inspection

I am studying mixture models and I would like your help with this question: Consider the distribution $\Gamma$ and assume it is a finite mixture distribution, i.e., $\Gamma=\sum_{k=1}^K \Gamma_k \...
Star's user avatar
  • 889
0 votes
0 answers
11 views

Continuous mixture distribution

I am currently working on the derivation of the negative binomial distribution as a result of a continuous mixture of Poisson and Gamma distribution for regression purposes. To obtain the entire ...
Marlon Brando's user avatar
0 votes
0 answers
67 views

Posterior distribution of shape & rate parameter in Poisson-Gamma Mixture

Currently I'm struggling to handle the following question. Suppose $x_i,(i=1,2,\dots,n)$ follows Poisson distribution: $$p(x_i|\theta) = \frac{\theta^{x_i}e^{-\theta}}{x_i!}, \quad x_i\in\mathbb N,\...
jason 1's user avatar
  • 311
1 vote
1 answer
44 views

Minimum entropy decomposition of probability distributions

Say you want to decompose a probability distribution (a PDF) into a mixture of distributions in such a way as to minimize the mean entropy of the component distributions. I have an idea that this is ...
zonofzin's user avatar
2 votes
1 answer
61 views

Estimating an unknown distribution from a mixture

I have two data sets, $\{x_i\}$ and $\{y_i\}$. I know that data set $\{x_i\}$ was sampled from some distribution $X$, and that data set $\{y_i\}$ is sampled from a mixture of the $X$, and some other ...
DBruwel's user avatar
  • 43
2 votes
1 answer
49 views

linear Combination of Normal and T-Distributions

Consider the following probability distribution function (PDF): \begin{equation} p(x) = a\mathcal{N}(x; \mu, \sigma^2) + b \mathcal{T}(x; \mu, \tau^2, v) \; \; st. \; \;a + b = 1 \end{equation} $p(x)$ ...
Snowy Baboon's user avatar
0 votes
0 answers
13 views

Online mixture inference; better alternatives than windowed EM?

I have an online Gaussian mixture estimation problem that I would appreciate some input on. To be more precise, I have a stream of scalar observations $x_1, x_2, \dotsc$ arriving over time which are ...
ummg's user avatar
  • 145
8 votes
3 answers
284 views

Sampling from $P(x) \propto \cosh^{m}(a x) e^{-x^{2}/2}$

Is there an efficient algorithm to draw samples $x \sim P(x)$ from the PDF: $$ P(x) \propto \cosh^{m}(a x) e^{-x^{2}/2} $$ where $a\ge0$ is a real parameter, and $m$ a positive integer? Since this is ...
a06e's user avatar
  • 4,440
3 votes
0 answers
20 views

Feature importance in expectation maximization

The context is using EM algorithm for a mixture model - more precisely Dirichlet Multinomial Mixture, as discussed in Dirichlet Multinomial Mixtures: Generative Models for Microbial Metagenomics. One ...
Roger V.'s user avatar
  • 4,429
0 votes
1 answer
50 views

EM algorithm for mixture with latent regression?

I have in the past implemented the EM algorithm for certain cases of mixture distributions. However, I'm attempting to implement it now for a given problem that's exposing my lack of understanding of ...
statsplease's user avatar
  • 2,842
1 vote
2 answers
88 views

Posterior of binomial and mixed prior

I'm currently studying posterior distribution with likelihood $y|\theta \sim B(n,\theta)$ and mixture of prior distribution $\theta \sim \pi Beta(\alpha_1, \beta_1) + (1-\pi)Beta(\alpha_2, \beta_2)$. ...
jason 1's user avatar
  • 311
1 vote
1 answer
124 views

Generate marginally dependent (with predetermined covariance) but conditionally independent data from a Mixture of Gaussians

Suppose you have three variables $y\in\{0,1\}$ and $x_1\in\mathbb{R}$ and $x_2\in\mathbb{R}$. I want to produce data with the following generative process which corresponds to a Mixture of Gaussians (...
Sergio's user avatar
  • 336
0 votes
0 answers
15 views

Calculating Mean Vector and Covariance Matrix of Mixture of Multivariate Normal Distributions [duplicate]

In an effort to better understand multivariate normal distributions, I am attempting to derive the mean vector and covariance matrix of multivariate random vector defined by a mixture distribution. ...
YessuhYessuhYessuh's user avatar
0 votes
0 answers
12 views

Combining factors, represented as normal distributions, to one combined factor, normally distributed

I'm trying to combine the different factors that may affect running pace, such as GPS-measured distance, grade, terrain, heat and other factors (such as wind etc.). Each factor is represented as a ...
Daniel Westergren's user avatar
0 votes
0 answers
84 views

How do we obtain the posterior of a beta binomial mixture of continuous and a discrete density?

In section 3.6 of Jim Albert's 2009 book "Bayesian Computation with R" he describes a test of whether a coin is fair using a mixture of priors. The coin tossing follows a binomial ...
peter2108's user avatar
  • 133
0 votes
2 answers
74 views

Flexmix maxima are not where they are expected to be

For my dataset I have plotted the density with ggplot. As the data's density is multimodal (a total of 6 destinct modi) I tried to gain insight on the normal distributions associated to each modus. ...
Lukas's user avatar
  • 111
5 votes
2 answers
1k views

Kolmognorov Smirnov test p values is 0

I am trying to use Kolmogorov-Smirnov test to check the goodness of fit of the distributions for the dataset. I have dataset consisting of 100,000 samples and I apply expectation-maximization ...
amitha's user avatar
  • 117
0 votes
0 answers
45 views

Log likelihood of mixture distributions

If I have a gamma mixture distribution represented as, $$f(x;\alpha,\beta) = \sum_{i=1}^{N}\pi_if(x;\alpha_i,\beta_i)$$ Where $\pi$ represents the weights of the $N$ components. For $N=2$ and the ...
amitha's user avatar
  • 117
1 vote
1 answer
300 views

Understanding gamma mixture model

I am trying to understand the gamma mixture models, especially the significance of the 'loc' parameter in scipy.stats. In the code below, I generate a mixture ...
amitha's user avatar
  • 117
1 vote
0 answers
37 views

Bayesian conjugate updating when the likelihood can be approximated by a finite mixture of normals?

I'm facing a situation where I'd like to do Bayesian conjugate updating, but both the prior and the likelihood (a Student-t) can only be approximated by a finite mixture of normals. I know that a ...
Björn's user avatar
  • 33.5k
0 votes
0 answers
191 views

Bayesian Gaussian mixture - is my prior correct?

I'd like to sample from the Bayesian Posterior of a Gaussian mixture model, but I am not sure about the correct Bayesian formulation of the latter. Is the following correct? I consider the 1-...
reloh100's user avatar
3 votes
1 answer
163 views

Modeling outliers in maximum likelihood estimation with gradient descent

Consider a set of 3D points $X = \{x_1, x_2, ...x_n\} $ with $ x_i\in\mathbb{R}^3$ on which we want to fit an arbitrary probability distribution. The distribution we want to fit models some ...
Daniel López's user avatar
0 votes
0 answers
51 views

Is every piecewise distribution a mixture distribution?

If I have a piecewise distribution with density $f$ on $[0,1)$ so that $f=f_1$ on $[0,0.5)$ and $f=f_2$ on $[0.5,1)$, with both $f_1$ and $f_2$ densities, is it necessarily a mixture distribution? My ...
giorgi nguyen's user avatar
0 votes
0 answers
13 views

What assumptions are made about categorical variables in regression models? [duplicate]

This is the formalization for a continuous regression model as I understand it: Assume that your outcome $Y$ is normally distributed. Assume that your predictors, $X_1, X_2, \cdots X_n$ are normally ...
djpassey's user avatar
0 votes
1 answer
31 views

Efficiently calculating comparisons between finite mixtures of Gaussians

Say I have two finite mixtures, each consisting of an equally weighted mix of $n$ Gaussians each with a known mean and standard deviation. Is there an efficient method of calculating the probability ...
TLW's user avatar
  • 313
0 votes
0 answers
38 views

Bayes estimate of mixture of exponential under the the Kullback-Leibler divergence loss function

Model framework: Suppose that the loss function is given by the Kullback-Leibler divergence (KLD) as follows: \begin{equation} \text{KL}(\Theta \parallel \hat{\Theta}) = \text{KL}\big(f(x;\Theta) \...
Statistics 's user avatar
2 votes
1 answer
297 views

When two distributions overlap, how to separate one distribution from the mixture distribution if I the other distribution is known?

I have a data which can be classified into two groups. As you see, Figure(a) shows that they are easily classified into group A and B. However, sometimes they are overlapped and it is impossible set a ...
Yong Hwan Kim's user avatar
0 votes
1 answer
69 views

Maximum likelihood estimate for mixture with components using both cartesian and polar coordinates

I have a set of points (x,y) that were generated from a mixture of two components: one component uses Cartesian coordinates, and the other polar coordinates. For example, with probability $\gamma$ I ...
student's user avatar
  • 15
0 votes
0 answers
86 views

Approximaion with uniform mixture density

Assume that a RV is drawn from a distribution with PDF $f(x)$. I would like to approximate this distribution as a mixture of infinitely many uniform distributions. Without loss of generality, assume ...
Ali Furkan's user avatar
1 vote
0 answers
80 views

Are there any fast algorithm to estimate a Bernoulli mixture model?

In a problem I need to estimate a Bernoulli mixture model with 3 mixing components. More specifically, we have a random vector $\mathbf{D}=(D_1,D_2,D_3,D_4,D_5)$. $D_1,D_2,D_3,D_4,D_5$ are drawn ...
ExcitedSnail's user avatar
  • 2,966
0 votes
0 answers
112 views

Is every Compound Poisson distribution a Mixture model?

We have two models: Let $N \sim \hbox Poisson (\lambda)$ and let $(X_k ; k =1,2,3,...)$ be a a sequence of independent and identically distributed random objects (random variables, vectors or ...
Fam's user avatar
  • 1,007
0 votes
0 answers
119 views

Extracting distributions from R density function

I am looking for a not terribly complicated solution to deconvolute a series of overlapping distributions. The data may be noisy and I am considering using R's density function. ...
user avatar
1 vote
1 answer
270 views

Maximum likelihood estimate for mixture of different distributions

I'd like to estimate the parameters of a mixture model using MLE. The density is: $$ f(x,y) = \mathcal{N}(x, y; \boldsymbol{\mu}, \Sigma) \cdot \alpha + \mathcal{N}(x; \mu, \sigma^2) \cdot \mathcal{U}...
student's user avatar
  • 15
1 vote
1 answer
146 views

Generating random variables from a mixture of Normal distributions and Exponential distribution using composition method

How can I sample from a mixture distribution in particular a mixture of Normal distributions and Exponential distribution in R using composition method? For instance if I want to sample from: $0.3\...
Tegig's user avatar
  • 39
0 votes
0 answers
132 views

Model Comparison within Bayesian Gaussian Mixture Model framework

Suppose that we conduct a simulation study, and the model that generated the data is the following Gaussian Mixture Model. $$f(x)=\pi_{1}N(x;\mu_{1},\sigma_{1}^{2})+\pi_{2}N(x;\mu_{2},\sigma_{2}^{2})+\...
Fiodor1234's user avatar
  • 2,286
2 votes
1 answer
73 views

MLE for a mixture of betas without using EM algorithm

Suppose I have a mixture of two Beta densities say $f_1 = \text{Beta}(1,1)$ and $f_2= \text{Beta}(1,\beta)$ where $\beta$ is unknown. The sample $X_1,....,X_n$ is observed based on latent Bernoulli ...
Maths Freak's user avatar
1 vote
1 answer
52 views

What is the skewness of the difference between two binomial random variables?

I have two binomial random variables: $$ X_1 \sim \text{Binom}(n_1,p_1), \\[6pt] X_2 \sim \text{Binom}(n_2,p_2). $$ I know that the individual skewnesses are: $$ \mathbb{Skew}(X_1) = \frac{1 - 2p_1}{\...
janosdivenyi's user avatar
0 votes
1 answer
15 views

How to test whether one cohort is a mixture of two other cohorts for a set of binary variables?

I'll illustrate my question with an example. Suppose I have 25 observations for 3 binary variables. The observations belong to 3 pre-labelled cohorts 1, 2, and 3. Let's say: ...
mrz123456's user avatar
0 votes
1 answer
1k views

Is the mixture of two Gaussians with same mean also Gaussian? [duplicate]

In my problem, both random variables have zero mean, are univariate, and are independent. They may have different variances. If they happen to have the same variance, of course the mixture is Gaussian ...
Vincent Granville's user avatar
0 votes
0 answers
59 views

Mixture distribution mean and variance

I'm currently studying zero-modified exponential distributions and working with neither continuous nor discrete distributions, specifically the following general case: $$ F_{p,m}(x) = \begin{cases} 0, ...
neo's user avatar
  • 13
1 vote
0 answers
28 views

What is the best way to calculate the fit of the mixture distribution to the actual data?

I have fitted the mixture of Gaussians to the natural log of the data. I know that the model is not a very good fit to the data in the tail region, however in the high density region the actual data ...
Karina's user avatar
  • 85
1 vote
1 answer
164 views

Is there a way to fit two distributions from exponential family and overcome discontinuity in parameters?

I have data that follows LogNormal distribution in the body (high density region) of the distribution and it seems like it has an Exponential tail after a particular 'cut' point. Given that these two ...
Karina's user avatar
  • 85
0 votes
0 answers
37 views

Probability of mixture distribution

Let \begin{equation} p_1=\Pr(0.8z_1^2+0.2z_2^2 > c), \qquad p_2=\Pr(0.2z_1^2+0.2z_2^2 > c), \end{equation} where $z_1,z_2 \sim N(0,1)$. Then, can I have $p_1 >p_2$ since $0.8z_1^2+0.2z_2^2 &...
user0131's user avatar
  • 357
0 votes
0 answers
57 views

How to perform calculations when the integral of a mixed exponential distribution pdf does not give 1?

Let Y be the time, for a mixture distribution with two exponential distributions, each multiplied by a and b and having 2 different parameters. How can calculations such as mean, variance and ...
tcengel's user avatar
  • 33
3 votes
1 answer
236 views

Maximum likelihood for mixture of Bernoullis with known mixture proportions

Given the hierarchical model $$ \begin{align} k & \sim \text{Categorical}(\pi_1, \dots, \pi_K) \\ X \mid k & \sim \text{Bernoulli}(\theta_{k}) \end{align} $$ and an i.i.d. sample $X_1, \dots, ...
tddevlin's user avatar
  • 3,387
1 vote
1 answer
111 views

Understanding conditional probability formulas in the context of class-conditionals in generative models

I am trying to understand the theory behind probabilistic generative models a bit better. If I model the class-conditionals as Gaussians, the formula is this: $$ \frac{1}{2\pi^{\frac{D}{2}}|\Sigma|^...
user3813234's user avatar
2 votes
1 answer
96 views

Are discrete mixtures Gauss quadrature-like integral approximations?

I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution. Let $p_{comp}(x)$ be the PDF of a compound ...
ForceBru's user avatar
  • 330
2 votes
0 answers
56 views

Bayesian inference when distribution depends on unobserved outcome with known distribution

Let's say we have an observed outcome $Y_i$ for an object $i=1,\ldots,I$ that arises like this: For each object a coin is tossed (outcome $X_i$ = $H$ or $T$). We know the coin is fair, so $X_i \sim \...
Björn's user avatar
  • 33.5k
1 vote
0 answers
167 views

Calculating Critical Value for Mixture of Two Chi-Square Distributions

Let's say I have a statistic X which is distributed as a mixture of two chi-square distributions; $$X \sim 0.5\chi^2_1 + 0.5\chi^2_2.$$ I'm wondering how I can calculate the critical value(and p-value)...
Pame's user avatar
  • 331

1
2 3 4 5
10