Questions tagged [mixture-distribution]

A mixture distribution is one that is written as a convex combination of other distributions. Use the "compound-distributions" tag for "concatenations" of distributions (where a parameter of a distribution is itself a random variable).

Filter by
Sorted by
Tagged with
2 votes
1 answer
38 views

Maximum likelihood for mixture of Bernoullis with known mixture proportions

Given the hierarchical model $$ \begin{align} k & \sim \text{Categorical}(\pi_1, \dots, \pi_K) \\ X \mid k & \sim \text{Bernoulli}(\theta_{k}) \end{align} $$ and an i.i.d. sample $X_1, \dots, ...
  • 3,257
1 vote
1 answer
28 views

Understanding conditional probability formulas in the context of class-conditionals in generative models

I am trying to understand the theory behind probabilistic generative models a bit better. If I model the class-conditionals as Gaussians, the formula is this: $$ \frac{1}{2\pi^{\frac{D}{2}}|\Sigma|^...
1 vote
0 answers
23 views

Are discrete mixtures Gauss quadrature-like integral approximations?

I noticed that the formula for Gauss (or Newton-Cotes) quadrature looks very similar to the formula for the PDF of a general mixture distribution. Let $p_{comp}(x)$ be the PDF of a compound ...
  • 250
1 vote
0 answers
16 views

Bayesian inference when distribution depends on unobserved outcome with known distribution

Let's say we have an observed outcome $Y_i$ for an object $i=1,\ldots,I$ that arises like this: For each object a coin is tossed (outcome $X_i$ = $H$ or $T$). We know the coin is fair, so $X_i \sim \...
  • 24k
0 votes
0 answers
32 views

Calculating Critical Value for Mixture of Two Chi-Square Distributions

Let's say I have a statistic X which is distributed as a mixture of two chi-square distributions; $$X \sim 0.5\chi^2_1 + 0.5\chi^2_2.$$ I'm wondering how I can calculate the critical value(and p-value)...
  • 249
0 votes
1 answer
18 views

Mixtures vs Multi-level models?

I'm confused on how mixture models and multi-level models are different (if at all.) Are there general rules for when to use one and not the other, pros/cons, etc?
  • 1,728
2 votes
0 answers
42 views

E-Step of the EM algorithm on mixture

Im trying to implement the EM algorithm on mixture model: $pg(x) + (1-p)h(x)$ where the sample $\bar{x} = (x_1, \ldots, x_n)$, is independently generated from the mixture, and $g(x) = e^{-x}$ and $h(x)...
  • 119
1 vote
0 answers
63 views

Likelihood in mixture models

As per my understanding, normally, when we talk about Bayes rule, we write: p(z|x) = [p(x|z) * p(z)] / p(x) where, p(z|x) is called posterior p(x|z) is called likelihood p(z) is called prior p(x) is ...
  • 161
2 votes
1 answer
37 views

Extracting statistical parameters from a mixture of two distributions of different kind

I have a dataset b (as a list in Python) of length 100 I know that is amounts to the mixture of two distributions: A normal distribution A uniform distribution ...
  • 81
1 vote
0 answers
40 views

Do Mixture Models "Defy" Entropy?

Recently, I have learned about the principle of Maximum Entropy with regards to Probability Distribution (https://www.youtube.com/watch?v=2gTrsLVnp9c) - in particular, when certain "information&...
  • 6,028
1 vote
0 answers
61 views

Entropy of Gaussian mixture when variance of one component gets larger

I want to prove or disprove the following relation of differential entropies: Conjecture: $\displaystyle h(f) \le h(g)$ where $f, g$ the density functions of Gaussian mixture models with equal ...
  • 1,139
2 votes
0 answers
29 views

High dimensional behavior of Dirichlet Process-based clustering?

I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed. Let $D$ be the dimensionality ...
0 votes
0 answers
63 views

How to identify a mixture of poisson distribution and Gaussian distribution from the data?

Here is the distribution of the data. It seeme to me that it is a mixture of a poisson distribution at the begining of zero value and a Gaussian distribution. I also used the ...
1 vote
1 answer
147 views

Fitting truncated normal mixtures in R

I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
  • 179
2 votes
1 answer
185 views

Distribution of sum of $n$ random variables with mixture of two exponential distributions

Suppose that the random variable $Y$ follows a mixture of two exponential distributions, that is \begin{equation} f_Y(y) = \sum_{i=1}^{2}\pi_i f(y| \lambda_i) \end{equation} where $\pi$ stands for ...
0 votes
0 answers
32 views

The lower bound of K-L divergence of a mixture

I'm wondering if there is a lower bound for a mixture when each single component K-L divergence in the mixture is lower bounded by some constants. Let $$D(p||q)=\int p(x)\log \frac{p(x)}{q(x)}dx$$ If $...
0 votes
0 answers
60 views

Posterior Expected Loss and Bayes Risk of a mixture model

I am trying to find the Posterior Expected Loss and Bayes Risk of a Bernoulli mixture model. Here is the setup: Given θi, 0 < θi < 1, a sequence of independent Bernoulli (θi) random variables ...
  • 23
0 votes
0 answers
28 views

Fisher Information of Weight in Mixture distribution

Let's assume $x$ follows a mixture of two arbitrary continuous probability distributions with probability density functions $p_1(x)$ and $p_2(x)$, respectively. The probability density function of $x$ ...
  • 782
1 vote
1 answer
38 views

Mixture probability depends on the sample

A mixture of two distributions has density which is the weighted sum of the components: $$f_{mix}(x) = p f_{1}(x) + (1-p) f_{2}(x).$$ What if the mixture weight is allowed to vary with the sample ...
0 votes
0 answers
43 views

Estimator for quantile of kernel of mixture distribution

I have a mixture distribution such as $$ X_t = \sum_{i} w^i_t X^i_t , \quad \text{with } w^i_t \in \{0, 1\}$$ and $\sum_{i} w^i_t = 1$. where $X^i_t$ are i.i.d , $\forall t$. I call the $X^i$ the ...
  • 1
2 votes
1 answer
105 views

Simultaneous Bayes Estimation

Given $\theta_i$, $0 < \theta_i < 1$, a sequence of independent Bernoulli ($\theta_i$) random variables from i subpopulations, that are also independent across subpopulations. Suppose i=2 (2 ...
  • 23
9 votes
1 answer
267 views

Marginalizing out discrete response variables in Stan

There's been quite a bit of discussion and confusion about how to marginalize out discrete response variables in Stan (e.g. binary or ordinal data). See, for instance: Impute binary outcome variable ...
  • 125
0 votes
0 answers
69 views

Benefits of Expectation Maximization for Mixture Models

What are the benefits of using expectation maximization for mixture models vs. direct maximization of the marginal likelihoods? Analytic maximization step In case of Gaussian mixtures the benefit is ...
  • 1,660
2 votes
0 answers
61 views

Can the EM algorithm be used to perform model selection?

Suppose that the samples $\mathcal{D} = \{x_1,x_2,\dots,x_N\}$ were independently sampled from a Gaussian distribution with mean $\mu$ and variance $\sigma^2$. However, suppose that we did not know ...
  • 3,159
1 vote
1 answer
52 views

What is a mixture of RNNs?

I am reading papers on different types of classification and prediction methods and keep coming across "Mixture of Recurrent Neural Networks" and "Mixture of Markov Chain Models". ...
2 votes
1 answer
639 views

How can I derive the EM algorithm for a mixture of two Bernoulli distributions?

How can I derive the E-step and M-step in the EM algorithm for a mixture of two Bernoulli distributions? Note that I am aware that there are several notes online that explain how to do this for the ...
  • 3,159
5 votes
1 answer
128 views

Name of a distribution similar to the exponential

for a simulation I'm using the continuous distribution $$F(x)=1-(1+x)e^{-cx} $$ for $x\geq 0$ with $c\geq 1$. Do you know if this distribution has a name?
  • 51
1 vote
1 answer
53 views

Mixture of a realization of uniform variable and noise

Suppose that $X \sim U[0,1]$. After $X = x$ has realized, we don't observe $x$, but we instead observe a noisy signal of $x$, defined as $S = \tau x + (1 - \tau) U$, where $\tau \sim Ber(p)$ and $U \...
  • 155
1 vote
0 answers
24 views

What determines performance in recoverying K in Gaussian Mixture Model?

My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm. For simplicity, let's consider the case in ...
  • 665
3 votes
1 answer
36 views

Probabilistic interpretation of sum of quantile functions

We know that the weighted sum of CDF $$ F(x) = w_1 F_1(x) + w_2 F_2(x), \,\, w_1 + w_2 = 1 $$ is the CDF of the mixture distribution. Is there a probabilistic interpretation for weighted sum of ...
  • 33
0 votes
0 answers
30 views

Mixtures of Dirichlet multivariates or Dirichlet processes

I am exploring the properties of Dirichlet distributions and their parameters. When mixing two Dirichlet distributed random bivariates $$\mathbf{X}\equiv(X_1,X_2)\sim\text{Dir}(\alpha_1,\alpha_2)$$ ...
0 votes
0 answers
12 views

Sample size necessary to detect that a mixture distribution is an appropriate model

I am working on an assignment, but having some trouble understanding how I would answer the following: "Comment on the shape of the histogram as the sample size increases. Include a description ...
  • 131
0 votes
0 answers
28 views

Mixture random variable density

I have $X = B_1X_1 + B_2X_2 + ... + B_nX_n$ where the $X_i$ are independent random variables and the $B_i$ are independent $Bern(p_i)$ such that $\sum_{i=1}^{n} p_i =1$. I want to find the PDF and CDF ...
  • 1
0 votes
1 answer
49 views

Make a scatter plot by drawing 100 items from a mixture distribution [closed]

I was asked the following: Make a scatter plot by drawing 100 items from a mixture distribution $0.3 N\left((5, 0)^{T}, \begin{pmatrix} 1 & 0.25 \\ 0.25 & 1\\ \end{pmatrix}\right) +0.7 N\...
  • 121
0 votes
0 answers
40 views

How to analyse this bimodal gamma distribution?

I have collected physiological data with multiple observations from 35 people, across four conditions. In planning the experiment, I had been hoping to perform inferential statistics comparing between ...
0 votes
0 answers
96 views

mixture of exponential and gamma distribution

I'm not particularly good at statistics and whatever elementary statistics I have had exposure to are now rusty. However, I am working on a problem that I am hoping to gain some insights into: My goal ...
  • 207
1 vote
1 answer
88 views

Expected log likelihood for mixture components with differing support

I was hoping to use the EM algorithm to fit a mixture model in which the mixture components can have differing support. I've run into a problem during the M step because the expected log-likelihood ...
  • 13
3 votes
0 answers
361 views

Resampling classes across weighted source distributions

I am sure this is a common problem, but googling only yielded false positives. I probably did not know what terms to search for. So here we go: I have $n$ classes from $m$ different sources. Each ...
1 vote
1 answer
86 views

Correlation and mixture

Suppose we have a mixture distribution (same parametric family) and we know that a third variable $X$ is correlated to each of those variables in the mixture with coefficients $\rho_1$, $\rho_2$, ...
  • 13
0 votes
1 answer
21 views

Computation of Posterior Mixture Weights

My question concerns the below example, where the author analyzes rainfall occurrences via a first order Markov chain. The transition probabilities are such that $p_{11} + p_{12} = 1$ and $p_{21} + p_{...
0 votes
0 answers
42 views

Discretized mixture distribution

I'm reading the article "Two Modeling Strategies for Empirical Bayes Estimation" by Bradley Efron, and I'm having some difficulty recreating one of the plots in it. We are given a discrete ...
1 vote
1 answer
33 views

Can we "reject" that a distribution is a finite mixture of normals?

Consider a one-dimensional distribution function $f(x)$. Suppose this distribution has all the nice properties, such as continuity, smoothness, etc. We observe $f(x)$. Suppose that we believe that $f(...
  • 421
1 vote
0 answers
79 views

How to define the space of the probability distribution that weighted sum of two independent random variables distributes as

Define a linear space $X=\Delta(Z)$. $Z=\mathbb R_+$ is the real valued outcome space and $\Delta$ is a probability simplex. Let $f$ and $g$ denote the elements in $X$. Suppose now I define a mixture $...
  • 359
1 vote
1 answer
95 views

Generating random variable from no closed-form marginal density [closed]

Suppose $u\sim N(0,I_p)$ and $Y|U\sim N(x(t),\sigma_e^2I_m)$, and the marginal distribution of $y$ is $f(y)=\int_u f(y|u)f(u)du$. $x(t)$ is composite function of $u$, basically $x(t)$ is a function of ...
  • 19
1 vote
0 answers
27 views

Generating random variable from mixture representation [duplicate]

Suppose $u\sim N(0,I_p)$ and $Y|U\sim N(x(t),\sigma_e^2I_m)$, and the marginal distribution of $y$ is $f(y)=\int f(y|u)f(u)du$. $x(t)$ is composite function of $u$. The problem is I need to generate ...
  • 19
5 votes
2 answers
653 views

Metropolis Hastings algorithm bivariate normals

I need some help implementing the (1) independence Gaussian proposal and (2) random walk Gaussian proposal to simulate from a mixture bivariate normal distribution. "If we have a continuous state ...
  • 533
6 votes
2 answers
448 views

Negative Binomial as Gamma-Poisson Mixture or Compound Logarithmic Poisson: can this correspondence be generalized to other distributions?

Preamble A random variable $X$ with a negative binomial distribution can be characterized in three ways: [Negative Binomial] $X\sim\operatorname{NegBin}(r,p)$ for some $r$ and $p$; [Gamma-Poisson ...
2 votes
0 answers
73 views

Approximate a distribution as mixture of $K$ other (known, fixed) distributions

I'd like to draw samples from some "target" probability density function $f(x)$. However, I don't have a way to do that -- instead I just have access to $N$ samples, each drawn from one of $...
  • 121
3 votes
2 answers
129 views

Sum of random variables that follow a finite normal mixture distribution

Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as $f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
  • 1,203
1 vote
1 answer
133 views

Goodness of fit for mixture model? [closed]

I have a problem with my vector, I thought that it was a mixture of 2 skew T and I intent to use the ks.test: ...

1
2 3 4 5
10