Questions tagged [mixture-distribution]

A mixture distribution is one that is written as a convex combination of other distributions. Use the "compound-distributions" tag for "concatenations" of distributions (where a parameter of a distribution is itself a random variable).

Filter by
Sorted by
Tagged with
1 vote
0 answers
32 views

Do Mixture Models "Defy" Entropy?

Recently, I have learned about the principle of Maximum Entropy with regards to Probability Distribution (https://www.youtube.com/watch?v=2gTrsLVnp9c) - in particular, when certain "information&...
user avatar
  • 5,688
1 vote
0 answers
36 views

Entropy of Gaussian mixture when variance of one component gets larger

I want to prove or disprove the following relation of differential entropies: Conjecture: $\displaystyle h(f) \le h(g)$ where $f, g$ the density functions of Gaussian mixture models with equal ...
user avatar
  • 1,099
2 votes
0 answers
25 views

High dimensional behavior of Dirichlet Process-based clustering?

I have a problem stemming from Dirichlet Process Gaussian Mixture Models (DP-GMMs) in high dimension. I'll write this question so that no knowledge of DP-GMMs is needed. Let $D$ be the dimensionality ...
user avatar
0 votes
0 answers
20 views

How to identify a mixture of poisson distribution and Gaussian distribution from the data?

Here is the distribution of the data. It seeme to me that it is a mixture of a poisson distribution at the begining of zero value and a Gaussian distribution. I also used the ...
user avatar
1 vote
1 answer
97 views

Fitting truncated normal mixtures in R

I have a vector x, lower_bound < x < upper_bound. I would like to fit a truncated normal mixture distribution to ...
user avatar
  • 189
2 votes
1 answer
150 views

Distribution of sum of $n$ random variables with mixture of two exponential distributions

Suppose that the random variable $Y$ follows a mixture of two exponential distributions, that is \begin{equation} f_Y(y) = \sum_{i=1}^{2}\pi_i f(y| \lambda_i) \end{equation} where $\pi$ stands for ...
user avatar
0 votes
0 answers
19 views

The lower bound of K-L divergence of a mixture

I'm wondering if there is a lower bound for a mixture when each single component K-L divergence in the mixture is lower bounded by some constants. Let $$D(p||q)=\int p(x)\log \frac{p(x)}{q(x)}dx$$ If $...
user avatar
0 votes
0 answers
39 views

Posterior Expected Loss and Bayes Risk of a mixture model

I am trying to find the Posterior Expected Loss and Bayes Risk of a Bernoulli mixture model. Here is the setup: Given θi, 0 < θi < 1, a sequence of independent Bernoulli (θi) random variables ...
user avatar
  • 23
0 votes
0 answers
20 views

Fisher Information of Weight in Mixture distribution

Let's assume $x$ follows a mixture of two arbitrary continuous probability distributions with probability density functions $p_1(x)$ and $p_2(x)$, respectively. The probability density function of $x$ ...
user avatar
  • 762
1 vote
1 answer
36 views

Mixture probability depends on the sample

A mixture of two distributions has density which is the weighted sum of the components: $$f_{mix}(x) = p f_{1}(x) + (1-p) f_{2}(x).$$ What if the mixture weight is allowed to vary with the sample ...
user avatar
0 votes
0 answers
37 views

Estimator for quantile of kernel of mixture distribution

I have a mixture distribution such as $$ X_t = \sum_{i} w^i_t X^i_t , \quad \text{with } w^i_t \in \{0, 1\}$$ and $\sum_{i} w^i_t = 1$. where $X^i_t$ are i.i.d , $\forall t$. I call the $X^i$ the ...
user avatar
  • 1
2 votes
1 answer
96 views

Simultaneous Bayes Estimation

Given $\theta_i$, $0 < \theta_i < 1$, a sequence of independent Bernoulli ($\theta_i$) random variables from i subpopulations, that are also independent across subpopulations. Suppose i=2 (2 ...
user avatar
  • 23
9 votes
1 answer
167 views

Marginalizing out discrete response variables in Stan

There's been quite a bit of discussion and confusion about how to marginalize out discrete response variables in Stan (e.g. binary or ordinal data). See, for instance: Impute binary outcome variable ...
user avatar
  • 135
0 votes
0 answers
37 views

Benefits of Expectation Maximization for Mixture Models

What are the benefits of using expectation maximization for mixture models vs. direct maximization of the marginal likelihoods? Analytic maximization step In case of Gaussian mixtures the benefit is ...
user avatar
  • 1,578
2 votes
0 answers
42 views

Can the EM algorithm be used to perform model selection?

Suppose that the samples $\mathcal{D} = \{x_1,x_2,\dots,x_N\}$ were independently sampled from a Gaussian distribution with mean $\mu$ and variance $\sigma^2$. However, suppose that we did not know ...
user avatar
  • 2,916
1 vote
1 answer
43 views

What is a mixture of RNNs?

I am reading papers on different types of classification and prediction methods and keep coming across "Mixture of Recurrent Neural Networks" and "Mixture of Markov Chain Models". ...
user avatar
1 vote
1 answer
356 views

How can I derive the EM algorithm for a mixture of two Bernoulli distributions?

How can I derive the E-step and M-step in the EM algorithm for a mixture of two Bernoulli distributions? Note that I am aware that there are several notes online that explain how to do this for the ...
user avatar
  • 2,916
5 votes
1 answer
121 views

Name of a distribution similar to the exponential

for a simulation I'm using the continuous distribution $$F(x)=1-(1+x)e^{-cx} $$ for $x\geq 0$ with $c\geq 1$. Do you know if this distribution has a name?
user avatar
  • 51
1 vote
1 answer
48 views

Mixture of a realization of uniform variable and noise

Suppose that $X \sim U[0,1]$. After $X = x$ has realized, we don't observe $x$, but we instead observe a noisy signal of $x$, defined as $S = \tau x + (1 - \tau) U$, where $\tau \sim Ber(p)$ and $U \...
user avatar
1 vote
0 answers
21 views

What determines performance in recoverying K in Gaussian Mixture Model?

My question is about what determines how hard it is to recover the number of components $K$ in a Gaussian mixture model (GMM), e.g. with the EM-algorithm. For simplicity, let's consider the case in ...
user avatar
  • 653
3 votes
1 answer
34 views

Probabilistic interpretation of sum of quantile functions

We know that the weighted sum of CDF $$ F(x) = w_1 F_1(x) + w_2 F_2(x), \,\, w_1 + w_2 = 1 $$ is the CDF of the mixture distribution. Is there a probabilistic interpretation for weighted sum of ...
user avatar
  • 33
0 votes
0 answers
23 views

Mixtures of Dirichlet multivariates or Dirichlet processes

I am exploring the properties of Dirichlet distributions and their parameters. When mixing two Dirichlet distributed random bivariates $$\mathbf{X}\equiv(X_1,X_2)\sim\text{Dir}(\alpha_1,\alpha_2)$$ ...
user avatar
0 votes
0 answers
12 views

Sample size necessary to detect that a mixture distribution is an appropriate model

I am working on an assignment, but having some trouble understanding how I would answer the following: "Comment on the shape of the histogram as the sample size increases. Include a description ...
user avatar
  • 131
0 votes
0 answers
27 views

Mixture random variable density

I have $X = B_1X_1 + B_2X_2 + ... + B_nX_n$ where the $X_i$ are independent random variables and the $B_i$ are independent $Bern(p_i)$ such that $\sum_{i=1}^{n} p_i =1$. I want to find the PDF and CDF ...
user avatar
  • 1
0 votes
1 answer
34 views

Make a scatter plot by drawing 100 items from a mixture distribution [closed]

I was asked the following: Make a scatter plot by drawing 100 items from a mixture distribution $0.3 N\left((5, 0)^{T}, \begin{pmatrix} 1 & 0.25 \\ 0.25 & 1\\ \end{pmatrix}\right) +0.7 N\...
user avatar
  • 121
0 votes
0 answers
26 views

How to analyse this bimodal gamma distribution?

I have collected physiological data with multiple observations from 35 people, across four conditions. In planning the experiment, I had been hoping to perform inferential statistics comparing between ...
user avatar
0 votes
0 answers
53 views

mixture of exponential and gamma distribution

I'm not particularly good at statistics and whatever elementary statistics I have had exposure to are now rusty. However, I am working on a problem that I am hoping to gain some insights into: My goal ...
user avatar
  • 207
0 votes
0 answers
12 views

Are the subsets of Mixture Models necessarily parametrized?

Im just learning about mixture models and they are described as a "mix of parametric and non parametric models". My question is how a non-parametric subset would look like? How would e.g. EM-...
user avatar
  • 1
1 vote
1 answer
80 views

Expected log likelihood for mixture components with differing support

I was hoping to use the EM algorithm to fit a mixture model in which the mixture components can have differing support. I've run into a problem during the M step because the expected log-likelihood ...
user avatar
  • 13
0 votes
0 answers
27 views

Mixture of normal and truncated normal variables

Imagine the following process: a gambler bets $\$1$ and receives and outcome $X_1$, with $X_i \sim \mathcal{N}(\mu,1), i=1,2$, in period 1. If the realization is positive, he stops playing, and his ...
user avatar
3 votes
0 answers
233 views

Resampling classes across weighted source distributions

I am sure this is a common problem, but googling only yielded false positives. I probably did not know what terms to search for. So here we go: I have $n$ classes from $m$ different sources. Each ...
user avatar
1 vote
1 answer
60 views

Correlation and mixture

Suppose we have a mixture distribution (same parametric family) and we know that a third variable $X$ is correlated to each of those variables in the mixture with coefficients $\rho_1$, $\rho_2$, ...
user avatar
  • 13
0 votes
1 answer
19 views

Computation of Posterior Mixture Weights

My question concerns the below example, where the author analyzes rainfall occurrences via a first order Markov chain. The transition probabilities are such that $p_{11} + p_{12} = 1$ and $p_{21} + p_{...
user avatar
0 votes
0 answers
25 views

Why isn't this term considered an expectation in EM algorithm?

The log-likelihood $\mathcal{L}$ for a mixture model can be written as: $$ \mathcal{L} = \log p(\boldsymbol X| \Delta, \boldsymbol \pi) = \sum_{n=1}^N \log \color{blue}{\sum_{k=1}^K \pi_k p(x_n|\...
user avatar
  • 113
0 votes
0 answers
38 views

Discretized mixture distribution

I'm reading the article "Two Modeling Strategies for Empirical Bayes Estimation" by Bradley Efron, and I'm having some difficulty recreating one of the plots in it. We are given a discrete ...
user avatar
1 vote
1 answer
28 views

Can we "reject" that a distribution is a finite mixture of normals?

Consider a one-dimensional distribution function $f(x)$. Suppose this distribution has all the nice properties, such as continuity, smoothness, etc. We observe $f(x)$. Suppose that we believe that $f(...
user avatar
  • 266
0 votes
0 answers
34 views

Calculating $p$ for mixture of Bernoulli distributions

I was reading this from University of Buffalo Mixture of Bernoulli lecture slides. So is the new $p_k$ or as they denote it $\mu_k$ for Bernoulli dist $k$ just the mean of all the points multiplied ...
user avatar
1 vote
0 answers
57 views

How to define the space of the probability distribution that weighted sum of two independent random variables distributes as

Define a linear space $X=\Delta(Z)$. $Z=\mathbb R_+$ is the real valued outcome space and $\Delta$ is a probability simplex. Let $f$ and $g$ denote the elements in $X$. Suppose now I define a mixture $...
user avatar
  • 359
1 vote
1 answer
82 views

Generating random variable from no closed-form marginal density [closed]

Suppose $u\sim N(0,I_p)$ and $Y|U\sim N(x(t),\sigma_e^2I_m)$, and the marginal distribution of $y$ is $f(y)=\int_u f(y|u)f(u)du$. $x(t)$ is composite function of $u$, basically $x(t)$ is a function of ...
user avatar
  • 19
1 vote
0 answers
27 views

Generating random variable from mixture representation [duplicate]

Suppose $u\sim N(0,I_p)$ and $Y|U\sim N(x(t),\sigma_e^2I_m)$, and the marginal distribution of $y$ is $f(y)=\int f(y|u)f(u)du$. $x(t)$ is composite function of $u$. The problem is I need to generate ...
user avatar
  • 19
0 votes
0 answers
14 views

Difference in computation between covariance matrix and single covariances in mixtures

I have a mixture distribution of k elements and I would like to compute the $COV(X_i, X_j)$ but I have some doubts. I have found the formula of the expectation and the variance on Wikipedia. I would ...
user avatar
  • 57
5 votes
2 answers
559 views

Metropolis Hastings algorithm bivariate normals

I need some help implementing the (1) independence Gaussian proposal and (2) random walk Gaussian proposal to simulate from a mixture bivariate normal distribution. "If we have a continuous state ...
user avatar
  • 428
6 votes
2 answers
332 views

Negative Binomial as Gamma-Poisson Mixture or Compound Logarithmic Poisson: can this correspondence be generalized to other distributions?

Preamble A random variable $X$ with a negative binomial distribution can be characterized in three ways: [Negative Binomial] $X\sim\operatorname{NegBin}(r,p)$ for some $r$ and $p$; [Gamma-Poisson ...
user avatar
2 votes
0 answers
59 views

Approximate a distribution as mixture of $K$ other (known, fixed) distributions

I'd like to draw samples from some "target" probability density function $f(x)$. However, I don't have a way to do that -- instead I just have access to $N$ samples, each drawn from one of $...
user avatar
  • 121
3 votes
2 answers
107 views

Sum of random variables that follow a finite normal mixture distribution

Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as $f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
user avatar
  • 1,183
1 vote
1 answer
102 views

Goodness of fit for mixture model? [closed]

I have a problem with my vector, I thought that it was a mixture of 2 skew T and I intent to use the ks.test: ...
user avatar
1 vote
0 answers
48 views

Log-likelihood of a finite mixture distribution (PDF overflowing)

I'm trying to use a finite mixture of Dirichlet distributions in a project, but am encountering problems with the PDF becoming so large for input values close to 0 that it overflows to infinity (as ...
user avatar
2 votes
0 answers
28 views

Mixture regression

I am wondering how to analyze and interpret something like the data shown in the figure below (the color is the log-density of the data points, which are a few hundred thousands in number). My ...
user avatar
  • 1,578
3 votes
1 answer
40 views

Difference of independent random variables that is not unimodal

This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
user avatar
  • 516
1 vote
1 answer
996 views

How to evaluate the loss on a Gaussian Mixture Model?

I successfully modeled my data using a Gaussian Mixture Model in scikit-learn but I can't figure out how I should say "how good" the model is by calculating the loss. My first thought was to ...
user avatar
  • 1,280

1
2 3 4 5
10