Questions tagged [mixture]

A mixture distribution is one that is written as a convex combination of other distributions.

Filter by
Sorted by
Tagged with
9
votes
4answers
922 views

Maximum likelihood function for mixed type distribution

In general we maximize a function $$ L(\theta; x_1, \ldots, x_n) = \prod_{i=1}^n f(x_i \mid \theta) $$ where $f$ is probability density function if the underlying distribution is continuous, and a ...
3
votes
2answers
218 views

Are we ignoring implications by de Finetti's theorem on regression?

De Finetti's theorem states that, if observations $(x_1, x_2, x_3, \cdots)$ are infinitely exchangeable, then their joint probability $p(x_1, x_2, \cdots, x_N)$ has a representation as a mixture: $$p(...
1
vote
0answers
258 views

Detecting distribution peaks and their significance

I have (a lot of) datasets with points having 1d distributions like these: Note, that the data is periodic in nature, like time of a day, so left and right sides of the plots above correspond to the ...
0
votes
1answer
168 views

Finite mixture model from empirical distributions

I'm working with experimental data. Participants were assigned to one of three conditions and made a single choice (A, B, C, or D). So the data look something like this, where the values are counts (...
0
votes
0answers
160 views

Mixture Proposal Distributions

I have a target distribution $\mu$ which I would like to investigate using, for instance Metropolis-Hastings-Green (MHG). So, given a Gaussian prior, $\pi$, and a likelihood $L$ such that $\mu(dx) \...
2
votes
0answers
553 views

Can a Gaussian mixture model be specified using a regression equation?

From: https://stats.stackexchange.com/a/236297/22199, I quote A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A ...
4
votes
0answers
101 views

Sampling from mixture of *unnormalized* densities

Suppose I have $n$ unnormalized densities $g_1(\textbf{x}), \ldots, g_n (\textbf{x})$, for $\textbf{x} \in \mathbb{R}^d$, and $n \gg 1$, which largely overlap but in a nontrivial way. I need to sample ...
0
votes
1answer
25 views

Sum of N four-component mixture variates

I asked a similar question with two-component mixture variates, and I was wondering how it extends to a four-component mixture variate. In other words, I have a list of random variables, $X_1$, $X_2$, ...
8
votes
2answers
4k views

What is “mixture” in a gaussian mixture model

We often study Gaussian Mixture model as a useful model in machine learning and its applications. What is the physical significance of this "Mixture"? Is it used because a Gaussian Mixture Model ...
2
votes
1answer
76 views

Sum of N two-component mixture variates

I have a list of random variables, $X_1$, $X_2$, ..., $X_N$, associated with binary random variables $A_i$ such that $P(A_i) = \pi$ is known. I also know that, for all $i$ $$X_i|A_i\sim f(x)\\ X_i|\...
4
votes
2answers
3k views

How to derive the MLE of a Gaussian mixture distribution

In my self-study, I consider a Gaussian mixture distribution: $$p(x)= p(k=1) N(x|\mu_1,\sigma^2_1) + p(k=0) N(x|\mu_0,\sigma^2_0)$$ where $p(k=1)+p(k=0)=\pi_1+\pi_0=1$. I am now asked to do three ...
3
votes
0answers
149 views

Sum of truncated Gammas and degenerate

I have a variable $X$ which I am modelling with a mixture model: $$\begin{aligned} (X|A) &\sim \mathbb{1}_{0 \leq x < w \cdot m} \cdot \frac{\text{Gamma}(\alpha,0,\beta / m)}{k_1} \\ (X|B) &...
2
votes
0answers
48 views

Efficient mixing of probability estimators

At each time-step $t$ we are given two probability estimators $p_0(t)$ and $p_1(t)$. We output a predicted probability $p(t)$ that we will next observe $1$, and then receive an outcome $y_{t+1} \in \{...
1
vote
0answers
220 views

How can I validate a Gaussian Mixture Model?

I am currently working with a dataset composed by 14 continuous variables and 100 observations. My goal is to select variables of interest to build consistent clusters and doing classification. I was ...
2
votes
1answer
538 views

Scale and shape parameters of Gamma mixture distributions

I used the function mix (package mixdist) to fit Gamma mixture distributions. The function gives mu and sigma parameters (output ...
3
votes
0answers
663 views

A mixture of conjugate priors is conjugate

I want to prove that a mixture of conjugate priors is itself conjugate. It does not look difficult, but I'm still a bit unsure when manipulating probabilities, especially in a Bayesian context. Is ...
8
votes
2answers
3k views

How to predict state probabilities or states for new data with DepmixS4 package, for Hidden Markov Models

It seems like I can learn the parameters just fine and find the posterior probabilities for the training data but I have no clue on how to make new predictions on new data. The problem in particular ...
1
vote
1answer
165 views

Mixture Model Distributions

I wonder, if there could be a Pareto Mixture Model, just like the Gaussian Mixture Model (GMM). How am I supposed to build a Pareto Mixture Model (PMM)?
1
vote
1answer
4k views

sampling from a mixture of two Gamma distributions

Assuming that all the mixture parameters are known, how can one sample from a mixture of $\texttt{Gamma}(\alpha,\beta)$ distributions: $$\theta \sim \pi \texttt{Gamma}(\alpha_1,\beta_1)+(1-\pi)\texttt{...
4
votes
1answer
1k views

Difference between a mixture of distributions and a convolution. Interpretation in a applied setting

From what I could gather Mixture: if $X_i\sim^{iid} f_i$, then W is a mixture with $f_W =\sum \frac{f_i}{n}$. This definition could also be for the CDF instead of the density. Convolution: To make it ...
1
vote
0answers
66 views

Finding a Scalable Approach to a modified coin-flip problem using MLE model

and thank you in advance for your help! I am interested in applying MLE to estimate parameters in a "modified" coin flip model, but have been having difficulties scaling the solution. The problem ...
2
votes
1answer
620 views

Defining overlapping periods

I have a dataset containing the abundance of migrating bird species. In the figure below you can see that there are two "bell" shapes that are overlapping somewhere around September. One of the bell ...
4
votes
0answers
584 views

Expectation and variance of sample mean with random sample size

I have a question regarding sampling where the sample size itself is a random variable. Say I have two sub-populations $A$ and $B$ from which I can sample a real valued random variable with ...
1
vote
1answer
208 views

Building a Regression Tree with one Gaussian Mixture Model at each node

I am trying to build a regression tree that outputs both a mean and a covariance matrix for each leaf of the tree. Ideally I would be able to have a Gaussian Mixture Model at each leaf. A first ...
3
votes
1answer
5k views

Advantages and disadvantages of EM algorithm vs trust region methods for nonlinear optimization

I have a set of observations X that I believe were generated by a mixture of several probability distributions (specifically, two von mises and one uniform). I'd like to find the maximum likelihood ...
1
vote
0answers
45 views

variance of a mixture in terms of the mean and variance of each component

I am following the MATLAB example to fit a mixture of two normal distribution that you can find here At some point it is defined the inital guess for the standard deviation as: ...
2
votes
1answer
668 views

Looking for proof of conditional dependence, when the conditioning variables are linearly related

Suppose we have three random variables, $X$, $Y_1$, and $e$ (for error). Variable $e$ is independent of $X$ and $Y_1$, but $X$ and $Y_1$ are dependent. Further suppose we construct a new mixture ...
1
vote
1answer
281 views

Probability distribution on a subset of a simplex

I want to define a probability distribution on a subset of a simplex. for example, on a 3-simplex, we know that $x_1+x_2+x_3+x_4=1$ and $X \sim Dirichlet$. Would it be possible to constraint $X$ more (...
1
vote
0answers
24 views

Combining monthly distributions of data with different right-censoring levels for prediction

I have a number of pricing datasets with integer values (prices in round dollars) for the same product which are right-censored at different levels (one level per dataset). I would like to combine ...
2
votes
1answer
729 views

How to combine two beta-binomial distributions

Say I have the following situation. I have two weighted coins: Coin 1: In the past I've seen this coin flipped 10 times, 8 of which it came up heads. So I can model the probability of $n$ heads out ...
1
vote
1answer
63 views

Models for nonnegative (incl. zero) positively skewed multivariate time series (trade volumes)

I want to build a Monte Carlo simulation that is based in part on share amounts that are traded in the market for a set of stocks. I need to be able to take into account the co-dependence of trade ...
0
votes
0answers
144 views

Tail dependence of mixture copulas

I am currently using (multivariate) mixture copulas to model a financial data set. The mixture has two components as follows: $$C_{mixture}=wC_1+(1-w)C_2$$ where $C_1$ and $C_2$ are copulas. I have ...
0
votes
1answer
488 views

solution to mixture CDF / inverse CDF of finite mixture

I am currently numerically solving the follwing equation, which is a convex combination/finite mixture of two marginal CDFs $F(x)$ and $G(x)$ (actually they are joint CDFs but all arguments but x are ...
4
votes
1answer
503 views

Estimate weighted variances in mixture models

Given a generic mixture model $X$ of $k$ components, with distribution $$ f(x)=∑_i\pi_if_i(x), $$ It is easy to show that the $k-th$ moment is just the weighted mean of the $k-th$ moments of the ...
2
votes
1answer
477 views

weights in a mixture Gaussian model

If the variance of a random variable is proportional to its mean, then what is the best way of making a mixture distribution that will faithfully reconstruct a data set coming from a mixture model. ...
0
votes
0answers
217 views

mixture of Gaussians vs mixture of quadratic denominators (Cauchy)

It is known that mixture of Gaussians are dense in the set of all distribution functions. A 1-dimensional Gaussian has the following density: $$ \frac{1}{\sqrt{2\pi \sigma^2}} e^{-\frac{(\omega-\beta)^...
2
votes
1answer
2k views

How can I create a topic model with a mixture of multinomials and EM?

I'm trying to create a topic model with a mixture of multinomials and the EM algorithm. I do not want to use a package. For reference, I'm implementing this in Python with numpy. Data Sets I have ...
0
votes
0answers
347 views

Is it possible to convert a Gaussian Mixture Model implementation into a Categorical Mixture Model?

I am modelling whether a customer will spend when given a voucher. I have a theory that a customer falls into one of two latent classes: call them spendthrift and miser. So I would like to fit a ...
1
vote
0answers
263 views

Mixture Density Networks — Recovering Conditionals

If we've got a network trained to generate a mixture of Gaussians and we want the conditional distributions based on a set/subset of outputs, can we easily recover these? Intuitively, yes, we should ...
3
votes
2answers
653 views

What is the difference between a mixture model and a multimodal distribution?

A distribution that is a "Mixture model" has a very similar definition as a "multimodal" distribution. Wikipedia Says: a multimodal distribution is a continuous probability distribution with two ...
1
vote
1answer
142 views

Fitting mixture distributions and probability estimation

I am working on continuous data set with ranges 0-1. I need to group them using mixed models (based on prior clinical/biological basis). From this model, I need to get the p-value for a value (say 0....
1
vote
1answer
422 views

Collapsed Gibbs Sampling in Mixture Models

I tried to learn how Gibbs sampling works on Mixture models by studying David Blei's notes: http://www.cs.columbia.edu/~blei/fogm/2015F/notes/mixtures-and-gibbs.pdf In the equation 28: $p(z_i = k| ...
0
votes
0answers
28 views

Composition of Normals — Finite mixture with some known parameters

I.e., the data was generated from 5 normal distributions: ...
2
votes
1answer
741 views

What is the difference between a mixture model and a hierarchical model?

What is the difference between mixture and hierarchical models? Are they of the same nature with different names or they are totally different things? If there are any references, I will be happy to ...
2
votes
1answer
465 views

EM and Kullback-Leibler divergence

Let $f$ be a density on $\mathbb{R}^{p}$. Let $f_{\theta} = \sum_{i=1}^{d} \alpha_{i}\mathcal{N}_{p}(\cdot \, ; \, \theta_{i})$ be a mixture of $d$ Gaussian distributions on $\mathbb{R}^{p}$. For each ...
16
votes
2answers
2k views

“ all of these data points come from the same distribution.” How to test?

I feel like I've seen this topic discussed here before, but I wasn't able to find anything specific. Then again, I'm also not really sure what to search for. I have a one dimensional set of ordered ...
4
votes
0answers
475 views

How to write unnormalized posterior when prior is a mixture of continuous and discrete

Suppose I want to do bayesian inference on the regression problem $\beta$ for Y = X$\beta$ + $\epsilon$ for $\epsilon_i \sim N(0,\sigma^2)$. The complication is that the prior for each component $\...
8
votes
1answer
1k views

Example of computing the expectation of a discrete RV using Riemann-Stieltjes integral?

Riemann-Stieltjes integral notation is used in expectation expressions in some probability texts. Basically, dF(x) pops up in the integral rather than f(x)dx in the integral, since the CDF F(x) may ...
6
votes
2answers
1k views

Gibbs sampler gets stuck in local mode

I am very new to statistics and trying to implement a Gibbs sampler. However, according to wikipedia https://en.wikipedia.org/wiki/Gibbs_sampling and this discussion thread http://metaoptimize.com/qa/...
2
votes
1answer
115 views

Efficiently sampling from mixture distribution posterior

I have the following model: $$ \begin{align} \pi_1\sim & \text{Unif}(0,1)\\ \lambda_1,\lambda_2\sim & \text{Ga}(1,1)\\ z_i\sim & \pi_1^{1(z_i=1)}\pi_2^{1(z_i=2)}\\ p(y_i|\lambda_1,\...

1 2 3
4
5
8