Questions tagged [mixture]

A mixture distribution is one that is written as a convex combination of other distributions.

131 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
8
votes
0answers
355 views

Help me understand the Bayesian kernel density estimation (Sibisi and Skilling, 1996)

Sibisi and Skilling (1996, also mentioned in the 1997 paper) define Bayesian kernel density as $$ f(x) = \int dx' \,\phi(x')\, K(x, x') \tag{2} $$ Here the kernel $K$ is an assigned smooth ...
8
votes
2answers
3k views

How to predict state probabilities or states for new data with DepmixS4 package, for Hidden Markov Models

It seems like I can learn the parameters just fine and find the posterior probabilities for the training data but I have no clue on how to make new predictions on new data. The problem in particular ...
6
votes
0answers
2k views

Label Switching in WinBugs/JAGS

I am using JAGS to estimate a Dirichlet Process Mixture of Normals. The code works well and the estimated density is accurate. However, I would like to know which component each observation is ...
5
votes
0answers
329 views

Relationship of Poisson regression to determining proportions in a mixture distribution

I am a particle physicist, and a very frequent task is: given data sampled from a distribution mixture $$ Z \sim \sum_i \phi_i F_i, $$ where $\phi_i$ are the mixture proportion / prior probabilities ...
5
votes
0answers
45 views

Estimate fraction of a known distribution in a mixture with unknown second distribution

Suppose I have a set of bulbs, which are known to be healthy. For each bulb I have a value of its brightness. The underlying distribution is not necessarily normal, and possibly have some complex ...
4
votes
0answers
101 views

Sampling from mixture of *unnormalized* densities

Suppose I have $n$ unnormalized densities $g_1(\textbf{x}), \ldots, g_n (\textbf{x})$, for $\textbf{x} \in \mathbb{R}^d$, and $n \gg 1$, which largely overlap but in a nontrivial way. I need to sample ...
4
votes
0answers
583 views

Expectation and variance of sample mean with random sample size

I have a question regarding sampling where the sample size itself is a random variable. Say I have two sub-populations $A$ and $B$ from which I can sample a real valued random variable with ...
4
votes
0answers
475 views

How to write unnormalized posterior when prior is a mixture of continuous and discrete

Suppose I want to do bayesian inference on the regression problem $\beta$ for Y = X$\beta$ + $\epsilon$ for $\epsilon_i \sim N(0,\sigma^2)$. The complication is that the prior for each component $\...
4
votes
0answers
3k views

Ftting a mixture of two Gaussians

I want to fit a mixture of two gaussian densities to my financial data. The data can be found here: http://uploadeasy.net/upload/2a7mw.rar the variable is called dat. The probability density of a ...
4
votes
0answers
244 views

Analysis hierarchical circular mixture data

I have circular data such that multiple human participants were, each shown a color from a color wheel, asked to remember it for a "retention interval", then report it back by clicking a color wheel. ...
3
votes
0answers
56 views

Concentration inequality for mean of Gaussian mixture

Say I have i.i.d. samples $X_1, \ldots, X_n \sim p \mathcal{N}(\mu_1, \sigma^2) + (1 - p) \mathcal{N}(\mu_2, \sigma^2)$. Then suppose I estimate the mean with the sample mean $$ \widehat{\mu} = \frac{...
3
votes
0answers
65 views

Distribution of difference of Gaussian mixtures: symmetric wrto zero?

I have the following 3-variate random vector $(X,Y,Z)$ which is distributed as a Gaussian mixture: (with some abuse of notation) $$ f(X,Y,Z)=\underbrace{w_a \mathcal{N}(\mu_a, \Sigma_a)}_{\text{...
3
votes
0answers
265 views

Question about E step in Em algorithm, a challenge part, any help please?

I am new to EM algorithm and copula. I was reading a paper in mixture pair-copula. The authors use $u=(u_r, u_s) = (u_r^t,u_s^t), (t= 1,...,T)$ to indictae to the vector of copula data. Then, they ...
3
votes
0answers
149 views

Sum of truncated Gammas and degenerate

I have a variable $X$ which I am modelling with a mixture model: $$\begin{aligned} (X|A) &\sim \mathbb{1}_{0 \leq x < w \cdot m} \cdot \frac{\text{Gamma}(\alpha,0,\beta / m)}{k_1} \\ (X|B) &...
3
votes
0answers
663 views

A mixture of conjugate priors is conjugate

I want to prove that a mixture of conjugate priors is itself conjugate. It does not look difficult, but I'm still a bit unsure when manipulating probabilities, especially in a Bayesian context. Is ...
3
votes
0answers
318 views

parameter estimation of a mixture distribution

I have this mixture distribution $f(x) =w \cdot \mathcal{LN}(\mu_1,\sigma) + (1-w)\cdot \mathcal{LN}(\mu_2,\sigma) $ where $\mathcal{LN}(\mu,\sigma)$ is a lognormal distribution. I now have $j$ ...
3
votes
0answers
338 views

Dirichlet process mixture model with Bayesian hierarchical clustering

I am doing Bayesian hierarchical clustering. From my understanding, there are three basic points for this algorithm. Use marginal likelihoods to decide which clusters to merge Asks what the ...
3
votes
0answers
85 views

Estimating parameters of inifinite scale mixture from data

Suppose that I have an infinite scale mixture of zero-mean normal distributions, whose mixing distribution is gamma with parameters $\alpha$ and $\beta$. The data is thus distributed according to a ...
3
votes
0answers
56 views

Compendium or catalog of compound distributions?

Does anyone know of a good compendium or catalog of compound distributions, or finite mixture representations of those distributions? I am trying to find out to what extent the common multi-...
3
votes
0answers
1k views

Mixture of binomial distributions

I am experimenting with a mixture of binomial models. Consider a binary variable $y_i$. Furthermore, there are two sub-groups in the population (not known a priori and not observable): $z_i=0$ or $z_i=...
3
votes
0answers
637 views

Calculating the Fisher Information of bivariate normal

I'm lost. If I estimated the a Gaussian mixture model, with a shared diagonal covariance, will the Fisher information of the means be $\Sigma^{-1}$ ?
2
votes
0answers
56 views

Can the likelihood ratio estimate multivariate confidence levels?

Wilks' theorem describes the log-ratio between the highest likelihood of a distribution $\mathcal{L}$ (aka the dominant mode, given at $\vec{x}_{m}$) and the likelihood of a distribution at a given ...
2
votes
0answers
43 views

Understanding short animation about Dirichlet Process Mixture Model

On the wikipedia page of Dirichlet Process, there is the following video. I don't get the point of the video. My first impression was that the video was showing the fitting of one-dimensional data ...
2
votes
0answers
36 views

Multivariate mixture models

I am new to mixture modeling and have successfully used bernoulli mixture models to cluster datasets of binary data. My real purpose, though is to cluster datasets with mixed data types: normal, ...
2
votes
0answers
67 views

Variance of “an observation” and problem 5.9.8 in BDA by Gelman, et. al

I would like to understand problem 5.9.8 in Bayesian Data Analysis by Gelman, et al.. In particular the problem asks: ... create a bimodal prior density for a normal mean, that is thought to be ...
2
votes
0answers
425 views

loglikelihood decrease very slightly in EM algorithm

I am working with a very large and complicated function. I am using EM algorithm to estimate the model parameters. The EM works very well. However, after 27 iteration I see that the values of the ...
2
votes
0answers
26 views

Is there a relationship between RKHS and mixtures?

Reproducing kernel Hilbert spaces (RKHS) (see here or here) seem to involve conic combinations of "kernel functions" (my understanding is very crude). (See pp. 34-35 of Behrends, unfortunately the ...
2
votes
0answers
624 views

Programming a mixture of a Gamma with a Normal distribution using R

I have some data x in R which seems to be a mixture of a Gamma and Normal distribution. Therefore I'd like to model this as a mixture model consisting of said distributions, but I don't know how to ...
2
votes
0answers
551 views

Can a Gaussian mixture model be specified using a regression equation?

From: https://stats.stackexchange.com/a/236297/22199, I quote A mixture distribution combines different component distributions with weights that typically sum to one (or can be renormalized). A ...
2
votes
0answers
48 views

Efficient mixing of probability estimators

At each time-step $t$ we are given two probability estimators $p_0(t)$ and $p_1(t)$. We output a predicted probability $p(t)$ that we will next observe $1$, and then receive an outcome $y_{t+1} \in \{...
2
votes
0answers
247 views

Gibbs sampling version for estimating Hierarchical Double Dirichlet Process Mixture of Gaussian Processes

I'm trying to implement Gibbs sampling to estimate the parameters of the following non-parametric model: $$\begin{align*} \beta|\gamma & \sim \text{GEM}(\gamma)\\ k_t|\beta & \sim \beta\\ \pi|\...
2
votes
0answers
332 views

Interpretation of Generalized Inverse Gaussian regression with GAMLSS

Background on my project: I am comparing proteins (nodes) between a network representing protein interactions in metastatic patients v/s another network representing protein interactions in non-...
2
votes
0answers
167 views

Prediction intervals for mixture models for time series forecasting - is it really an average of the prediction intervals of the averaged models?

I'm trying to find out how to do forecasting with a mixture model (averaging the forecasts of an ets, an arima and an ...
2
votes
0answers
108 views

Separating mixture with little a priori knowledge

I have a dataset (time series data) of measured signal power values from a radio receiver. The data does not originate from a controlled experiment. I have limited knowledge of the underlying ...
2
votes
0answers
280 views

when is an estimator consistent?

Say there are parameters $\theta$ such that $\theta_i > 0$ and $\sum_i \theta_i = 1$ and a model such as $p(x) = \sum_{i=1}^n \theta_i p_i(x)$ where $p_i(x)$ are fixed and defined over a domain of ...
2
votes
0answers
316 views

Orthogonality of Hermite Polynomials

As we know probabilistic Hermite polynomials are orthogonal with respect to the weight function $\frac{1}{\sqrt{2 \pi}} e^{-x^2/2}$ (density of standard normal). I have a distribution which is a ...
2
votes
0answers
411 views

Merge covariance matrices

I have two 2x2 covariance matrices, stemming from bivariate datasets that are approximately normally distributed. I want to create a mixture distribution and for that I need to merge the covariance ...
2
votes
0answers
343 views

Simple problem formulation for EM algorithm

I understand the EM algorithm, I understand for example how we get $Q(\theta, \theta^t)$, but I have trouble translating a real-world problem into the EM framework. For example, I'm given this ...
2
votes
0answers
3k views

Quick and simple cluster analyses for univariate data

Can you suggest any quick and simple clustering analyses, for univariate real-valued data? In other words, I have $n$ real numbers, $x_1,\dots,x_n$ where $x_i \in \mathbb{R}^+$, and I want to cluster ...
2
votes
1answer
471 views

weights in a mixture Gaussian model

If the variance of a random variable is proportional to its mean, then what is the best way of making a mixture distribution that will faithfully reconstruct a data set coming from a mixture model. ...
1
vote
0answers
9 views

Number of events in a segment if waiting times are drawn from a mixture of two exponential distributions

What is the probability for $n$ events to occur over a period of time $t$, if the duration of each individual event is $\tau_1$ with the probability $p$ and $\tau_2$ with the probability of $(1-p)$? ...
1
vote
1answer
79 views

Calculation of AIC in finite mixture modeling

I have a question about calculation the AIC to find my optimal amount of clusters. I am applying mixture modeling with the EM algorithm. I know the formula AIC = -2ln(log-lik) + 2k. These are my log-...
1
vote
0answers
56 views

Modelling a random variable that is mostly zero, but otherwise exponential (PyMC3)

I'm new to probabilistic programming, and have run into problems of this kind a few times now. Simply put: I often find myself wanting to model a random variable that mostly has some nice, continuous ...
1
vote
0answers
21 views

Variance analysis for state dependent mixture importance sampling estimators

Let $(E,\mathcal E,\lambda)$ be a measure space $k\in\mathbb N$ $q_i:E\to[0,\infty)$ be $\mathcal E$-measurable with $$\int q_i\:{\rm d}\lambda=1$$ and $\nu_i:=q_i\lambda$ for $i\in\{1,\ldots,k\}$ $...
1
vote
0answers
28 views

Closed form posterior for product of inverse Gamma and Normal distribution

I am currently reading a book about mixture analysis, and in the textbook a posterior for the parameters $\mu1,\mu2,\sigma_1^2,\sigma_2^2$ of a two-component gaussian mixture is derived as follows (S ...
1
vote
0answers
6 views

Ideas for fixing manual misclassification

Imagine the following problem. Mechanics fix cars and use a set of parts which is recorded on invoices. They also write down a classification of the work that was done from a fixed set of possible ...
1
vote
0answers
26 views

A 2 component mixture is symmetric if and only if $\lambda\in \{0,1,\frac{1}{2}\}$

Consider the following mixture of two densities $$ f(x)=\lambda g(x-\mu_1)+(1-\lambda)g(x-\mu_2) $$ with $\lambda\in [0,1]$, $g(\cdot)$ symmetric around zero, $\mu_1<\mu_2$. Claim: the mixture is ...
1
vote
0answers
213 views

How to implement Exponential Power distribution in JAGS

I would like to fit a simulated data to Exponential Power likelihood using uniform mixture with gamma mixing presented in "Scale Mixtures Distributions In Statistical Modelling" by Choy and Chan: $EP(...
1
vote
0answers
24 views

Does mixture of sigmoids make sense given the theories about mixture of bernoullis?

Mixture of bernoullis is the combination of bernoulli distributions, which can be illustrated by the sampling process of K bags of D coins, here is a quick tutorial about it https://cedar.buffalo.edu/~...
1
vote
0answers
76 views

Why always AIC and BIC are used in mixture model than Vuong test

I am working with mixture models. I fitted more than one model to the data and then try to select the most appropriate model using different selection criteria, for example, AIC. My supervisors asked ...