Questions tagged [mixture-distribution]

A mixture distribution is one that is written as a convex combination of other distributions. Use the "compound-distributions" tag for "concatenations" of distributions (where a parameter of a distribution is itself a random variable).

Filter by
Sorted by
Tagged with
0
votes
0answers
25 views

mixture of exponential and gamma distribution

I'm not particularly good at statistics and whatever elementary statistics I have had exposure to are now rusty. However, I am working on a problem that I am hoping to gain some insights into: My goal ...
0
votes
0answers
11 views

Are the subsets of Mixture Models necessarily parametrized?

Im just learning about mixture models and they are described as a "mix of parametric and non parametric models". My question is how a non-parametric subset would look like? How would e.g. EM-...
1
vote
1answer
59 views

Expected log likelihood for mixture components with differing support

I was hoping to use the EM algorithm to fit a mixture model in which the mixture components can have differing support. I've run into a problem during the M step because the expected log-likelihood ...
0
votes
0answers
21 views

Mixture of normal and truncated normal variables

Imagine the following process: a gambler bets $\$1$ and receives and outcome $X_1$, with $X_i \sim \mathcal{N}(\mu,1), i=1,2$, in period 1. If the realization is positive, he stops playing, and his ...
3
votes
0answers
175 views

Resampling classes across weighted source distributions

I am sure this is a common problem, but googling only yielded false positives. I probably did not know what terms to search for. So here we go: I have $n$ classes from $m$ different sources. Each ...
1
vote
1answer
49 views

Correlation and mixture

Suppose we have a mixture distribution (same parametric family) and we know that a third variable $X$ is correlated to each of those variables in the mixture with coefficients $\rho_1$, $\rho_2$, ...
0
votes
1answer
16 views

Computation of Posterior Mixture Weights

My question concerns the below example, where the author analyzes rainfall occurrences via a first order Markov chain. The transition probabilities are such that $p_{11} + p_{12} = 1$ and $p_{21} + p_{...
0
votes
0answers
23 views

Why isn't this term considered an expectation in EM algorithm?

The log-likelihood $\mathcal{L}$ for a mixture model can be written as: $$ \mathcal{L} = \log p(\boldsymbol X| \Delta, \boldsymbol \pi) = \sum_{n=1}^N \log \color{blue}{\sum_{k=1}^K \pi_k p(x_n|\...
0
votes
0answers
35 views

Discretized mixture distribution

I'm reading the article "Two Modeling Strategies for Empirical Bayes Estimation" by Bradley Efron, and I'm having some difficulty recreating one of the plots in it. We are given a discrete ...
1
vote
1answer
25 views

Can we "reject" that a distribution is a finite mixture of normals?

Consider a one-dimensional distribution function $f(x)$. Suppose this distribution has all the nice properties, such as continuity, smoothness, etc. We observe $f(x)$. Suppose that we believe that $f(...
0
votes
0answers
16 views

Calculating $p$ for mixture of Bernoulli distributions

I was reading this from University of Buffalo Mixture of Bernoulli lecture slides. So is the new $p_k$ or as they denote it $\mu_k$ for Bernoulli dist $k$ just the mean of all the points multiplied ...
1
vote
0answers
27 views

How to define the space of the probability distribution that weighted sum of two independent random variables distributes as

Define a linear space $X=\Delta(Z)$. $Z=\mathbb R_+$ is the real valued outcome space and $\Delta$ is a probability simplex. Let $f$ and $g$ denote the elements in $X$. Suppose now I define a mixture $...
1
vote
1answer
49 views

Generating random variable from no closed-form marginal density [closed]

Suppose $u\sim N(0,I_p)$ and $Y|U\sim N(x(t),\sigma_e^2I_m)$, and the marginal distribution of $y$ is $f(y)=\int_u f(y|u)f(u)du$. $x(t)$ is composite function of $u$, basically $x(t)$ is a function of ...
1
vote
0answers
25 views

Generating random variable from mixture representation [duplicate]

Suppose $u\sim N(0,I_p)$ and $Y|U\sim N(x(t),\sigma_e^2I_m)$, and the marginal distribution of $y$ is $f(y)=\int f(y|u)f(u)du$. $x(t)$ is composite function of $u$. The problem is I need to generate ...
0
votes
0answers
14 views

Difference in computation between covariance matrix and single covariances in mixtures

I have a mixture distribution of k elements and I would like to compute the $COV(X_i, X_j)$ but I have some doubts. I have found the formula of the expectation and the variance on Wikipedia. I would ...
0
votes
1answer
34 views

Entropy of gaussian mixture

Does the entropy of a gaussian mixture depend on its means? It is not the case for a single Gaussian and when the components of the mixture are far spread out, we can approximate the entropy by a ...
5
votes
2answers
328 views

Metropolis Hastings algorithm bivariate normals

I need some help implementing the (1) independence Gaussian proposal and (2) random walk Gaussian proposal to simulate from a mixture bivariate normal distribution. "If we have a continuous state ...
6
votes
2answers
135 views

Negative Binomial as Gamma-Poisson Mixture or Compound Logarithmic Poisson: can this correspondence be generalized to other distributions?

Preamble A random variable $X$ with a negative binomial distribution can be characterized in three ways: [Negative Binomial] $X\sim\operatorname{NegBin}(r,p)$ for some $r$ and $p$; [Gamma-Poisson ...
2
votes
0answers
37 views

Approximate a distribution as mixture of $K$ other (known, fixed) distributions

I'd like to draw samples from some "target" probability density function $f(x)$. However, I don't have a way to do that -- instead I just have access to $N$ samples, each drawn from one of $...
3
votes
2answers
60 views

Sum of random variables that follow a finite normal mixture distribution

Let $X_1,X_2,\dotsc,X_n$ be $n$ random variables, and $X_i, i=1,\dotsc,n$ has a density function as $f_i(x)=\lambda_{i1} g_1(x)+\dotsm+\lambda_{im} g_m(x)$, where $g_j, j=1,...m$ are density functions ...
1
vote
1answer
54 views

Goodness of fit for mixture model? [closed]

I have a problem with my vector, I thought that it was a mixture of 2 skew T and I intent to use the ks.test: ...
1
vote
0answers
26 views

Log-likelihood of a finite mixture distribution (PDF overflowing)

I'm trying to use a finite mixture of Dirichlet distributions in a project, but am encountering problems with the PDF becoming so large for input values close to 0 that it overflows to infinity (as ...
2
votes
0answers
19 views

Mixture regression

I am wondering how to analyze and interpret something like the data shown in the figure below (the color is the log-density of the data points, which are a few hundred thousands in number). My ...
3
votes
1answer
35 views

Difference of independent random variables that is not unimodal

This paywalled article shows that the difference of two i.i.d. random variables is unimodal and symmetric if the distribution of the random variables is unimodal. Is there a non-unimodal distribution ...
0
votes
0answers
32 views

Recommended/Commonly Used Likelihoods for TF-IDF Observations in Mixture Models?

What are recommended or commonly used likelihoods for TF-IDF observations in mixture models? The below related questions ask about whether a Multinomial likelihood can be used (and if I understand ...
1
vote
1answer
225 views

How to evaluate the loss on a Gaussian Mixture Model?

I successfully modeled my data using a Gaussian Mixture Model in scikit-learn but I can't figure out how I should say "how good" the model is by calculating the loss. My first thought was to ...
3
votes
1answer
62 views

Does the sum of two variables second order stochastic dominate the mixture of two variables

hello I have two independent variable P and Q. They are both non-negative. Let $\alpha \in (0,1)$. Now I define two new variables on them: The first variable is the sum of the two variables $$R_1:=\...
0
votes
0answers
14 views

Modeling ideas on biologics manufacturing data with complex genealogy

This question is about finding options to model the data for a biologics drug manufacturing. The manufacturing process is divided into upstream and downstream, where the output material from an ...
3
votes
2answers
133 views

Estimating individual components of a mixture distribution

I am trying to jointly estimate the components of a mixture distribution. I have a sampling from a mixture, XY, composed of X ...
2
votes
0answers
39 views

Choosing the Dirichlet prior in a mixture model

Consider the following mixture model with $K < \infty$ components, $$ f\left(x \mid \theta_{1}, \ldots, \theta_{K}, \pi_{1}, \ldots, \pi_{K}\right)=\sum_{k=1}^K \pi_{k} \varphi\left(x \mid \theta_{...
2
votes
1answer
16 views

Compute likelihood of mixture distribution while avoiding floating point problems

I have a mixture distribution with likelihood function $$ L(\theta) = \prod_{i=1}^N \sum_{k=1}^K f(X_i|\theta_k) \lambda_k $$ where $N$ is the sample size, $K$ is the number of component, $\theta_k$ ...
0
votes
1answer
32 views

Mixture model when K=1

Assume that I want to estimate the parameters of a distribution like for example the Gaussian distribution, but I have the code only for the estimation of the parameters of a mixture of Gaussian ...
0
votes
1answer
49 views

Testing a sample is drawn from a mixture

Suppose you have data $(Y_1, \dots, Y_N)$ drawn from a finite mixture population with $K$ components and you estimate the model parameters $(\theta_1,\dots, \theta_K)$ and the mixture weights $(\phi_1,...
1
vote
0answers
128 views

Chain rule for KL divergence, conditional measures

The chain rule for KL divergence is widely seen in the theoretical machine learning literature and generally referenced to [1, Theorem 2.5.3]: $$ \text{KL}[p(x, y) \mid q(x, y)]= \text{KL}[p(x) \mid q(...
1
vote
0answers
20 views

Bayespy model construction

I am trying to solve a classification model using a bayesian approach. In particular I am using as reference the following work: Modeling Analysts’ Recommendations via Bayesian Machine Learning The ...
0
votes
0answers
24 views

Estimate parameters of a concrete categorical mixture model (information retrieval)

Let $f_{i,d}$ be the frequency of the word $i$ in the document $d$ and $l_d$ be the length of the document $d$. Then $P(X = i \mid D = d) = \frac{f_{i,d}}{l_{d}}$ is the probability of drawing the ...
0
votes
0answers
35 views

Transform Copula Marginals to come from Mixture Distribution

I am trying to create a joint distribution that has a specific copula (e.g. Clayton) and whose marginals come from a mixture distribution (e.g. the mixture of two Gaussian distributions). My idea was ...
0
votes
1answer
26 views

Expectation of ratios of probability density functions

I'm trying to solve/simply the expression below:- $\large \mathbb{E_{x \sim b(x)}} B\ [log\left(1 - \frac{A\ a(x)}{2\ c(x)}\right)]$, or $B \large \int_{x}b(x)log\left(1 - \frac{A\ a(x)}{2\ c(x)}\...
0
votes
1answer
23 views

simulation of mixture GARCH models

I want to simulate data that follow a mixture - GARCH specification. The conditional density of the return series $r_t $, based on information up to time t is given by $ f_{t-1}(r_t;\theta) = \sum_{i=...
0
votes
1answer
14 views

How to design an algorithm for k means-like problem?

I have some observation about a random variable x and I want to estimate the parameters of its probability distribution function. For example: data = {1, 2, 3, 4, 5} and choose normal distribution as ...
2
votes
2answers
82 views

Estimating weights of known component distributions in a mixture distribution

Given $n$ probability density functions ($p_1$, ..., $p_n$) with known distributions, what are the ways of estimating the weights ($w_1$, ..., $w_n$) of these component distributions given a sample ...
0
votes
0answers
15 views

How to sample from the distribution of the difference between two dependent mixture models

How to sample from the distribution of the difference between two dependent mixture models I have two distributions that represent a value before and after an event. Each of these distributions is ...
0
votes
0answers
644 views

Fitting mixture model of Gaussians and uniform distributions to real data

I have times series of wind direction and velocity. For now, I leave aside the velocity and focus on the distribution of wind directions. Over there, there is usually three main wind directions, and ...
0
votes
0answers
31 views

What is the nature of $r_i^t$?

Using Expectation Maximization (EM) algorithm, I want to vary the number of clusters used according to $K = [2,4, ... 50]$ for a normal distribution initialized randomly (...
3
votes
1answer
77 views

How to find the Percentile of this distribution?

Let say, we have 2 IIDs $X, Y \sim N \left( 0, 1 \right) + \eta $ Now $\eta$ has a discrete distribution with values 0(-10) with probabilities 0.991 & 0.009 respectively. Also assume that $Z = X+Y$...
2
votes
1answer
49 views

Nonparametric mixture estimation

Let's assume that we have two samples $\{X_i\}_{i=1..N}$ and $\{Y_i\}_{i=1..M}$ corresponding to random variables $X$ and $Y$. Let there also be a sample $\{Z_i\}_{i=1..K}$ corresponding to random ...
1
vote
1answer
99 views

Method of Moments for Mixture distribution

The restaurant owner also wants to reconfigure her seating layout, and has asked you for help in modeling her clients. She gives you a dataset of past reservations, and tells you that she gets a mix ...
7
votes
1answer
99 views

Evaluating (Uniform) Expectations over Non-simple Region

Background. Let $V = (X,Y)$ be a random vector in 2-dimensions uniformly distributed over two disjoint regions $R_X \cup R_Y$ defined as follows: $$ \begin{align} R_X &= ([0,1] \times [0,1]) \...
1
vote
0answers
16 views

Bivariate random effect problems in selection models (Mixture Cure model)

I am currently working on a mixed effects selection model. The selection model is a logistic model with a Gaussian random effect. The principal model is a survival model with a Gaussian random effect (...
0
votes
0answers
19 views

How do I interpretably identify mixture distributions?

I am trying to measure the impact of a binary variable $B$ on continuous $y$, with some covariates $X$. I fit an OLS regression and a mixture model of $2$ OLS regressions. The mixture model fits much ...

1
2 3 4 5
9