An optimization algorithm often used for maximum-likelihood estimation in the presence of missing data.

learn more… | top users | synonyms (1)

0
votes
1answer
23 views

non-EM algorithm approach to mixture model?

I have a mixture model and the components are further parameterized by ~200 variables. Originally I use EM-algorithm to get a MLE estimation of the parameters. The algorithm works quite well and ...
0
votes
0answers
29 views

How to separate distributions from weighted dataset?

I’m trying to separate two component distributions of an apparent finite mixture from a weighted dataset (determined by a weighted.histogram). I've a set of data and weights only for a part of the ...
1
vote
0answers
39 views

How to normalize bimodal (or multimodal) distributions?

If I have multiple data series, a = [a1, a2, ... a100] ~ bimodal with mu_a1, mu_a2, sigma_a1, sigma_a2, b = [b1, b2, ... b100] ~ bimodal with mu_b1, mu_b2, sigma_b1, sigma_b2, c = [c1, c2, ... ...
0
votes
0answers
11 views

How to treat “Missing Value Analysis” test results (problems)

I have a problem with Missing Value Analysis. I am using SPSS version 20. I am trying to test whether missing values are at complete random. As I know in order to ensure missing values are completely ...
6
votes
1answer
98 views

Why does the EM algorithm have to be iterative?

Suppose that you have a population with $N$ units, each with a random variable $X_i \sim \text{Poisson}(\lambda)$. You observe $n = N-n_0$ values for any unit for which $X_i > 0$. We want an ...
0
votes
0answers
14 views

A framework for comparing the performance of Expectation Maximization

I have my own implementation of the Expectation Maximization (EM) algorithm based on the following paper http://pdf.aminer.org/000/221/588/fuzzy_k_means_clustering_with_crisp_regions.pdf I would like ...
0
votes
0answers
40 views

Estimation parameters for latent (unobserved) variable

Here is my problem: I have 3 variables $X,Y,Z$ : $X$ is the number of clicks we observed on an web advertisement; $Y$ is the number of time a customer do a sign-up on the website after clicking ...
2
votes
0answers
32 views

Relation between variational Bayes and EM

I read somewhere that Variational Bayes method is a generalization of the EM algorithm. Indeed, the iterative parts of the algorithms are very similar. In order to test whether the EM algorithm is a ...
1
vote
1answer
43 views

Relation between Gaussian mixture models and maximum likelihood?

I need some help understanding the relation between the maximum likelihood and Gaussian mixture models. I have seen that there is a relationship between the expectation maximization algorithms and ...
0
votes
0answers
28 views

Expectation maximization with variant length at observing data

Imagine one loaded dice. Based on EM algorithm, how could we compute how much it loaded if we introduced: Variant length on each rolling attempt (look at first and second attempt below 1st one has 6 ...
1
vote
0answers
26 views

Principled approach for PCA on correlated variables?

Related to Should one remove highly correlated variables before doing PCA?, PCA is used a lot in population genetics to essentially cluster individuals into ethnic group based on their genetic markers ...
2
votes
1answer
26 views

Gaussian clusters and original distributions

In Gaussian clustering (i.e. General Mixture Models) we model the data with some clusters. For example, in the below figure, we have two clusters $C_1, C_2$, each of which are modeled with a Gaussian ...
1
vote
0answers
23 views

EM Algorithm - Expectation w.r.t. a subset of current parameters

Suppose I want to make inference on a parameter vector $\theta $=$(\theta_{1},\theta_{2},\theta_{3})$ and I have some missing data $Y_{mis}$. I would like to use the EM algorithm to find the mode of ...
1
vote
0answers
48 views

Concavity of log likelihood for hidden markov models

Could you give me a good link where the concept of concavity of the log likelihood related to hidden markov model EM algorithm is clarified? Thank you in advance.
1
vote
0answers
24 views

What is delta in the maximization step in this EM algorithm?

The algorithm is used to classify english vs non-english tweets from unlabeled data. Given n observed tweets (x1 ... xn) where each tweet xi is a collection of d words (xi1 ... xid). y is the class ...
0
votes
0answers
22 views

peak detection - points / sigma ratio

I would like to ask you (statistics experts) if the following approach is kosher or a nonsense. Problem My EM based detection algorithm ends up (for some data) with a result that looks like attached ...
0
votes
1answer
45 views

Bayes' theorem in 1-d EM algorithm

I'm watching a video on the EM algorithm, When we use Bayes' Theorem to calculate $b_i$, how do I find $P(b)$ and $P(a)$ initially? It says we can estimate the priors $P(b)$ and $P(a)$ but that's ...
4
votes
2answers
95 views

EM algorithm Practice Problem

This is a practice problem for a midterm exam. The problem is an EM algorithm example. I am having trouble with part (f). I list parts (a)-(e) for completion and in case I made a mistake earlier. Let ...
3
votes
1answer
42 views

EM algorithm decreases!

I have used the Bayes Net Toolbox to build a small network, which consists of 3 nodes and is shown below. Node 1 is a Bernoulli random variable, node 2 is a Gaussian random variable and node 3 is a ...
1
vote
1answer
33 views

Why Baum-Welch algorithm is an instantiation of EM algorithm?

$\newcommand{\E}{\mathrm{E}}$ I don't understand why Baum-Welch algorithm is an instantiation of EM algorithm. Indeed, why computing $\alpha_t(i)$ and $\beta_t(i)$ corresponds to Expectation step. ...
1
vote
0answers
23 views

Mode of Joint Posterior - Maximization Problems

I have a problem whereby I get two different answers if I try to maximize a function. let $ \begin{bmatrix} Y_{o}\\ Y_{a} \end{bmatrix}|\phi\sim N (0,\phi^{-1}A^{-1}) $ $\pi(\phi)=\frac{1}{\phi}$, ...
5
votes
1answer
128 views

Why is optimizing a mixture of Gaussian directly computationally hard?

Consider the log likelihood of a mixture of guassians: $$l(S_n; \theta) = \sum^n_{t=1}logP(x^{(t)}|\theta) = \sum^n_{t=1}log\sum^k_{i=1}p_iP(x^{(t)}|\mu^{(i)}, \sigma^2_i)$$ I was wondering why it ...
0
votes
0answers
15 views

Maximum likelihood hypothesis vs Expectation maximization

Maximum Likelihood is given by the formula $h_{ML}=arg\space max_{h \in H } \space\space p(D/h)$ I want to transform it in terms of mechanism involved in Expectation maximization. $h_{ML}=arg\space ...
0
votes
0answers
19 views

Possibilities of accelerating EM algortihm

I'm trying to use the EM to estimate some parameters. I've programmed and it delivers. The problem however is that for each run of my programme, it can take either 5 seconds, 1min, 3min or more to ...
1
vote
0answers
53 views

In EM derivation why can I sum over the iid variables in the conditional expectation?

In EM when you take the expectation: $E[\log P(y,x \mid \theta)\mid x, \theta']$ $= \sum\limits_yP(y\mid x, \theta') \log P(y,x\mid \theta)$ I understand this but the following part I don't ...
0
votes
0answers
54 views

Proper likelihood function in acceptance probability of Gibbs Sampler

I have a question about the acceptance ratio used when implementing a random walk M-H in a gibbs sampler to generate sample paths of an unobservable process. When computing the likelihood of a set of ...
0
votes
0answers
23 views

Simulating EM versus listwise deletion--unexpected results

I'm preparing a presentation on missing data strategies and conducted a simulation to compare listwise deletion (LD) to the EM. Here's what I expected based on the literature: Standard errors will ...
1
vote
1answer
44 views

Efficience of Expectation-Maximization algorithm in function of learning dataset size

I have datasets of increasing sizes identically distributed. I have tried to fit a gaussian mixture to these datasets using Expectation-Maximization algorithm. To check the quality of this fit, I ...
2
votes
0answers
29 views

Computing Standard Errors in EM algorithm

I'm applying the EM to a hidden markov chain (the $\mathbf{Z}=\{Z_1,...,Z_n\}$ variable), with observations(the $\mathbf{Y}=\{Y_0,...,Y_n\}$ variable) dependent not only on the hidden markov chain, ...
0
votes
0answers
56 views

Too good results from EM for gaussian mixture

My task is to identify parameters (mean, standard deviation, height) of gaussian peaks in given histogram data with as lowest CV as possible. Number of peaks and approximate means are known (pointed ...
0
votes
0answers
42 views

Learning Conditional Random Fields using EM Algorithm (from unaligned data)

I am trying to learn CRF from the unaligned data in Natural Language Understanding application. There is one paper in this field which does exactly the same, Learning conditional random fields from ...
2
votes
1answer
132 views

Estimating parameters for univariate skew t

How can I solve the MLE for the skew-t distribution via EM? I am comfortable with the EM methods for t, so could someone show it for the skew-t?
1
vote
1answer
108 views

t-distribution parameter estimation

I know there are already several threads on this, but none seem to explicitly cover what I want. I have a set of financial data (pulled straight from Bloomberg) and am trying to fit a t-distribution ...
1
vote
1answer
82 views

Variance of EM mean estimates in a simple mixture of two normals

Consider a mixture of two normal distributions: $ f(x) = p N(x|u_1, S_1) + (1-p) N(x|u_2, S_2) $ where N() is the normal pdf. $p$, $S_2$, and $S_2$ are known. The means are not. You can get the MLE ...
0
votes
1answer
80 views

Expectation maximization and Gaussian mixtures - bad results

I am supposed to find parameters of individual gaussians in a 1D mixture with a known number of components. I use my own implementation of EM algorithm; however, I am not able to find the right ...
0
votes
0answers
26 views

EM Algorithm to maximize the censored likelihood

I want to implement the EM algorithm to maximize the following censored likelihood when the data is normal: The complete data likelihood is: resulting in the expected complete-data log-likelihood ...
10
votes
0answers
111 views

Training a basic Markov Random Field for classifying pixels in an image

I am attempting to learn how to use Markov Random Fields to segment regions in an image. I do not understand some of the parameters in the MRF or why the expectation maximisation I perform fails to ...
0
votes
0answers
32 views

Expectation-Maximization (EM) method for parameter estimation using fuzzy logic

I am sorry if my question is not fit here. If so, please recommend me the correct forum. I am thinking of estimating a fuzzy model using the EM method. I have a set of observations from a nonlinear ...
10
votes
1answer
201 views

Why is Expectation Maximization algorithm guaranteed to converge to minimum, even local?

I have read a couple of explanations of EM algorithm (e.g. from Bishop's Pattern Recognition and Machine Learning and from Roger and Gerolami First Course on Machine Learning). The derivation of EM is ...
1
vote
0answers
37 views

EM on product of multinomials

I have the following conditional density: $$ P(x | \theta, \pi) = \prod_{i=1}^I \prod_{j=1}^J t_{ij}! \prod_{k=1}^K \frac{1}{x_{ijk}!}(\sum_{l=1}^L \theta_{il} \pi_{jkl})^{x_{ijk}} $$ Here, $x$ is ...
0
votes
1answer
29 views

Global search operators for approximate MAP inference?

In complicated Bayesian models, like for instance a hierarchical nonparameteric one, often times it's intractable to do Gibbs or other MCMC sampling methods to convergence. Rather, people tend to do ...
2
votes
0answers
61 views

Confusion about the EM algorithm

I am reading through the EM-Algorithm but on the slide 39, I don't get how $$P(D) = P(D|A)P(A) + P(D|B)P(B)$$ I am trying to understand it in order to get my head around modelling with mixture of ...
1
vote
0answers
79 views

Comparison of Variational Bayes and Expectation Maximization algorithms

I need to learn both the VB and EM methods for Bayesian Networks. Before going into detail of both algorithms, which I am a bit aware of, I need to EXACTLY understand the basic motivations behind ...
10
votes
2answers
193 views

Does MLE always mean we know the underlying PDF of our data, and does EM mean we don't?

I have some simple conceptual questions that I would like clarified regarding MLE (Maximum Likelihood Estimation), and what link it has, if any, to EM (Expectation Maximization). As I understand it, ...
3
votes
1answer
75 views

Understanding Dempster et al. on expectation maximization

I'm reading about expectation maximization from Dempster, Laird and Rubin's original paper which can be found from the following link: http://web.mit.edu/6.435/www/Dempster77.pdf My questions are ...
2
votes
2answers
80 views

What happens when you initialize EM with a consistent estimate?

I have a certain family of models with latent random variables (not observed in the data), which I have a consistent estimator for. I now run EM on top of it, meaning, I get the consistent ...
2
votes
1answer
41 views

Examples of “Hard” Expectation Maximization other than clustering?

Are there examples of learning algorithms(other than k-means clustering) which fit the paradigm of Hard-EM? By hard EM, I mean the variant described in here.
0
votes
0answers
26 views

Sliding window EM algorithm for PLSA?

I'm wondering is there a version of sliding window EM? I'm working on an application involving a stream of text (tweets). We are thinking of using 1 day data as training set, and extract the topics ...
2
votes
2answers
326 views

Clustering with K-Means and EM: how are they related?

I have studied algorithms for clustering data (unsupervised learning): EM, and k-means. I keep reading the following : k-means is a variant of EM, with the assumptions that clusters are ...
0
votes
0answers
49 views

Normal Data with missing values (EM Algorithm)

Suppose we have multivariate normal data (some missing entries, at random), with known covariance matrix. We would like to estimate the mean vector by the EM algorithm. How would one approach this? ...