Linked Questions
13 questions linked to/from Numerical example to understand Expectation-Maximization
17
votes
2
answers
12k
views
Relation between MAP, EM, and MLE
I am a beginner in machine learning. I can do programming fine but the theory confuses me a lot of the times.
What is the relation between Maximum Likelihood Estimation (MLE), Maximum A posteriori (...
- 439
5
votes
1
answer
4k
views
How can I derive the EM algorithm for a mixture of two Bernoulli distributions?
How can I derive the E-step and M-step in the EM algorithm for a mixture of two Bernoulli distributions? Note that I am aware that there are several notes online that explain how to do this for the ...
- 5,120
2
votes
1
answer
5k
views
Expectation-Maximization Algorithm for Binomial
I have a multinomial distribution with four outcomes, with a pdf:
$$p(x_1,x_2,x_3,x_4)=\frac{n!}{x_1!x_2!x_3!x_4!}p_1^{x_1}p_2^{x_2}p_3^{x_3}p_4^{x_4}, \sum_{i=1}^4x_i=n, \sum_{i=1}^4p_i=1$$
The ...
- 165
3
votes
4
answers
2k
views
Are there any clustering algorithms that do not exclude/impute missing data?
From my understanding, clustering algorithms require complete data. Based on this, if there are missing values in my dataset I have two options:
Impute missing information using some sort of ...
- 415
5
votes
1
answer
3k
views
EM-algorithm and missing data
Does EM-algorithm only work with missing data? If not, what is the idea to assume that we have missing data?
user130885
1
vote
1
answer
2k
views
Example of manual implementation of baum-welch algorithm in R
Is there any code out there that implements the baum-welch algorithm for a very basic problem? It would be very helpful to actually see the algorithm in action to better understand how it works.
I ...
- 51
4
votes
0
answers
2k
views
Expectation-Maximization with a MLE package in R
As a follow up to one answer of the topic Expectation-Maximization with a coin toss:
One of the user posted an R-code with MLE example almost a year ago (and his last online time here was 3 months ago,...
- 1,175
2
votes
1
answer
1k
views
baum-welch parameter estimation numeric example
I have an HMM (picture below), with a single parameter $\theta$ I want to estimate using Baum-Welch.
I have a single training example X="HHT", and I start with an ...
- 3,350
2
votes
2
answers
348
views
Understanding numerical example of expectation maximization
I was trying to understand Expectation maximization algorithm. This is how it is defined in Andrew Ng's Stanford CS229 course:
$$
\text{Repeat until convergence \{}\quad\quad\quad\quad\quad\quad\...
- 235
1
vote
1
answer
192
views
Real time example: Estimation for incomplete data
Following is from Csiszar and Shields' FnT monograph "Information Theory and Statistics":
The expectation–maximization or EM algorithm is an iterative
method frequently used in statistics to ...
- 617
4
votes
1
answer
252
views
Linear regression where some known records have a measurement error in dependent variable
I am modelling data where the dependent variable is the number of units of a certain product sold each month in each area.
In all areas, the product is sold by a chain of shops 'A' and we have exact ...
- 41
2
votes
0
answers
61
views
mixed noise and gaussian
I have a large number of data sets. Each data set has something 200K data points lying in a square times a circle. The square is solid $I\times I$. The circle $S^1$ is hollow (dim 1). By reasoning ...
- 1,147
1
vote
0
answers
44
views
How to determine proportion of a mixture of 2 different normal distributions [closed]
Adult Northern and Southern Yeti heights can each be represented as normal distributions:
Yeti Type mean sd
Northern 92in 4.5in
Southern 95in 4.2in
20,000 Yeti adults ...
- 19