Linked Questions

12 votes
2 answers
7k views

Relation between MAP, EM, and MLE

I am a beginner in machine learning. I can do programming fine but the theory confuses me a lot of the times. What is the relation between Maximum Likelihood Estimation (MLE), Maximum A posteriori (...
user avatar
  • 389
2 votes
1 answer
4k views

Expectation-Maximization Algorithm for Binomial

I have a multinomial distribution with four outcomes, with a pdf: $$p(x_1,x_2,x_3,x_4)=\frac{n!}{x_1!x_2!x_3!x_4!}p_1^{x_1}p_2^{x_2}p_3^{x_3}p_4^{x_4}, \sum_{i=1}^4x_i=n, \sum_{i=1}^4p_i=1$$ The ...
user avatar
5 votes
1 answer
2k views

EM-algorithm and missing data

Does EM-algorithm only work with missing data? If not, what is the idea to assume that we have missing data?
user avatar
4 votes
0 answers
2k views

Expectation-Maximization with a MLE package in R

As a follow up to one answer of the topic Expectation-Maximization with a coin toss: One of the user posted an R-code with MLE example almost a year ago (and his last online time here was 3 months ago,...
user avatar
  • 1,025
1 vote
1 answer
1k views

Example of manual implementation of baum-welch algorithm in R

Is there any code out there that implements the baum-welch algorithm for a very basic problem? It would be very helpful to actually see the algorithm in action to better understand how it works. I ...
user avatar
2 votes
4 answers
897 views

Are there any clustering algorithms that do not exclude/impute missing data?

From my understanding, clustering algorithms require complete data. Based on this, if there are missing values in my dataset I have two options: Impute missing information using some sort of ...
user avatar
  • 345
1 vote
1 answer
997 views

baum-welch parameter estimation numeric example

I have an HMM (picture below), with a single parameter $\theta$ I want to estimate using Baum-Welch. I have a single training example X="HHT", and I start with an ...
user avatar
  • 2,606
1 vote
1 answer
357 views

How can I derive the EM algorithm for a mixture of two Bernoulli distributions?

How can I derive the E-step and M-step in the EM algorithm for a mixture of two Bernoulli distributions? Note that I am aware that there are several notes online that explain how to do this for the ...
user avatar
  • 2,916
1 vote
1 answer
179 views

Real time example: Estimation for incomplete data

Following is from Csiszar and Shields' FnT monograph "Information Theory and Statistics": The expectation–maximization or EM algorithm is an iterative method frequently used in statistics to ...
user avatar
  • 577
3 votes
1 answer
164 views

Linear regression where some known records have a measurement error in dependent variable

I am modelling data where the dependent variable is the number of units of a certain product sold each month in each area. In all areas, the product is sold by a chain of shops 'A' and we have exact ...
user avatar
  • 31
1 vote
1 answer
69 views

Understanding numerical example of expectation maximization

I was trying to understand Expectation maximization algorithm. This is how it is defined in Andrew Ng's Stanford CS229 course: $$ \text{Repeat until convergence \{}\quad\quad\quad\quad\quad\quad\...
user avatar
  • 123
2 votes
0 answers
51 views

mixed noise and gaussian

I have a large number of data sets. Each data set has something 200K data points lying in a square times a circle. The square is solid $I\times I$. The circle $S^1$ is hollow (dim 1). By reasoning ...
user avatar
1 vote
0 answers
43 views

How to determine proportion of a mixture of 2 different normal distributions [closed]

Adult Northern and Southern Yeti heights can each be represented as normal distributions: Yeti Type mean sd Northern 92in 4.5in Southern 95in 4.2in 20,000 Yeti adults ...
user avatar