# Tagged Questions

63 views

### Understanding Dempster et al. on expectation maximization

I'm reading about expectation maximization from Dempster, Laird and Rubin's original paper which can be found from the following link: http://web.mit.edu/6.435/www/Dempster77.pdf My questions are ...
60 views

### In stochastic gradient descent, is there only one update to $\theta$ for each iteration?

I have read that the update equation for stochastic gradient descent is as shown below, for each iteration, k. Does one iteration correspond to one training example? So for each example is there only ...
47 views

### Neural Nets: cost functions

Multi-Layer Neural Nets' implementation of one of the commercial packages contains the following cost functions: squared error, cross-entropy, maximum likelihood and perceptron convergence. While ...
83 views

### Why doesn't ML point estimate equal MAP point estimate even though I'm using uniform prior?

I asked a previous question about why the ML and MAP estimates are the same when using a uniform prior (How does a uniform prior lead to the same estimates from maximum likelihood and mode of ...
154 views

### How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
111 views

### Computing the likelihood gradient on a simple directed graphical model with hidden unit

SHORT VERSION: We have a ('visible') random variable $X$ and a ('hidden') random variable $Z$. We have chosen appropriate distributions $P(X|Z)$ and $P(Z;w)$ where $w$ is the parameter of the model. ...
112 views

### How to choose the distribution and parameters for continuous probability density functions in naive Bayes using maximum likelihood?

Let's assume I want to train a binary naive Bayes classifier, with classes $y_0, y_1$ and $n$-dimensional data. For this one needs to calculate the conditional probabilities $P(x_i | y_j)$ for all ...
133 views

### What are speed differences beetwen ML implementations in different languages?

I am trying to write my own ML library. For speed reasons I started out writing things in C using BLAS, but then I learned that NumPy and Theano also use BLAS. I am wondering if there are huge speed ...
217 views

### List of likelihood-based classification techniques

This is a basic statistical pattern recognition question. I'm aware of LDA classification, Naive Bayes Classification techniques which give output as a likelihood (of data belonging to a certain ...
53 views

### Adjusting existing algorithm - likelihood for presence-only data

Logistic regression fits a model that predicts a binary variable whilst performing a logit transformation of the linear combination (LC) of predictors: 1/1 + exp(-LC). I have a working machine ...
258 views

### Maximum Likelihood Estimation question - minimum log likelihood

I know the formula for the likelihood of some parameters given the data. The result has to be maximised and I can avoid multiplication using the log. How can I make this a minimisation problem (i.e. ...
73 views

### Paired multiarm bandit

I have a set of independent experiments with different distributions and I'm trying to determine which has the highest mean payoff. I would like to treat this as a multi-arm bandit problem, but the ...