a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

0
votes
0answers
10 views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
3
votes
2answers
98 views

Under what circumstances is the log likelihood function of a point process concave?

I am trying to understand under what circumstances the log likelihood function of a point process concave. Assume that the process can be defined by a conditional intensity function and that the log ...
3
votes
2answers
66 views

How to compute (or numerically estimate) the standard error of the MLE

I have a model for which I know the log likelihood function, the gradient of the log likelihood and the Hessian of the log likelihood. For given data I can compute the MLE using a generic optimizer ...
5
votes
1answer
80 views

Estimating $n$ and $p$ for Binomial distribution, repeated counting of partly hidden population

A brief motivation: $n$ critters live in an aquarium, where sadly they often hide in, under or behind things. When the aquarium is observed, each critter is only seen with probability $p$ ...
0
votes
0answers
22 views

How to justify a distribution fit is good

I've got 5 datasets to which I would like to fit a model distribution. I'd like to use the same distribution for each dataset but with different parameters. So I use MLE to compute the best parameters ...
1
vote
1answer
18 views

Supervised Pattern Recognition with Probabilistic Labels

I am interested in supervised pattern recognition problems where the the label associated with each pattern gives the probability of membership for each of the $c$ classes, rather than assigning each ...
0
votes
0answers
20 views

Numerical estimation of MLE in Python — normal distribution and gradient is close to zero away from the mean

I am exploring how to model a data set using normal distributions with both mean and variance defined as linear functions of independent variables. Something like $\mathcal{N} \sim \left (f(x), ...
4
votes
1answer
59 views

It it possible in R to specify a regression formula for the hazard rate for a survival analysis model?

I am currently trying to fit a survival analysis model which has the following survival function: $S(t) = \lambda_i e^{-\lambda_i t}$ but with $\lambda_i = e^{\beta_0 +\beta_1 log(1+X_i)}$ where ...
1
vote
0answers
18 views

Nagelkerke pseudo-R2 with positive log likelihoods

I'm trying to calculate a pseudo-R2 for linear mixed models using Nagelkerke's method . My understanding is that Nagelkerke's pseudo-R2=1-EXP[(-2/n)(l(B)-l(0))], where l(B) and l(0) are the ...
0
votes
1answer
25 views

Mixing probabilities in mixture models using EM

Assume we want to estimate the mixing probabilities ($\pi_{k}$) for each member distribution in the mixture model. We know that $\sum_{m}^{K}\pi_{m}=1$, so we can formulate the optimization problem ...
1
vote
0answers
49 views

Why are these MLE estimates biased?

I estimate the parameters of survival data with censoring which is simulated from Weibull distribution. The mean time to event was set to 10 by choosing the combinations of shape and scale ...
12
votes
7answers
1k views

MLE in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
-1
votes
0answers
33 views

In the glm function for logistic regression, where is the likelihood function stored? Is it in family? [migrated]

I am currently trying to run a logistic regression on my own, using the functions optim, nlm, etc. However, I am somehow getting ...
4
votes
1answer
192 views

Likelihood and estimates for mixed effects Logistic regression

First let's simulate some data for a logistic regression with fixed and random parts: ...
1
vote
1answer
43 views

Help with MLE regression

I have a data set containing two variables x and y. I want to estimate the parameters for a regression model. The regression ...
8
votes
2answers
239 views

What is the maximum likelihood estimate of the covariance of bivariate normal data when mean and variance are known?

Suppose we have a random sample from a bivariate normal distribution which has zeroes as means and ones as variances, so the only unknown parameter is the covariance. What is the MLE of the ...
0
votes
0answers
27 views

Problem with maximum likelihood estimation in R: NaNs produced

I'm trying to estimate 4 maximum likelihood estimators on a data set containing two variables: x and y. When using the following ...
5
votes
1answer
71 views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
1
vote
0answers
22 views

Document classification problem

Assume we have $L$ labelled documents, and $U$ unlabeled ones, where all the documents from class $k$ were generated from a multinomial or Naive Bayes distribution with parameter $\theta_k$, and ...
1
vote
1answer
47 views

Consistency of the extremum estimators

I am trying to understand intuitively the assumptions mentioned in the following theorem: Theorem. If there is a function $Q_0(\theta)$ s.t. (i) $Q_0(\theta)$ is uniquely maximized at $\theta_0$; ...
3
votes
1answer
46 views

Hessian of Laplace distribution

The density of the Laplace distribution is given by: $$f(x;\mu,\sigma)=\frac{1}{2\sigma}\exp\left(-\frac{\vert x- \mu\vert}{\sigma}\right).$$ It is easy to see that this function is not ...
0
votes
0answers
34 views

Confusion between empirical Bayes and Hierarchical Bayes Model for beta-binomial model

I am confuse with difference between empirical Bayes method and hierarchical Bayes method. Take an example (ref : http://www.cs.cmu.edu/~xuerui/papers/ctr.pdf). Suppose we have : $C$ is the ...
2
votes
1answer
42 views

Estimating standard error in a probit: econometrics or programming problem?

This question has two parts, as I do not understand whether my problem is theoretical (identification of the parameters) or practical (insufficient R skills). Econometrics Most "probit" style ...
0
votes
2answers
40 views

Likelihood ratio test to determine if average number of accidents has dropped?

I can't find any worked (non-trivial) practical example for a likelihood ratio test, believe me I have spent hours looking. Here is a question I've been trying to complete but I can't get any further. ...
0
votes
0answers
23 views

Gaussian Process Regression with additional Basis Functions

I'm working through Rasmussen's Gaussian Processes book, and I have a question about the possibility of optimizing additional basis function hyperparameters (in section 2.7 ...
1
vote
1answer
51 views

A modelling question about point processes with heavy tails

I am trying to model a number of point processes for which I have data. If I choose to model each one using a (different) homogeneous Poisson process and estimate the rate using MLE then for some of ...
0
votes
1answer
38 views

2x2 contingency table - maximum likelihood estimate of the odds ratio and exact confidence intervals

I have two questions regarding the following 2x2 contingency table: 1, How can I derive the maximum likelihood estimate of the odds ratio ($OR = (a*d)/(c*b)$) 2, How can I derive exact confidence ...
3
votes
2answers
41 views

Generalized log likelihood ratio test for non-nested models

I understand that if I have two models A and B and A is nested in B then, given some data, I can fit the parameters of A and B using MLE and apply the generalized log likelihood ratio test. In ...
1
vote
0answers
28 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
0
votes
1answer
33 views

Why does maximum likelihood estimation not work in estimating signal in deterministic chaotic noise

I have few conceptual questions related to application of chaos in communications. In few application such as radar Chaotic signal reconstruction with application to noise radar system, cryptography, ...
1
vote
0answers
14 views

Simulated MLE does not exist, when trying to Bootstrap likelihood combinant

Consider this simple logistic model: We have ten $0/1$ observations $y_1,...,y_{10}.$ We model with an intercept and a predictor variable.The ten first observations have predictor value $X_i=0$, ...
0
votes
0answers
30 views

Expected value of likelihood function: Coin flips, biased and unbiased estimators

I've been reading the following (great!) book http://www.inference.phy.cam.ac.uk/itila/, which has sparked some questions about MLE. I'm comfortable with the notion that ML estimators are often ...
0
votes
0answers
26 views

Random effects models / Integrate over the random effect

I am trying to do maximum likelihood estimation and trying to see if the problem can be formulated using a random effect model. Here is the problem description: There are $100$ pairs $(N_i, D_i)$ ...
2
votes
1answer
34 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
0
votes
0answers
50 views

Estimation parameters for latent (unobserved) variable

Here is my problem: I have 3 variables $X,Y,Z$ : $X$ is the number of clicks we observed on an web advertisement; $Y$ is the number of time a customer do a sign-up on the website after clicking ...
2
votes
1answer
114 views

Question with MLE

I'm having some problems with this question, and was hoping someone here could help. Let $X_1,\ldots,X_2$ be $n$ determinations of a physical constant $\theta$. Consider the model $X_i = ...
0
votes
1answer
25 views

Marquardt Loglikelihood Calculation in Eviews

I paper I am trying to replicate used Eviews to estimate their state space model (by maximizing the associated maximum likelihood). They used the BHHH and Marquardt algorithms. My question is given ...
1
vote
0answers
20 views

Maximum Likelihood through a noisy channel

I have a random variable $X$, which can take $n$ values and is distributed according to multinomial $\Theta=(\theta_1, \theta_2, \cdots, \theta_n)$. I observe a random variable $Y$, where I have that ...
1
vote
1answer
33 views

Heavy-tailed distribution with closed-form ML fit from data

Which (if any) heavy-tailed distributions can we compute the maximum likelihood parameters of, given some data to fit the distribution to?
1
vote
0answers
18 views

Quasi maximum likelihood estimation versus pseudo MLE

If I'm not wrong both "quasi" and "pseudo" denote the same thing, namely the optimization under wrong distributional assumptions. Moreover I think that the terms are not restricted to the assumption ...
4
votes
1answer
90 views

Tricky question about MLE

$X_1, \ldots, X_n$ iid ~ Pois($\lambda$). Suppose, you don't know the value of each $X_i$, but you know if $X_i = 0$ or not for every i. Find MLE for $\lambda$. Does MLE always exist? I ...
1
vote
1answer
42 views

Maximum likelihood estimator for variance in two linear models

I am learning MLE's at my inference class and this is a problem I came accross. Consider two simple linear models. $y_{1j}=\alpha _1+\beta_{1}x_{1j}+\epsilon_{1j}$ and $y_{2j}=\alpha ...
2
votes
1answer
100 views

Asymptotically unbiased estimator using MLE

I am learning Maximum likelihood estimators for a inference class. And this is a problem I came across. Let $X_1,X_2,X_3,\ldots, X_n$ be a random sample with p.m.f $$p(X)=\theta(1-\theta)^x; ...
0
votes
0answers
17 views

Maximum Likelihood estimators in reation to linear models

Consider two simple linear models. $y_{1j}=\alpha _1+\beta_{1}x_{1j}+\epsilon_{1j}$ and $y_{2j}=\alpha _2+\beta_{2}x_{2j}+\epsilon_{2j}$ , $ j=1,2,...,n>2$ where $ ...
0
votes
1answer
18 views

Partial Parameter Estimation (MLE/LMMSE)

I have a basic question. MLE/LMSSE is introduced as follows: $$Y = H\theta + W$$ where $H$ is the linear model matrix, $W$ is measurement noise (let's assume it is normal so MLE = LMSSE). $\theta$ is ...
0
votes
0answers
27 views

Standard Errors of Transformed Variables

I am carrying out an MLE where some I use a log transformation on the variance parameters which are being optimized. When I calculate the standard errors (se) the se of the transformed variables is ...
0
votes
0answers
17 views

Fisher Scoring v/s Coordinate Descent for MLE in R

R base function glm() uses Fishers Scoring for MLE, while the glmnet uses the coordinate descent method to solve the same equation ? Coordinate descent is more time efficient than Fisher Scoring as ...
0
votes
0answers
27 views

Log likelihood - understand depper

I want to use log likelihood formula to relate between two items. The formula is: LLR = 2 sum(k) (H(k) - H(rowSums(k)) - H(colSums(k))) When this is the table: ...
4
votes
1answer
169 views

Parameter estimation based on MLE estimate of another parameter

There is a situation explained below where I intend to apply MLE. The problem statement is that I am estimating a measure $X$. This measure is obtained my Maximum Likelihood estimation technique. ...
0
votes
0answers
18 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...