a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

1
vote
0answers
31 views

Why are these MLE estimates biased?

I estimate the parameters of survival data with censoring which is simulated from Weibull distribution. The mean time to event was set to 10 by choosing the combinations of shape and scale ...
10
votes
6answers
858 views

MLE in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
-1
votes
0answers
33 views

In the glm function for logistic regression, where is the likelihood function stored? Is it in family? [migrated]

I am currently trying to run a logistic regression on my own, using the functions optim, nlm, etc. However, I am somehow getting ...
4
votes
0answers
49 views
+50

Likelihood and estimates for mixed effects Logistic regression

First let's simulate some data for a logistic regression with fixed and random parts: ...
1
vote
1answer
43 views

Help with MLE regression

I have a data set containing two variables x and y. I want to estimate the parameters for a regression model. The regression ...
8
votes
2answers
223 views

What is the maximum likelihood estimate of the covariance of bivariate normal data when mean and variance are known?

Suppose we have a random sample from a bivariate normal distribution which has zeroes as means and ones as variances, so the only unknown parameter is the covariance. What is the MLE of the ...
0
votes
0answers
25 views

Problem with maximum likelihood estimation in R: NaNs produced

I'm trying to estimate 4 maximum likelihood estimators on a data set containing two variables: x and y. When using the following ...
4
votes
0answers
40 views

Exponential Family: Observed vs. Expected Sufficient Statistics

My question arises from reading reading Minka's "Estimating a Dirichlet Distribution", which states the following without proof in the context of deriving a maximum-likelihood estimator for a ...
1
vote
0answers
19 views

Document classification problem

Assume we have $L$ labelled documents, and $U$ unlabeled ones, where all the documents from class $k$ were generated from a multinomial or Naive Bayes distribution with parameter $\theta_k$, and ...
1
vote
1answer
46 views

Consistency of the extremum estimators

I am trying to understand intuitively the assumptions mentioned in the following theorem: Theorem. If there is a function $Q_0(\theta)$ s.t. (i) $Q_0(\theta)$ is uniquely maximized at $\theta_0$; ...
2
votes
0answers
30 views

Hessian of Laplace distribution

The density of the Laplace distribution is given by: $$f(x;\mu,\sigma)=\frac{1}{2\sigma}\exp\left(-\frac{\vert x- \mu\vert}{\sigma}\right).$$ It is easy to see that this function is not ...
0
votes
0answers
30 views

Confusion between empirical Bayes and Hierarchical Bayes Model for beta-binomial model

I am confuse with difference between empirical Bayes method and hierarchical Bayes method. Take an example (ref : http://www.cs.cmu.edu/~xuerui/papers/ctr.pdf). Suppose we have : $C$ is the ...
2
votes
1answer
38 views

Estimating standard error in a probit: econometrics or programming problem?

This question has two parts, as I do not understand whether my problem is theoretical (identification of the parameters) or practical (insufficient R skills). Econometrics Most "probit" style ...
0
votes
2answers
38 views

Likelihood ratio test to determine if average number of accidents has dropped?

I can't find any worked (non-trivial) practical example for a likelihood ratio test, believe me I have spent hours looking. Here is a question I've been trying to complete but I can't get any further. ...
0
votes
0answers
22 views

Gaussian Process Regression with additional Basis Functions

I'm working through Rasmussen's Gaussian Processes book, and I have a question about the possibility of optimizing additional basis function hyperparameters (in section 2.7 ...
1
vote
1answer
49 views

A modelling question about point processes with heavy tails

I am trying to model a number of point processes for which I have data. If I choose to model each one using a (different) homogeneous Poisson process and estimate the rate using MLE then for some of ...
0
votes
1answer
36 views

2x2 contingency table - maximum likelihood estimate of the odds ratio and exact confidence intervals

I have two questions regarding the following 2x2 contingency table: 1, How can I derive the maximum likelihood estimate of the odds ratio ($OR = (a*d)/(c*b)$) 2, How can I derive exact confidence ...
3
votes
2answers
36 views

Generalized log likelihood ratio test for non-nested models

I understand that if I have two models A and B and A is nested in B then, given some data, I can fit the parameters of A and B using MLE and apply the generalized log likelihood ratio test. In ...
1
vote
0answers
26 views

When does l1 regularisation give a sparse solution?

I was maximising a likelihood function, which is convex. I know that the system has a K-sparse solution. I wanted to know the conditions (or some sufficient conditions) on the likelihood function ...
0
votes
1answer
31 views

Why does maximum likelihood estimation not work in estimating signal in deterministic chaotic noise

I have few conceptual questions related to application of chaos in communications. In few application such as radar Chaotic signal reconstruction with application to noise radar system, cryptography, ...
1
vote
0answers
14 views

Simulated MLE does not exist, when trying to Bootstrap likelihood combinant

Consider this simple logistic model: We have ten $0/1$ observations $y_1,...,y_{10}.$ We model with an intercept and a predictor variable.The ten first observations have predictor value $X_i=0$, ...
0
votes
0answers
27 views

Expected value of likelihood function: Coin flips, biased and unbiased estimators

I've been reading the following (great!) book http://www.inference.phy.cam.ac.uk/itila/, which has sparked some questions about MLE. I'm comfortable with the notion that ML estimators are often ...
0
votes
0answers
26 views

Random effects models / Integrate over the random effect

I am trying to do maximum likelihood estimation and trying to see if the problem can be formulated using a random effect model. Here is the problem description: There are $100$ pairs $(N_i, D_i)$ ...
-1
votes
0answers
7 views

Maximum likelihood estimator for Weibull Parameters calculation [duplicate]

Could you please provide me an example showing the calculation of weibull parameters using MLE method?
2
votes
1answer
32 views

Comparing OLS and ML through log likelihood value

The log-like likelihood values that are computed when I do a regression (by for instance eviews), are they comparable for different estimation techniques, specifically OLS and Maximum Likelihood? My ...
0
votes
0answers
47 views

Estimation parameters for latent (unobserved) variable

Here is my problem: I have 3 variables $X,Y,Z$ : $X$ is the number of clicks we observed on an web advertisement; $Y$ is the number of time a customer do a sign-up on the website after clicking ...
2
votes
1answer
114 views

Question with MLE

I'm having some problems with this question, and was hoping someone here could help. Let $X_1,\ldots,X_2$ be $n$ determinations of a physical constant $\theta$. Consider the model $X_i = ...
0
votes
1answer
23 views

Marquardt Loglikelihood Calculation in Eviews

I paper I am trying to replicate used Eviews to estimate their state space model (by maximizing the associated maximum likelihood). They used the BHHH and Marquardt algorithms. My question is given ...
1
vote
0answers
20 views

Maximum Likelihood through a noisy channel

I have a random variable $X$, which can take $n$ values and is distributed according to multinomial $\Theta=(\theta_1, \theta_2, \cdots, \theta_n)$. I observe a random variable $Y$, where I have that ...
1
vote
1answer
33 views

Heavy-tailed distribution with closed-form ML fit from data

Which (if any) heavy-tailed distributions can we compute the maximum likelihood parameters of, given some data to fit the distribution to?
1
vote
0answers
18 views

Quasi maximum likelihood estimation versus pseudo MLE

If I'm not wrong both "quasi" and "pseudo" denote the same thing, namely the optimization under wrong distributional assumptions. Moreover I think that the terms are not restricted to the assumption ...
4
votes
1answer
88 views

Tricky question about MLE

$X_1, \ldots, X_n$ iid ~ Pois($\lambda$). Suppose, you don't know the value of each $X_i$, but you know if $X_i = 0$ or not for every i. Find MLE for $\lambda$. Does MLE always exist? I ...
0
votes
0answers
31 views

Maximum likelihood estimator for variance in two linear models

I am learning MLE's at my inference class and this is a problem I came accross. Consider two simple linear models. $y_{1j}=\alpha _1+\beta_{1}x_{1j}+\epsilon_{1j}$ and $y_{2j}=\alpha ...
2
votes
1answer
99 views

Asymptotically unbiased estimator using MLE

I am learning Maximum likelihood estimators for a inference class. And this is a problem I came across. Let $X_1,X_2,X_3,\ldots, X_n$ be a random sample with p.m.f $$p(X)=\theta(1-\theta)^x; ...
0
votes
0answers
17 views

Maximum Likelihood estimators in reation to linear models

Consider two simple linear models. $y_{1j}=\alpha _1+\beta_{1}x_{1j}+\epsilon_{1j}$ and $y_{2j}=\alpha _2+\beta_{2}x_{2j}+\epsilon_{2j}$ , $ j=1,2,...,n>2$ where $ ...
0
votes
1answer
18 views

Partial Parameter Estimation (MLE/LMMSE)

I have a basic question. MLE/LMSSE is introduced as follows: $$Y = H\theta + W$$ where $H$ is the linear model matrix, $W$ is measurement noise (let's assume it is normal so MLE = LMSSE). $\theta$ is ...
0
votes
0answers
27 views

Standard Errors of Transformed Variables

I am carrying out an MLE where some I use a log transformation on the variance parameters which are being optimized. When I calculate the standard errors (se) the se of the transformed variables is ...
0
votes
0answers
15 views

Fisher Scoring v/s Coordinate Descent for MLE in R

R base function glm() uses Fishers Scoring for MLE, while the glmnet uses the coordinate descent method to solve the same equation ? Coordinate descent is more time efficient than Fisher Scoring as ...
0
votes
0answers
26 views

Log likelihood - understand depper

I want to use log likelihood formula to relate between two items. The formula is: LLR = 2 sum(k) (H(k) - H(rowSums(k)) - H(colSums(k))) When this is the table: ...
4
votes
1answer
169 views

Parameter estimation based on MLE estimate of another parameter

There is a situation explained below where I intend to apply MLE. The problem statement is that I am estimating a measure $X$. This measure is obtained my Maximum Likelihood estimation technique. ...
0
votes
0answers
18 views

Combining BHHH and Levenberg Marquardt

I already asked a question related to this here: When is Maximum Likelihood the same as Least Squares I know understand how Levenberg Marquardt (LM) can be applied to the objective function. In ...
4
votes
1answer
64 views

When is Maximum Likelihood the same as Least Squares

In this paper on p315: http://www.ssc.upenn.edu/~fdiebold/papers/paper55/DRAfinal.pdf They explain that they use Levenberg Marquardt (LM) (along with BHHH) to maximize the likelihood. However as I ...
0
votes
0answers
26 views

co-twin study regression modelling

I want to conduct a regression analysis on a dataset of identical twins. There is a paper by Carlin et. al. 2005 which reviews the models that are designed for twin studies and includes the Stata ...
2
votes
1answer
24 views

Scale parameter MLE scheme known but how to find according distribution PDF?

For known location, we can find the scale parameter of a normal distribution by calculating the sum of squared differences to the location, then dividing by n-1 and taking the square root. This is the ...
1
vote
0answers
55 views

GARCH(1,1) Implementation question

Could someone shed some light into the implementation of the GARCH(1,1) model contained in page6 of the following document? ...
2
votes
1answer
193 views

Why is this likelihood function equal to the noise PDF?

My professor has this slide up here: Here, $y$ is an observed signal. $H$ is a deterministic transformation, which is assumed known. $f$ is the original signal (which we dont know), and $w$ is ...
3
votes
1answer
79 views

Maximum likelihood estimation for mixed Poisson and Gaussian data

Background I've been doing a little bit of work lately on maximum likelihood estimation (MLE), for cases where the data is normally-distributed and also for cases where the data is Poisson ...
0
votes
0answers
9 views

Accounting for minimum dependent measure in data when fitting a distribution

I have what is possible a naive question. I am current comparing various models (i.e. distributions). And the comparisons do not involve different distributions but rather how the model is fed the ...
3
votes
1answer
49 views

MLE2 (in R): I can't find the right starting values

I am trying to fit several MLEs on a small dataset. I have managed to get it done for all my potential models except for this last one. This last one is actually the model that I expected to be the ...
1
vote
0answers
21 views

Power of maximum likelihood parameter estimates for a linear model

The maximum likelihood parameter estimates for the linear model where $\Pr(Y|X\beta) \sim \mathcal{N}(0,\sigma^2)$ are: $$\hat{\beta} = (X'X)^{-1}X'Y$$ How do you compute the statistical power of ...