# Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

449 questions
Filter by
Sorted by
Tagged with
69k views

### Maximum Likelihood Estimation (MLE) in layman terms

Could anyone explain to me in detail about maximum likelihood estimation (MLE) in layman's terms? I would like to know the underlying concept before going into mathematical derivation or equation.
7k views

### Can you give a simple intuitive explanation of IRLS method to find the MLE of a GLM?

Background: I'm trying to follow Princeton's review of MLE estimation for GLM. I understand the basics of MLE estimation: likelihood, ...
2k views

### Maximum likelihood function for mixed type distribution

In general we maximize a function $$L(\theta; x_1, \ldots, x_n) = \prod_{i=1}^n f(x_i \mid \theta)$$ where $f$ is probability density function if the underlying distribution is continuous, and a ...
8k views

### What kind of information is Fisher information?

Suppose we have a random variable $X \sim f(x|\theta)$. If $\theta_0$ were the true parameter, the the likelihood function should be maximized and the derivative equal to zero. This is the basic ...
80k views

### Basic question about Fisher Information matrix and relationship to Hessian and standard errors

Ok, this is a quite basic question, but I am a little bit confused. In my thesis I write: The standard errors can be found by calculating the inverse of the square root of the diagonal elements of ...
87k views

### What is "restricted maximum likelihood" and when should it be used?

I have read in the abstract of this paper that: "The maximum likelihood (ML) procedure of Hartley aud Rao is modified by adapting a transformation from Patterson and Thompson which partitions the ...
3k views

### Can the empirical Hessian of an M-estimator be indefinite?

Jeffrey Wooldridge in his Econometric Analysis of Cross Section and Panel Data (page 357) says that the empirical Hessian "is not guaranteed to be positive definite, or even positive semidefinite, for ...
3k views

### How to construct a cross-entropy loss for general regression targets?

It's common short-hand in neural networks literature to refer to categorical cross-entropy loss as simply "cross-entropy." However, this terminology is ambiguous because different probability ...
25k views

There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
2k views

### Estimating parameters for a binomial

First of all I'd like to precise that I'm not an expert of the subject. Suppose to have two random variables $X$ and $Y$ that are binomial, respectively $X\sim B(n_1,p)$ and $Y\sim B(n_2,p),$ note ...
9k views

### ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
5k views

### the relationship between maximizing the likelihood and minimizing the cross-entropy

There is a statement that maximizing the likelihood is equivalent to minimizing the cross-entropy. Are there any proof for this statement?
4k views

### Properties of logistic regressions

We're working with some logistic regressions and we have realized that the average estimated probability always equals the proportion of ones in the sample; that is, the average of fitted values ...
15k views

### Maximum likelihood estimators for a truncated distribution

Consider $N$ independent samples $S$ obtained from a random variable $X$ that is assumed to follow a truncated distribution (e.g. a truncated normal distribution) of known (finite) minimum and maximum ...
6k views

### Idea and intuition behind quasi maximum likelihood estimation (QMLE)

Question(s): What is the idea and intuition behind quasi maximum likelihood estimation (QMLE; also known as pseudo maximum likelihood estimation, PMLE)? What makes the estimator work when the actual ...
12k views

### How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
5k views

### Why does one have to use REML (instead of ML) for choosing among nested var-covar models?

Various descriptions on model selection on random effects of Linear Mixed Models instruct to use REML. I know difference between REML and ML at some level, but I don't understand why REML should be ...
28k views

### Weibull distribution parameters $k$ and $c$ for wind speed data

Hi can the same be shown to obtain shape and scale parameter for modified maximum likelihood method
7k views

### Bias of maximum likelihood estimators for logistic regression

I would like to understand a couple of fact on maximum likelihood estimators (MLEs) for logistic regressions. Is it true that, in general, the MLE for logistic regression is biased? I would say "yes"....
8k views

### Invariance property of maximum likelihood estimator?

Here is an excerpt from one of the stats books I have been reading: But as a counter example, let's suppose we have five possible values for $\theta$ and $\theta_5$ is the ML estimate, with the ...
41k views

### REML or ML to compare two mixed effects models with differing fixed effects, but with the same random effect?

Background: Note: My data set and R code are included below text I wish to use AIC to compare two mixed effects models generated using the lme4 package in R. Each ...
9k views

### Can we use MLE to estimate Neural Network weights?

I just started to study about stats and models stuff. Currently, my understanding is that we use MLE to estimate the best parameter(s) for a model. However, when I try to understand how the neural ...
5k views

### Is least squares the standard method to fit a 3 parameters Gaussian function to some x and y data?

A participant in one experiment needs to decide whether a flash and a sound are simultaneous or not for many possible asynchronies between the flash and the sound (x in seconds). For each asynchrony, ...
34k views

### What does the inverse of covariance matrix say about data? (Intuitively)

I'm curious about the nature of $\Sigma^{-1}$. Can anybody tell something intuitive about "What does $\Sigma^{-1}$ say about data?" Edit: Thanks for replies After taking some great courses, I'd ...
37k views

### In R, given an output from optim with a hessian matrix, how to calculate parameter confidence intervals using the hessian matrix?

Given an output from optim with a hessian matrix, how to calculate parameter confidence intervals using the hessian matrix? ...
20k views

### Estimating parameters of Student's t-distribution

What are the maximum-likelihood estimators for the parameters of Student's t-distribution? Do they exist in closed form? A quick Google search didn't give me any results. Today I am interested in the ...
3k views

### Linear regression: any non-normal distribution giving identity of OLS and MLE?

This question is inspired from the long discussion in comments here: How does linear regression use the normal distribution? In the usual linear regression model, for simplicity here written with ...
5k views

### Which distributions have closed-form solutions for maximum likelihood estimation?

Which distributions have closed-form solutions for the maximum likelihood estimates of the parameters from a sample of independent observations?
6k views

### Is Maximum Likelihood Estimation (MLE) a parametric approach?

There are two main probabilistic approaches to novelty detection: parametric and non-parametric. The non-parametric approach assumes that the distribution or density function is derived from the ...
93k views

### What is the difference in Bayesian estimate and maximum likelihood estimate?

Please explain to me the difference in Bayesian estimate and Maximum likelihood estimate?