Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

984 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1k views

Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
566 views

Does using bootstrapped samples improve parameter estimates for a fitted distribution?

The R package retimes has a function for fitting an ex-Gaussian distribution to a set of observations. The method involves taking multiple bootstrapped samples of the observations, and fitting the ex-...
306 views

When using L2 regularization outside of linear regression, do the same MAP estimation assumptions hold?

Some context is shared below, and my question is bolded at the end. In the linear regression setting, we learn model weights $\hat{\mathbf{w}}$ to make predictions $\mathbf{\hat{y}}$ from new samples ...
306 views

when can I substitute an inverse with a pseudo-inverse in an estimator

Short Version: can I substitute the Moore-Penrose generalized inverse of a matrix (R function ginv()) for a matrix inverse (R function ...
258 views

asymptotic unbiasedness of weibull mle

It's known that the MLEs of the two-parameter Weibull distribution scale and shape parameters are not available in a closed form. It is, however, known that they do exist, are unique, and moreover, ...
201 views

Why does MLE tend to normal distribution

We have $X_1,\dots, X_n$ are iid (the distribution can be of any type, e.g. Bernoulli (p), normal ($\mu, \sigma^2$), Poisson ($\lambda$). If we use MLE $\hat \theta$ to estimate any parameter $\theta$ ...
171 views

Maximum likelihood and degenerate Fisher information

I am wondering if there are some standard results to find rates of convergence of the MLE for different sub-parameters, when the Fisher information is degenerate. More precisely, suppose that I ...
139 views

Bayesian inference via approximate data likelihood

Suppose that we have a very large i.i.d. sample $x_1,...,x_n$ and a data likelihood defined by $$p(x | \theta,\beta) = \prod_ip(x_i | \theta,\beta)$$. Further suppose that $\theta$ is the parameter ...
1k views

Why is Fisher Scoring easier to compute?

In practice, the observed information matrix (Newton-Raphson) is usually replaced by its expectation, known as Fisher scoring. Link: https://en.wikipedia.org/wiki/Scoring_algorithm#Fisher_scoring ...
739 views

Likelihood maximization: MCEM algorithm versus MCMC algorithm

Hello Everyone this is my first question. I am a particle physicist and I am doing some empirical studiues on parameters estimation using different methods (this might give me some handle to study on ...
2k views

Deriving the maximum likelihood for a generative classification model for K classes

In Christopher Bishop's book "Pattern Recognition and Machine learning", there is the following question: Consider a generative classification model for $K$ classes defined by the prior class ...
2k views

"weight" input in glm.nb function in R. How exactly does the weight affect the likelihood?

I would like to understand how the weight argument of glm.nb is affecting the likelihood function. I understand that glm.nb find ...
158 views

Is this problem Bayesian? And can I use variational approximation?

Suppose there are $N$ samples of observations $\mathbf X(n)$ ($n=1,\cdots,N$), which are given by probability distribution $p(\mathbf X(n)|\mathbf Z(n))$ with their conditions are given by hidden ...
2k views

Parameter Estimation for Naive Bayes - Maximum a posteriori and Maximum Likelihood

I am wondering if I understand those terms correctly. To summarize my thoughts: In naive Bayes, our decision rule is basically the Maximum a posteriori (MAP) estimate of our hypothesis. We assign an ...
596 views

1k views

Maximum Likelihood & Bayesian inference minimizing Kullback-Leibler divergence?

I have heard/read that Bayesian and Maximum Likelihood inference can be justified as asymptotically minmizing the KL divergence between the pdf $p(x)$ actually describing the data and the ...
72 views

Automatic fitting of normalization constant as a parameter in noise contrastive estimation

In the paper on Noise Contrastive Estimation, the authors define a parameterized density function $p_m^0\left(x;\alpha\right)$ to estimate the unnormalized PDF of the data, and then further define a ...
742 views

Implementing ARMA Log Likelihood with the Kalman Filter Algorithm

A popular algorithm to determine the (complex) log likelihood function of an ARMA(p,q) process involves generating it through the use of a state-space model and the Kalman Filter. I started reading ...
299 views

Connecting Poisson and multinomial models

Let's say we have multinomial counts $y_{jp}$ (corresponding to observations $j$ over categories $p=1,...P$) that are arranged in a table of $n$ rows and $P$ columns. Then say we have the log-linear ...
303 views

Closed form of Maximum Likelihood Estimator?

I have this Maximum Likelihood (ML) problem, which gives after simplification: {x_{\hat{\eta}}}^T y_{\hat{\eta}} \times {\mathbb{1}}^T \mathbb{1} - {x_{\hat{\eta}}}^T \mathbb{1} \times {y_{\hat{\eta}...
I've come across an interesting exercise. We are given four classification models for binary response and a $d$-dimensional independent variable: A Linear Discriminant Analysis model where the ...