Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,214
questions
1
vote
1
answer
65
views
What does it mean in Deep Learning, that L2 loss or L2 regularization induce a gaussian prior?
A prior for what?
Let's take a toy example of a Deep Neural Network with weights w being fit on a dataset D with an L2/Mean Square Error loss.
I looked up Maximum Likelihood Estimation (MLE) and ...
- 51
4
votes
3
answers
108
views
Estimating the parameter $\beta$
The lifetime of computer monitors has a exponential distribution where the expected value can be written as:
$\mu(s) = \frac{\beta}{s}$
Where $s$ is how bright the monitor is and both $s$ and $\beta$ ...
- 75
0
votes
0
answers
17
views
Nakagami Distribution Shape Parameter
I've problem to solve the parameter estimation using the Maximum Likelihood Function for Nakagami distribution. I've tried to derive the likelihood function with shape and scale parameters. then, I ...
0
votes
0
answers
13
views
About Quasi-ML and its convergence to true parameters
I'm reading about quasi-maximum likelihood estimation. As I understand, if $f$ is the true density, $m(x, \theta)$ some index, and $h$ is a possibly misspecified density, then if we assume that $E[y \...
- 239
0
votes
2
answers
56
views
Why use parameter set $\theta$ in Bayes Rule
I understood the Bayes rules described in the above equation
However in the above equation, class y replaced by parameter set $\theta $
I want to know why we don't immediately use y, but we use the ...
0
votes
0
answers
65
views
Critical Value for a 2-sided Wald's test
Consider the following set of hypotheses:
$H_0:\theta = 1$
$H_1:\theta ≠ 1$
AFAIK, the Wald's test converges in distribution as follows:
$\sqrt{nI(\hat{\theta}_ {MLE})}(\hat{\theta}_{MLE}-\theta)\...
- 107
0
votes
0
answers
22
views
Why is the solution to the Quasi Exponential MLE a minimum of the log-likelihood function?
In the paper "Quasi-maximum exponential likelihood estimator and portmanteau test of double AR(P) model based on Laplace(a,b)" (Xuan, Song, Cig, Sun, Dai, 2018), they want to estimate the ...
- 35
5
votes
2
answers
139
views
Asymptotics of MLE without closed form solutions
Suppose $L(\theta;X)$ denotes the likelihood of a model where $\theta$ is the parameter and $X$ is the data. $L(\theta;X)$ doesn't have a closed-form solution for MLE. I use a numerical procedure to ...
- 153
0
votes
0
answers
194
views
Find the Fisher Information for geometric distribution
Given $X1,\dotsc,Xn \sim \mathcal{Geo}(p)$ , and I need to find the MLE and the CI for the MLE.
I found the MLE for this distribution, using the maximum likelihood function: $L(p;X) = (1-p)^(Xi-1) * p$...
- 413
0
votes
1
answer
87
views
Fitting Gumbel distribution based the maximal observation
Assume that we only consider $$G(x)=\exp(-\exp(\frac{x-\mu}{\sigma}))$$ is the Gumbel distribution.
Question: Suppose we have a set of maximum values $\{Y_i\}_{i=1}^m$, why can the article directly (...
- 557
0
votes
0
answers
3
views
Alternative-specific slope ordered probit
Consider the following ordered response model:
$Pr[Y=j|X]=Pr[\alpha_{j-1}-x'\beta_{j-1}<u<\alpha_{j}-x'\beta_{j}]\quad j=1,2,3,4$
i.e. the coefficients of x is alternative-specific.
Can we ...
- 755
0
votes
0
answers
6
views
Cross-section probit with panel data
Suppose that we have a (balanced) panel data set with $t=1,2$ (i.e. two period panel data).
Here, we have a unit-specific binary random variable:
$$D_i=\begin{cases}1 \quad if\quad X_{i1}'\beta<\...
- 755
4
votes
1
answer
83
views
What do you think of this proof for Fisher information?
I want to prove This formula:
The score function is basically the derivative of the maximum likelihood's log, so to get the information I make another derivative of that:
$$ -E[∂/∂θ s(X;θ)] = -E[∂/∂θ ...
- 389
2
votes
1
answer
80
views
The Monte Carlo of the mean square error of the maximum likelihood estimates
I try to get mean square error of the maximum likelihood estimators in R (using Monte Carlo).
I can write the calculation for the MLE that is repeated once, but I need to repeat the Monte Carlo ...
- 557
1
vote
1
answer
78
views
How do I use MLE for non-iid actual data?
In this paper, the author try to fit the Gumbel distribution based on the r largest value of each year using the maximal likelihood estimators: the likelihood function for r largest values $X_{n1},\...
- 557
0
votes
0
answers
19
views
Maximum likelihood estimation and sampling bias, application of class weights
If I train a model (e.g. a neural net) via maximizing the likelihood function on a given training set, then do I need to adjust for sampling bias?
Let me give you an example for a classification task (...
- 431
1
vote
1
answer
47
views
How to understand the smaller the standard error, the greater the precision?
I am studying the maximal likelihood method to get the estimators. I read the following sentence and I am confused about that
The standard deviation of the sampling distribution of estimator $\hat{\...
- 557
1
vote
1
answer
144
views
How can I compare the MLE by standard error?
In the paper Extreme value theory based on the r largest annual events
(page 32), the idea is that he wants to fit the Gumbel distribution using a dataset. In this dataset, we have the largest ten ...
- 557
0
votes
0
answers
22
views
Likelihood in Discrete distribution vs Continuous distribution in Bayes' Theorem [duplicate]
I would like to have a clear understanding of what exactly is a likelihood function in Bayes' Theorem, and why isn't considered a probability. As well as the distinction between the likelihood in ...
2
votes
1
answer
37
views
MLE for a mixture of betas without using EM algorithm
Suppose I have a mixture of two Beta densities say $f_1 = \text{Beta}(1,1)$ and $f_2= \text{Beta}(1,\beta)$ where $\beta$ is unknown. The sample $X_1,....,X_n$ is observed based on latent Bernoulli ...
- 217
4
votes
1
answer
195
views
Can we fit extreme value distribution by build-in package?
I try to find a package in R to fit Gumbel distribution by Block Maxima Approach using maximal likelihood function (see here)
$$
G(x; \mu , \sigma)=\exp[-e^{-\frac{x-\mu}{\sigma}}].
$$
The block ...
- 557
0
votes
0
answers
20
views
Can we get a Global Maxima by using EM algorithm? [duplicate]
Actually, I read many articles and blogs that EM converges to local maxima can someone help me that it converges to local maxima always? Can we get a global maximum?
7
votes
1
answer
188
views
EM algorithm for Bivariate Normal
Consider a random sample $X_i = (U_i,V_i)$ where $i=1,2,...,n$ from a bivariate normal population with mean $(\mu_1,\mu_2)$ and variances $(\sigma_1 ^2, \sigma_2 ^2)$ and correlation $\rho$. Let's ...
- 217
2
votes
0
answers
26
views
Estimating parameters for a set of related random variables
Suppose I have some random variables
$$X_i \sim Dist(\theta_i)$$
for $i = 1, ..., n$ where $Dist$ is some known probability distribution family and $\theta_i$ are some parameters which may vary ...
- 143
0
votes
0
answers
8
views
How do entropy-based estimators relate to more conventional ML, least square, and GMM estimators
Over the years i have done a lot of analysis, mostly of parameters of linear approximations to the data or a forecast, and I have used linear and nonlinear least squares, maximum likelihood, and GMM ...
- 2,757
0
votes
0
answers
44
views
MLE for initial probability for Hidden Markov Model (supervised learning)
Suppose I have a sequence of observations $\mathbf{o} = (o_1, ..., o_T)$ and a corresponding sequence of states natomiast $\mathbf{q} = (q_1, ..., q_T)$, where $q_i \in \{1,2,...,N\}$
Let $\mathcal{L}(...
- 101
0
votes
0
answers
24
views
When does MLE = MAP? With ELI5 example
Definitions:
MLE = Maximum likelihood estimation.
MAP = Maximum Apriori Estimation.
If we were to use a simple example and say "Use MLE to find the number of free throws made out of 10", ...
- 503
1
vote
0
answers
44
views
what does it mean when the NLL becomes negative?
i was comparing the results of 3 different techniques for regression task( Deep ensembles, variational inference and concrete dropout) and i got these results
from the table looks like everything is ...
- 259
0
votes
0
answers
18
views
Poisson Regression and homogenous association
Let's consider a log-linear Poisson model with three variables A, F, C such that the model is a homogeneous line-by-line association model in AF.
How on earth the maximum likelihood equations are, for ...
- 3
1
vote
0
answers
26
views
How to derive a decision rule as derived from Bayes rule?
I have a dataset consisting of 5 classes and the prior probability is $p(\omega_c)=\frac{|D_c|}{\sum_{i = 1}^{5}|D_i|}$. Suppose each class $c$ associated with the likelihood $p(x|ω_c)\,=\,\text{N}(\...
- 11
2
votes
1
answer
82
views
Maximum likelihood estimation with censored data that include repeated measures
I have received the review for my latest journal article in which I present an experimental study with human subjects. In this study, we have loaded our subjects with impact loads created by a ...
- 35
0
votes
0
answers
8
views
do probabilistic neural networks learn the mean values before the variances?
Hi i trained a simple probabilistic MLP with the maximum likelihood approach. I have assumed my target values normally distributed around the true value and i have plotted the values that the NLL and ...
- 259
0
votes
0
answers
28
views
Problem with maximizing likelihood
I am following the paper of C. Binder for inflation expectations. I have bi-monthly data from 2012 to 2022 (64 waves of data) and for each time period I have to estimate 5 parameters $\mu_h,\sigma_h,\...
0
votes
0
answers
55
views
Computing MLE of a random process with unknown distribution, but known $\text{E}[\cdot]$ and $\text{Var}[\cdot]$?
TLDR: Use Chebshev inequality to derive MLE parameter expressions in case where distribution is unknown, but first and second-order moments are. Is it possible?
Let $X=f(\theta ; x )$ be a stochastic ...
- 111
0
votes
0
answers
20
views
Differences between Maximum Likelihood Estimation and Backpropagation? [duplicate]
Reading the definition of MLE, it sounds like it is: "Given a likelihood function, estimate the most likely parameters."
When I read that, it sounds like it has the same goal of what ...
- 503
0
votes
0
answers
6
views
Maximum likelihood estimation - effect of reducing the number of data rows (by averaging)
Say I want to estimate $\overline x$ given matrix $A$ and $\overline y$ and assume $\overline \epsilon \sim N(0,\sigma^2 I)$:
$$\overline y = A \overline x + \overline\epsilon$$
I want to study the ...
- 101
0
votes
0
answers
69
views
Proof Maximum likelihood ratio test to be a $\chi^2$ distribution
I have been struggling with this demonstration and I can not finish it, I want to demonstrate that for Gaussian samples (of $\sigma$ and $\mu$) the maximum likelihood ratio test holds for a $\chi^2$ ...
- 123
2
votes
1
answer
205
views
VQ-VAE objective - is it ELBO maximization, or minimization of the KL-divergence between the posterior and its approximation?
I'm reading two descriptions of the VQ-VAE objective:
Kingma claims in page 18 that we want to maximize the ELBO, and shows that it can be written as $ELBO = logp_{\theta}(x) - KL(q_{\phi}(z|x)||p_{\...
- 2,810
0
votes
0
answers
117
views
How to prove the MLE for Weibull distribution is biased or unbiased?
My attempt is like this:
Let $X \sim \text{Weibull}(\alpha, \lambda)$ be a random variable following a Weibull distribution with pdf
$fx(x; \lambda) = \begin{cases}
{\alpha}{\lambda^{-\alpha}} x^{\...
- 1
2
votes
0
answers
30
views
How to prove the MLE for theta?
I started off with finding L(Xij; θ), but I'm still struggling to get to this equation in the problem. Any help would be appreciated. Thank you!
- 21
7
votes
4
answers
456
views
Why the variance of Maximum Likelihood Estimator(MLE) will be less than Cramer-Rao Lower Bound(CRLB)?
Consider this example. Suppose we have three events to happen with probability $p_1=p_2=\frac{1}{2}\sin ^2\theta ,p_3=\cos ^2\theta $ respectively. And we suppose the true value $\theta _0=\frac{\pi}{...
- 83
1
vote
2
answers
115
views
How to calculate the MLE for Gilbert's Sine Distribution?
$$ f(x) = \frac{\pi}{2b}\sin\left(\pi\frac{x-a}{b}\right),~~x\in[a,a+b]$$
So it's a pretty simple distribution, which is just a scaled sine wave. As an experiment, I was trying to find the MLE ...
- 131
0
votes
1
answer
48
views
Can MLE be indepedent of the observations?
Let the random variable $X$ follow the distribution:
$$ f(x;\theta) = \theta^2(x+1)(1-\theta)^x$$
where $x$ takes values in $[0, \infty)$ and $\theta$ in $[0, 1]$. The likelihood is defined as:
$$\...
- 221
1
vote
1
answer
38
views
Confusion about the optimized parameters when doing maximum likelihood
When I do ML estimation, I always get confused about whether I should $\max_\theta\prod P(D | \theta)$ or $\max_\theta\prod P(\theta | D)$, where $D=\{x_1,x_2,\dots,x_N\}$ and $\theta$ are the ...
- 291
1
vote
1
answer
196
views
Is likelihood a conditional probability?
If we have a set of observations $\mathcal{D} = \{x_i\}_{i=1}^n$ then the likelihood $\mathcal{L}$ is:
$$ \mathcal{L}(\theta \mid \mathcal{D}) = P_\theta(\mathcal{D})$$ and if the observations are ...
- 221
1
vote
2
answers
200
views
What is the Maximum Likelihood Estimator for which coin was tossed? [closed]
You are given three coins with the following probabilities of observing a Head when tossed:
Coin 1 has a P(H) = 1/2,
Coin 2 has a P(H) = 1/3,
Coin 3 has a P(H) = 1/4.
You observed two heads among ...
- 31
1
vote
0
answers
14
views
When imputing missing labels Y1 == NaN during training, how do additional target vectors (Y2 != NaN) impact learning Y1==NaN?
I am training a Mixture Density Network (MDN) to map from continuous input vectors X to continuous targets Y [i.e. X -> Y]. There are missing labels on vector Yi, which I impute from the mdn (as ...
- 11
2
votes
1
answer
79
views
Finding likelihood function of exponential distribution
I am curious as to how do find the likelihood function for the exponential distribution with parameters such as this:
$$X \sim \exp(\beta- \mu) $$
With the following assumptions,
$\beta$ is known
$\...
0
votes
1
answer
57
views
Properties of Estimators Using Dice Rolls?
I am an MBA student taking courses in statistics.
Today, some of the students from the actual statistics faculty presented a seminar on estimation and probability - it was really interesting! In our ...
- 7,030
0
votes
1
answer
51
views
Problem understanding explanation of why minimizing negative log likelihood gives the same solution as maximizing the likelihood
I am having problems following the wording of this explanation.
Likelihood takes values between 0 and 1. Negative log of this range is
between $\infty$ and zero. So when we maximise the likelihood it ...
- 639