Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
2,737
questions
0
votes
1answer
17 views
normalizing flow training
I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong.
There are $\{y_1,y_2,...,y_n\}$ samples from an unknown distribution $p_y(y)$ that we ...
4
votes
1answer
47 views
Why do we multiply log likelihood times -2 when conducting MLE?
When we are performing maximum likelihood estimation (MLE) to estimate parameters, the fit function is often to -2 * LL, rather than just LL. I also see this "-2LL" term expressed as "...
0
votes
1answer
21 views
Link between Maximum Likelihood and Maximum Probability [duplicate]
How can I see that the maximum likelihood approach finds the parameter values of the probability distribution that maximize the probability of the observed sample?
Maximum likelihood is not the ...
0
votes
0answers
12 views
Confusion about Maximum likelihood estimation [duplicate]
How can I see that the maximum likelihood approach finds the parameter values of the probability distribution that maximize the probability of the observed sample?
Maximum likelihood is not the ...
0
votes
0answers
22 views
Where does the name “score function” come from?
Given a dataset $\; \mathcal{D} = \{ x_i \}_{i=1}^n \;$ and a model $ \;p(\cdot \mid \theta)\;$ with parameters $\; \theta \;$, the likelihood of the parameters is defined as $\; \mathcal{L}(\theta) = ...
0
votes
0answers
4 views
R: INAR / Integer-valued Autoregression order 1 with Poisson and Negative Binomial
I have panel data that contains a number of claims of an individual (i) in a few periods (t).
I want to fit the INAR(1) model with generalization to Poisson and Negative Binomial Distributions.
Did ...
5
votes
1answer
107 views
Bernouilli ML parameter estimation from indirect observations
Suppose I have a coin with a probability $p$ for heads and $1-p$ for tails.
My aim is to estimate $p$ using the max likelihood criterion.
I flip the coin several times but cannot directly observe the ...
0
votes
0answers
5 views
Panel Data with R: Poisson Regression with Random Effect Gamma in Panel Data
Can someone please tell me how to find the estimate parameter with package if I applying this model (look model with RE Estimator 1) in panel data?
I've been searching on it, and I found hglm() ...
2
votes
0answers
18 views
Are these statements about the maximum likelihood estimator and efficiency correct?
I'm trying to understand efficiency and its relation with maximum likelihood estimators so I need someone to confirm or correct these statements I deduced :
1/ If the maximum likelihood estimator ...
2
votes
0answers
40 views
Fisher information of $\rho$ in a symmetric normal $N_p(\mathbf 0,\Sigma)$ distribution
Suppose $\boldsymbol X\sim N_p(\mathbf0,\Sigma)$ where $\Sigma=(1-\rho)I_p+\rho\mathbf1\mathbf1'$ is positive definite. I am trying to find the Fisher information of $\rho$ based on $\boldsymbol X$. ...
0
votes
0answers
13 views
Bounds of the optimisation problem in NICE paper
In the paper https://arxiv.org/abs/1410.8516, a model called Non-linear independent component estimation is proposed. Following are the governing equations for the forward pass in the network.
--------...
0
votes
0answers
17 views
Theoretical properties of joint maximum likelihood estimator on returns and options when fitting an option pricing model
Suppose we have a simple GARCH option pricing model
$$ R_t = \sqrt{h(t)} z(t)$$
$$ h(t) = \omega + \alpha z(t-1) + \beta h(t-1)$$
where $R_t$ is the daily log return, $h(t)$ is the conditional ...
1
vote
1answer
21 views
Negative Log Likelihood for AIC
I was looking at AIC, which is given by AIC = 2K - ln(L).
However, to my understanding, and observation, L = Log-Likelihood can be negative.
So in the case where L is negative, is AIC not applicable ?
...
0
votes
0answers
14 views
Change mixing parameters in likelihood
My understanding of the likelihood function is that it is insensitive to certain changes in parameters, ie if ${\cal L}(x, y)$ is maximal for $\hat{x}$, $\hat{y}$, then ${\cal L}(z=f(x), y)$ is ...
0
votes
1answer
33 views
Does marginal likelihood have closed-form solution for hyperparameters in Bayesian linear regression?
We know that marginal likelihood has the following form in Bayesian linear regression,
$$
\mathbf{K} = \sigma_w^2XX^T + \sigma_n^2I\\
p(\mathbf{y}|X) \sim \mathcal{N}(0, K)\\
\log p(\mathbf{y}|X) =
-...
1
vote
0answers
38 views
Converting Log-Likelihood to Chi-square
I'm using two different algorithms to get a periodogram. One outputs log-likelihood and the other outputs chi-squared test statistic, but I would like a way to convert from log-likelihood to $\chi^2$ ...
3
votes
2answers
79 views
Finding region of rejection with likelihood ratio test
Let $X_1,\ldots,X_n$ be i.i.d. from a Gamma distribution with p.d.f. $f(x;\theta) = \theta^{-2} x e^{-x/\theta}$ for $x>0$ where $\theta$ is an unknown parameter.
I would like to test the ...
0
votes
0answers
8 views
Candidate methods for maximizing multivariable constrained nonlinear loglikelihood function
I want to approximate the maximum value of a nonlinear loglikelihood function with 53 strictly positive variables via numerical methods that do not use derivatives. According to literature there are ...
0
votes
1answer
24 views
Multiple roots of the likelihood equations vs. consistency
I'm trying to understand the implications of the Huzurbazar-Chanda theorem in finite samples. The result basically says that of all roots of the likelihood equations, one and only one tends in ...
0
votes
0answers
21 views
What's the maximum likelihood estimation of $\theta$ in this density?
Suppose we have a n-sample $X=(X_1,..,X_n)$ with a distribution $f(x,\theta)=exp(\theta - x)\unicode{x1D7D9}_{x \geq \theta}(x)$.
Find the maximum likelihood estimator $T$ of $\theta$ and prove that $...
0
votes
1answer
44 views
Induced Likelihood Function for Max Likelihood Estimators
A small doubt on the notation for the induced likelihood function and invariance properties of maximum likelihood estimators - It says in a textbook I am reading that:
Let $X = {X1, X2,..., Xn}$ be a ...
1
vote
0answers
18 views
Finding maximum likelihood solution for mean when data is given which share the same mean but have different variance
I have some 'X sample points say (x1,x2,x3 ...) each of the samples form a Gaussian distribution with mean 'm' and variance v1,v2, ... All the distributions have the same mean but differ in variance. ...
1
vote
0answers
37 views
MLE and MOM estimates coincide in Normal distribution
Let $(X_1,Y_1),...,(X_n,Y_n)$ be a sample form $N(\mu_x,\mu_y,\sigma^2_x,\sigma^2_y,\rho)$ population. If $n\geq 5$ and $\mu_x$ and $\mu_y$ are unknown, I want to conclude that the estimates of all ...
3
votes
1answer
45 views
Geometric Waiting Time MLE
If the time is measured in discrete periods, a model that is often used for the time $X$ failure of an item is:$$P_{\theta}[X=k]=\theta^{k-1}(1-\theta), k=1,2,...$$
where $0<\theta<1$. Suppose ...
1
vote
0answers
16 views
How to obtain uncertainty estimate around MLE parameters based on KL-Divergence?
Suppose I know some true distribution $S(x)$, and I have a method of approximating $S$ based on a transformation of another distribution $G(x|\theta)$. We denote the approximation as $S^*(x|\theta)$. ...
3
votes
1answer
73 views
MLE exists and is unique for iid series
Let $X_1,...,X_n\in R^p$ be i.i.d. with density,
$$f_{\mathbf{\theta}}(\mathbf{x})=c(a)\exp(-|\mathbf{x-\theta}|^a), \mathbf{\theta}\in \mathbb R^p, a\geq 1$$
where $$c^{-1}(a)=\int_{R^p}\exp(-|\...
13
votes
2answers
613 views
MLE for normal distribution with restrictive parameters
Suppose that $X_1, . . . , X_n$, $n\geq 2$, is a sample from a $N(\mu,\sigma^2)$ distribution. Suppose $\mu$ and $\sigma^2$ are both known to be nonnegative but otherwise unspecified. Now, I want to ...
1
vote
0answers
43 views
For a nonlinear regression task, is either Maximum Likelihood Estimation or Least Squares easier to learn a neural network model with?
I have data (x,y) and I want to create a model f(x) that will best approximate y. Let's ...
1
vote
0answers
14 views
0
votes
1answer
91 views
What is Type II maximum likelihood?
It might be some straight forward thing.. But I referred to some threads already over internet to understand what exact does it mean when we use terms "evidence maximization" "type II ...
0
votes
0answers
13 views
How to test whether each sample is generated from bivariate gaussian distribution?
I have dataset that each samples has its frequency along x and y axis. A sample can be visaulized as above. Each sample can be thought as an image which has only one channel.
Each image has central ...
1
vote
2answers
88 views
OLS Loss Function - MLE assumptions?
What are the model assumptions necessary for p(y|x,w) in order for the MLE estimate to provide the OLS loss function?
I know that MLE applied to linear regression gives OLS, I'm just not sure what ...
2
votes
3answers
98 views
Differences between MLE and MAP estimators
Generally speaking, what are the differences between an MLE and a MAP estimator?
If I wanted to improve the performance of a model, how would these differences come into play? Are there specific ...
1
vote
0answers
14 views
Modelling Biprobit using the ML command on STATA
thanks in advance for your help.
For a research project, I am trying to replicate some built-in MLE commands using the ml stata routine.
Specifically, I am trying to replicate the results from the ...
1
vote
0answers
26 views
MLE and Minimal Sufficiency of Parameters in a Piecewise Random Variable [closed]
Problem Setting:
$X_i$ is i.i.d. from a piecewise distribution which is
$$
f_{\theta_1, \theta_2}(x) = \frac{1}{\theta_1+\theta_2}e^{-\frac{x}{\theta_1}}I_{[x>0]} + \frac{1}{\theta_1+\theta_2}e^{\...
0
votes
0answers
37 views
Finding MLE of this confusing setup of distributions
I have been given two random variables $X$ and $Y$ following exponential with means $\lambda_1$ and $\lambda_2$. Let $Z_1 = min(X,Y)$ and $Z_2$ will be 0 if $Z_1 = X_1$ and it will be 1 if $Z_1 = X_2$...
1
vote
0answers
18 views
What is the covariance between function and its argument?
For example, we have log-likelihood function $\ell(\theta)$ and maximum likelihood estimator $\hat \theta_{ML}$ obtained as $\ell(\theta)' = 0$. What would be
$$Cov(\ell(\theta), \hat \theta_{ML}) ?$$
...
0
votes
0answers
36 views
What is an initial consistent estimator and how do I find one?
When maximizing a likelihood function $L(\psi)$, the gradient-based optimization procedure is generally
$$ \tag{5.1}
\hat{\psi}_{r+1} = \hat{\psi}_{r} + \left| I^{*}(\hat{\psi}_{r}) \right|^{-1} D \...
1
vote
1answer
40 views
Find covariance of estimator and derivative of the log-likelihood function
Problem:
Given and estimator $\hat k$. The estimation method is unknown (so, it can be max. likelihood, method of moments or another method), however, we know that $bias(\hat k) = 0$.
Let $L$ be the ...
6
votes
1answer
160 views
What is the score function of two parameters?
According to this wikipedia article, score is the derivative of the log-likelihood function. However, I don't understand what if we have two parameters? For example, the logarithm of pdf has the ...
2
votes
0answers
14 views
Can we say that MSE of MLE is always at most equal to that of UMVUE?
The question is in the title. I was wondering if that is the case, because we are considering a more general class, rather than focusing on the class of unbiased estimators, as we do in case of UMVUE. ...
0
votes
0answers
29 views
Consistency of a hard EM type approach to dealing with latent variables (in a particular setting)
Suppose we observe a sample from the $d$ dimensional random vectors $Y_{1,t},\dots,Y_{N,t}$ for $t=1,\dots,T$.
Suppose further that the data generating process (DGP) is the following, for $t=1,\dots,T$...
0
votes
0answers
28 views
Maximizing likelihood function of continuous normal distribution [duplicate]
If I wanted to maximum likelihood estimator for $f(x)=\frac{1}{\sqrt{\sigma^22\pi}}e^{-\frac{1}{2}(\frac{x-\mu}{\sigma})^2}$ where $-\infty\leq x \leq \infty$, my original plan was to do $\Pi^{n=\...
0
votes
1answer
37 views
MLE with uneasy density function
I want to find the MLE of $\theta$ with the following density.
$ (1- \lvert x - \theta \rvert) \mathbb{1}_{[\theta-1, \theta+1]} (x)$, to confirm : $\widehat{\theta_{n}} = \overline{X_{n}}$ found ...
1
vote
0answers
20 views
Is there a variance/ uncertainty tradeoff when we use log likelihood instead of likelihood? [duplicate]
I am looking at a plot of likelihood against parameters for a Bernoulli context (i.e. finding the p of a coin facing heads) and am comparing it to the equivalent plot for a log likelihood one.
I ...
0
votes
1answer
35 views
Uniformly most powerful test does not exists
I am having tough time understanding this concept
The book says:
“We caution the reader that UMP tests for testing H0 : θ1 ≤ θ ≤ θ2 and
H0′ : θ = θ0 for the one-parameter exponential family do not ...
0
votes
0answers
10 views
Generalized Likelihood Ratio Test with null hypothesis defined by union of sets
Suppose you have a model with likelihood $ \mathcal L(\theta;\boldsymbol X) $, where $\theta \in \Theta$ are parameters and $\boldsymbol X = (X_1, \ldots, X_n ) $ denotes i.i.d. data.
Likelihood ratio ...
3
votes
1answer
39 views
Determine probability of an event using maximum likelihood estimation
Problem
I have a bag of many red and green balls. To find out the ratio between the two, I randomly picked balls with replacement. Out of the 100 outcomes, 60 were red balls and 40 were green balls. ...
3
votes
1answer
148 views
Why use KL-Divergence as loss over MLE?
I have came across this statement several time now
Maximizing likelihood is equivalent to minimizing KL-Divergence
(Sources: Kullback–Leibler divergence and Maximum likelihood as minimizing the ...
6
votes
1answer
87 views
Can long run variance of a time series be used to test mean of the series?
Let $Y_t$ be a stationary time series with $Y_t \sim N(\mu, \gamma_0) \,\, \forall t$.
Further define, $$\bar Y_T \equiv (1/T)\sum\limits_{t=1}^TY_t$$
Further, let $\sigma^2_l$ be the long-run ...
