Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
0
votes
0answers
8 views

How to apply 3PL IRT in software i have developed a 3pl model but dont know how to fit in maximum likelihood function [closed]

I have developed a software or an app that calculate theta of an individual only by applying 3PL equation. Calculating all 3 parameters a,b, c on the behalf of available research papers. The problem ...
2
votes
0answers
23 views

Practical example of a non-measurable maximum likelihood estimator

This post gives an example of a situation where the MLE is not measurable. However, this doesn't seem to be a situation that you would ever encounter in practice. Is there a more practical example of ...
2
votes
1answer
57 views

normalizing flow training [closed]

I've been learning about normalizing flow. This is my understanding, and please correct me whenever I am wrong. There are $\{y_1,y_2,...,y_n\}$ samples from an unknown distribution $p_y(y)$ that we ...
8
votes
1answer
124 views

Why do we multiply log likelihood times -2 when conducting MLE?

When we are performing maximum likelihood estimation (MLE) to estimate parameters, the fit function is often to -2 * LL, rather than just LL. I also see this "-2LL" term expressed as "...
0
votes
1answer
21 views

Link between Maximum Likelihood and Maximum Probability [duplicate]

How can I see that the maximum likelihood approach finds the parameter values of the probability distribution that maximize the probability of the observed sample? Maximum likelihood is not the ...
0
votes
0answers
12 views

Confusion about Maximum likelihood estimation [duplicate]

How can I see that the maximum likelihood approach finds the parameter values of the probability distribution that maximize the probability of the observed sample? Maximum likelihood is not the ...
0
votes
0answers
24 views

Where does the name “score function” come from?

Given a dataset $\; \mathcal{D} = \{ x_i \}_{i=1}^n \;$ and a model $ \;p(\cdot \mid \theta)\;$ with parameters $\; \theta \;$, the likelihood of the parameters is defined as $\; \mathcal{L}(\theta) = ...
0
votes
0answers
4 views

R: INAR / Integer-valued Autoregression order 1 with Poisson and Negative Binomial

I have panel data that contains a number of claims of an individual (i) in a few periods (t). I want to fit the INAR(1) model with generalization to Poisson and Negative Binomial Distributions. Did ...
5
votes
1answer
113 views

Bernouilli ML parameter estimation from indirect observations

Suppose I have a coin with a probability $p$ for heads and $1-p$ for tails. My aim is to estimate $p$ using the max likelihood criterion. I flip the coin several times but cannot directly observe the ...
0
votes
0answers
6 views

Panel Data with R: Poisson Regression with Random Effect Gamma in Panel Data

Can someone please tell me how to find the estimate parameter with package if I applying this model (look model with RE Estimator 1) in panel data? I've been searching on it, and I found hglm() ...
2
votes
0answers
18 views

Are these statements about the maximum likelihood estimator and efficiency correct?

I'm trying to understand efficiency and its relation with maximum likelihood estimators so I need someone to confirm or correct these statements I deduced : 1/ If the maximum likelihood estimator ...
4
votes
1answer
111 views

Fisher information of $\rho$ in a symmetric normal $N_p(\mathbf 0,\Sigma)$ distribution

Suppose $\boldsymbol X=(X_1,\ldots,X_p)'\sim N_p(\mathbf0,\Sigma)$ where $\Sigma=(1-\rho)I_p+\rho\mathbf1\mathbf1'$ is positive definite. The objective is to obtain the asymptotic variance of the MLE ...
0
votes
0answers
13 views

Bounds of the optimisation problem in NICE paper

In the paper https://arxiv.org/abs/1410.8516, a model called Non-linear independent component estimation is proposed. Following are the governing equations for the forward pass in the network. --------...
0
votes
0answers
18 views

Theoretical properties of joint maximum likelihood estimator on returns and options when fitting an option pricing model

Suppose we have a simple GARCH option pricing model $$ R_t = \sqrt{h(t)} z(t)$$ $$ h(t) = \omega + \alpha z(t-1) + \beta h(t-1)$$ where $R_t$ is the daily log return, $h(t)$ is the conditional ...
1
vote
1answer
22 views

Negative Log Likelihood for AIC

I was looking at AIC, which is given by AIC = 2K - ln(L). However, to my understanding, and observation, L = Log-Likelihood can be negative. So in the case where L is negative, is AIC not applicable ? ...
0
votes
0answers
15 views

Change mixing parameters in likelihood

My understanding of the likelihood function is that it is insensitive to certain changes in parameters, ie if ${\cal L}(x, y)$ is maximal for $\hat{x}$, $\hat{y}$, then ${\cal L}(z=f(x), y)$ is ...
0
votes
1answer
33 views

Does marginal likelihood have closed-form solution for hyperparameters in Bayesian linear regression?

We know that marginal likelihood has the following form in Bayesian linear regression, $$ \mathbf{K} = \sigma_w^2XX^T + \sigma_n^2I\\ p(\mathbf{y}|X) \sim \mathcal{N}(0, K)\\ \log p(\mathbf{y}|X) = -...
1
vote
0answers
38 views

Converting Log-Likelihood to Chi-square

I'm using two different algorithms to get a periodogram. One outputs log-likelihood and the other outputs chi-squared test statistic, but I would like a way to convert from log-likelihood to $\chi^2$ ...
3
votes
2answers
82 views

Finding region of rejection with likelihood ratio test

Let $X_1,\ldots,X_n$ be i.i.d. from a Gamma distribution with p.d.f. $f(x;\theta) = \theta^{-2} x e^{-x/\theta}$ for $x>0$ where $\theta$ is an unknown parameter. I would like to test the ...
0
votes
0answers
8 views

Candidate methods for maximizing multivariable constrained nonlinear loglikelihood function

I want to approximate the maximum value of a nonlinear loglikelihood function with 53 strictly positive variables via numerical methods that do not use derivatives. According to literature there are ...
0
votes
1answer
25 views

Multiple roots of the likelihood equations vs. consistency

I'm trying to understand the implications of the Huzurbazar-Chanda theorem in finite samples. The result basically says that of all roots of the likelihood equations, one and only one tends in ...
0
votes
0answers
23 views

What's the maximum likelihood estimation of $\theta$ in this density?

Suppose we have a n-sample $X=(X_1,..,X_n)$ with a distribution $f(x,\theta)=exp(\theta - x)\unicode{x1D7D9}_{x \geq \theta}(x)$. Find the maximum likelihood estimator $T$ of $\theta$ and prove that $...
0
votes
1answer
45 views

Induced Likelihood Function for Max Likelihood Estimators

A small doubt on the notation for the induced likelihood function and invariance properties of maximum likelihood estimators - It says in a textbook I am reading that: Let $X = {X1, X2,..., Xn}$ be a ...
1
vote
0answers
19 views

Finding maximum likelihood solution for mean when data is given which share the same mean but have different variance

I have some 'X sample points say (x1,x2,x3 ...) each of the samples form a Gaussian distribution with mean 'm' and variance v1,v2, ... All the distributions have the same mean but differ in variance. ...
1
vote
0answers
37 views

MLE and MOM estimates coincide in Normal distribution

Let $(X_1,Y_1),...,(X_n,Y_n)$ be a sample form $N(\mu_x,\mu_y,\sigma^2_x,\sigma^2_y,\rho)$ population. If $n\geq 5$ and $\mu_x$ and $\mu_y$ are unknown, I want to conclude that the estimates of all ...
3
votes
1answer
45 views

Geometric Waiting Time MLE

If the time is measured in discrete periods, a model that is often used for the time $X$ failure of an item is:$$P_{\theta}[X=k]=\theta^{k-1}(1-\theta), k=1,2,...$$ where $0<\theta<1$. Suppose ...
1
vote
0answers
16 views

How to obtain uncertainty estimate around MLE parameters based on KL-Divergence?

Suppose I know some true distribution $S(x)$, and I have a method of approximating $S$ based on a transformation of another distribution $G(x|\theta)$. We denote the approximation as $S^*(x|\theta)$. ...
3
votes
1answer
73 views

MLE exists and is unique for iid series

Let $X_1,...,X_n\in R^p$ be i.i.d. with density, $$f_{\mathbf{\theta}}(\mathbf{x})=c(a)\exp(-|\mathbf{x-\theta}|^a), \mathbf{\theta}\in \mathbb R^p, a\geq 1$$ where $$c^{-1}(a)=\int_{R^p}\exp(-|\...
13
votes
2answers
616 views

MLE for normal distribution with restrictive parameters

Suppose that $X_1, . . . , X_n$, $n\geq 2$, is a sample from a $N(\mu,\sigma^2)$ distribution. Suppose $\mu$ and $\sigma^2$ are both known to be nonnegative but otherwise unspecified. Now, I want to ...
1
vote
0answers
43 views

For a nonlinear regression task, is either Maximum Likelihood Estimation or Least Squares easier to learn a neural network model with?

I have data (x,y) and I want to create a model f(x) that will best approximate y. Let's ...
1
vote
0answers
14 views

Extract Maximum Likelihood for AR(2) model

I created the AR(2) model you can see below: ...
0
votes
1answer
101 views

What is Type II maximum likelihood?

It might be some straight forward thing.. But I referred to some threads already over internet to understand what exact does it mean when we use terms "evidence maximization" "type II ...
0
votes
0answers
13 views

How to test whether each sample is generated from bivariate gaussian distribution?

I have dataset that each samples has its frequency along x and y axis. A sample can be visaulized as above. Each sample can be thought as an image which has only one channel. Each image has central ...
1
vote
2answers
88 views

OLS Loss Function - MLE assumptions?

What are the model assumptions necessary for p(y|x,w) in order for the MLE estimate to provide the OLS loss function? I know that MLE applied to linear regression gives OLS, I'm just not sure what ...
2
votes
3answers
99 views

Differences between MLE and MAP estimators

Generally speaking, what are the differences between an MLE and a MAP estimator? If I wanted to improve the performance of a model, how would these differences come into play? Are there specific ...
1
vote
0answers
14 views

Modelling Biprobit using the ML command on STATA

thanks in advance for your help. For a research project, I am trying to replicate some built-in MLE commands using the ml stata routine. Specifically, I am trying to replicate the results from the ...
1
vote
0answers
27 views

MLE and Minimal Sufficiency of Parameters in a Piecewise Random Variable [closed]

Problem Setting: $X_i$ is i.i.d. from a piecewise distribution which is $$ f_{\theta_1, \theta_2}(x) = \frac{1}{\theta_1+\theta_2}e^{-\frac{x}{\theta_1}}I_{[x>0]} + \frac{1}{\theta_1+\theta_2}e^{\...
0
votes
0answers
37 views

Finding MLE of this confusing setup of distributions

I have been given two random variables $X$ and $Y$ following exponential with means $\lambda_1$ and $\lambda_2$. Let $Z_1 = min(X,Y)$ and $Z_2$ will be 0 if $Z_1 = X_1$ and it will be 1 if $Z_1 = X_2$...
1
vote
0answers
18 views

What is the covariance between function and its argument?

For example, we have log-likelihood function $\ell(\theta)$ and maximum likelihood estimator $\hat \theta_{ML}$ obtained as $\ell(\theta)' = 0$. What would be $$Cov(\ell(\theta), \hat \theta_{ML}) ?$$ ...
0
votes
0answers
36 views

What is an initial consistent estimator and how do I find one?

When maximizing a likelihood function $L(\psi)$, the gradient-based optimization procedure is generally $$ \tag{5.1} \hat{\psi}_{r+1} = \hat{\psi}_{r} + \left| I^{*}(\hat{\psi}_{r}) \right|^{-1} D \...
1
vote
1answer
40 views

Find covariance of estimator and derivative of the log-likelihood function

Problem: Given and estimator $\hat k$. The estimation method is unknown (so, it can be max. likelihood, method of moments or another method), however, we know that $bias(\hat k) = 0$. Let $L$ be the ...
6
votes
1answer
163 views

What is the score function of two parameters?

According to this wikipedia article, score is the derivative of the log-likelihood function. However, I don't understand what if we have two parameters? For example, the logarithm of pdf has the ...
2
votes
0answers
14 views

Can we say that MSE of MLE is always at most equal to that of UMVUE?

The question is in the title. I was wondering if that is the case, because we are considering a more general class, rather than focusing on the class of unbiased estimators, as we do in case of UMVUE. ...
0
votes
0answers
29 views

Consistency of a hard EM type approach to dealing with latent variables (in a particular setting)

Suppose we observe a sample from the $d$ dimensional random vectors $Y_{1,t},\dots,Y_{N,t}$ for $t=1,\dots,T$. Suppose further that the data generating process (DGP) is the following, for $t=1,\dots,T$...
0
votes
0answers
28 views

Maximizing likelihood function of continuous normal distribution [duplicate]

If I wanted to maximum likelihood estimator for $f(x)=\frac{1}{\sqrt{\sigma^22\pi}}e^{-\frac{1}{2}(\frac{x-\mu}{\sigma})^2}$ where $-\infty\leq x \leq \infty$, my original plan was to do $\Pi^{n=\...
0
votes
1answer
37 views

MLE with uneasy density function

I want to find the MLE of $\theta$ with the following density. $ (1- \lvert x - \theta \rvert) \mathbb{1}_{[\theta-1, \theta+1]} (x)$, to confirm : $\widehat{\theta_{n}} = \overline{X_{n}}$ found ...
1
vote
0answers
20 views

Is there a variance/ uncertainty tradeoff when we use log likelihood instead of likelihood? [duplicate]

I am looking at a plot of likelihood against parameters for a Bernoulli context (i.e. finding the p of a coin facing heads) and am comparing it to the equivalent plot for a log likelihood one. I ...
0
votes
1answer
36 views

Uniformly most powerful test does not exists

I am having tough time understanding this concept The book says: “We caution the reader that UMP tests for testing H0 : θ1 ≤ θ ≤ θ2 and H0′ : θ = θ0 for the one-parameter exponential family do not ...
0
votes
0answers
10 views

Generalized Likelihood Ratio Test with null hypothesis defined by union of sets

Suppose you have a model with likelihood $ \mathcal L(\theta;\boldsymbol X) $, where $\theta \in \Theta$ are parameters and $\boldsymbol X = (X_1, \ldots, X_n ) $ denotes i.i.d. data. Likelihood ratio ...
3
votes
1answer
39 views

Determine probability of an event using maximum likelihood estimation

Problem I have a bag of many red and green balls. To find out the ratio between the two, I randomly picked balls with replacement. Out of the 100 outcomes, 60 were red balls and 40 were green balls. ...

1
2 3 4 5
55