Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,217
questions
1
vote
0
answers
22
views
Estimating parameters when drawn from known distribution
I'm trying to estimate a model where the data can be partitioned into subsets that each depend on a single parameter value, where the set of these parameters follow a known distribution. Because the ...
- 11
2
votes
2
answers
50
views
Find the MLE say $\hat{\theta}$ and the asymptotic distribution of $\hat{\theta}$
Let $X_1,\dots, X_n$ be iid sample from $X\sim f(x;\theta)=(\theta+1)x^\theta$ for $\theta>0$. Find the MLE say $\hat{\theta}$ and the asymptotic distribution of $\hat{\theta}$.
My work: The log-...
- 609
0
votes
0
answers
3
views
Need Help in Deriving CRLB of GNSS Direct Position Estimation
I'm trying to code for the CRLB of position for GNSS Direct Position Estimation, as derived by Closas et al. in "Cramér–Rao Bound Analysis of Positioning Approaches in GNSS Receivers" in ...
0
votes
1
answer
25
views
Implementing GARCH(2,2) QMLE: where does the data (squared returns) come into play?
I am trying to implement a QMLE estimation of GARCH(2,2) model as a side project.
We can represent GARCH(2,2) as follows:
\begin{aligned}
r_{t} &= \mu_{t} + \epsilon_{t}, \\
\mu_{t} &= 0, \\
\...
- 157
0
votes
0
answers
18
views
Custom Maximum Likelihood [closed]
Suppose I have some data on $Y$ and $X$, I believe the following function form (plus a normal error term)
$$ Y = X^{-(Y-\gamma)/\omega} $$
holds, and I want to get the estimates for the two parameters ...
- 269
0
votes
0
answers
11
views
Gamma Distribution MLE After New Condition
I found the MLE of $\theta$ where $(x_1,...,x_n)$ is a sample from a $Gamma(\alpha, \theta)$ distribution where $\alpha>0$ is known to be $\frac{\alpha}{\bar{x}}$. However, suppose it is now known ...
- 155
2
votes
1
answer
16
views
Likelihood estimation with 2 samples from a geometric distribution
Context
Given there are 2 groups that can be modelled as a geometric distribution as follows:
\begin{align*}
f(x_i;p_1) &= p_1(1-p_1)^{x_i - 1} \; x_i = 1,2,... \; 0<p_1 <1 \\
f(y_i;p_2) &...
- 23
1
vote
1
answer
80
views
Maximum of a distribution not invariant in different coordinate systems?
Let's say we have a probability distribution in x,y space:
$$
p(x, y)=\frac{1}{4 \pi} \sqrt{x^2+y^2} \exp \left(- \sqrt{x^2+y^2}\right)
$$
This can be converted into polar coordinates as:
$$
p(r, \...
- 11
0
votes
1
answer
69
views
MLE phi derivation
Let dataset $D = \{(x_1,y_1),...,(x_n,y_n)\}$ where $x\in\mathbb{R}^d$ and $y_i\in\{0,1\}$
There are 2 mean vectors $\mu_0,\mu_1\in\mathbb{R}^d$ that represent the means of each feature split by label....
- 13
1
vote
0
answers
30
views
Asymptotic normality of penalized MLE?
Let $L(\theta;X)$ denote the log-likelihood of a model and I maximize the following to estimate $\theta$,
$$
\arg\max_{\theta}L(\theta;X)+\lambda\theta^2
$$
If $\lambda$ is 0, then the asymptotic ...
- 131
0
votes
1
answer
37
views
Use the delta method to find confidence intervals
Given $X_1 ... X_n \sim \textrm{Exp}(\lambda)$, I found the MLE : $$\hat{\lambda} = \frac{1}{\bar{X}}$$
Now I need to find confidence intervals for: $$\eta = \lambda \cdot \log(\lambda)$$
To do so, I ...
- 413
1
vote
0
answers
44
views
The confusing derivation in the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy
In the section 15.5 of the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy, it discusses the Gaussian Process Latent Variable Model. The log-likelihood objective function is ...
- 11
0
votes
0
answers
15
views
How to solve non-identifiability problem in point estimation
I am working with a normal model $X \sim N(0, \sigma^2(\theta))$, where $\sigma^2(\theta) = \frac{1}{e}\cos^2(\theta)+e\sin^2(\theta)$. My goal is to estimate $\theta$ within the range $[0, 2\pi]$.
My ...
- 63
0
votes
0
answers
31
views
Maximum simulated likelihood of binary logit model implementation in python
I am working through the "Microeconometrics and MATLAB" book, translating the code to Python.
Here is my implementation for estimating the parameters of a binary logit model (Colab link). I ...
- 237
2
votes
1
answer
50
views
Help Deriving Likelihood Term When the Target is Known Probabilistically
I am trying to model data $\{Y_t,Q_t\}_{t=1}^T$, where the model is parameterized by $\theta$. $Y_t$ is a quantity where the model prediction can be solved in closed form, $\hat{Y}_t(\theta)$, where ...
2
votes
0
answers
62
views
Problem with the Fisher information matrix in case of N measurements of two observables
Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model ...
- 21
10
votes
2
answers
480
views
Is quantile regression a maximum likelihood method?
Quantile regression allows to estimate a conditional quantile for y (like e.g. the median of y,...) from data x.
I do not see any distributional assumptions about y being made. This seems in contrast ...
- 431
0
votes
0
answers
27
views
Is the EM algorithm guaranteed to converge if the log likelihood is concave
As the EM algorithm is guaranteed to increase the log likelihood at each iteration. If the log likelihood is concave is it guaranteed to converge to the maximum of the likelihood, that is will we get ...
- 55
1
vote
0
answers
14
views
If the most likely value is that which minimizes squared-error, what are the possible distributions?
Gauss uniquely characterised the 1D normal distribution by asking for a distribution that:
is symmetric
is decreasing on either side of some center point $\mu$
has the data likelihood maximized by ...
- 121
-1
votes
0
answers
38
views
What are the good starting values in this question?
since when x = 1 the mean value of y is 200, so we have $\frac{\theta_1}{\theta_2+1} \approx 200$?
and does $Y_i \sim N(\frac{\theta_1+x_i}{\theta_2 x_i},\sigma^2)$ mean var(y) is a good estimation of ...
- 411
1
vote
1
answer
125
views
How to calculate the conditional maximum likelihood of independent negative binomial variables conditioned on the likelihood of the sum?
In the paper Small-sample estimation of negative binomial dispersion, with applications to SAGE data, section 4.1 Conditional maximum likelihood:
For RNA sequencing data, assume the counts for a ...
- 63
0
votes
0
answers
30
views
Optimising Box-Cox lambda analytically
I'm taking a university course in statistics where the Box-Cox transform is being discussed. As I understand it, we assume that there is some $\lambda$ that makes the sample normally distributed after ...
- 101
2
votes
1
answer
38
views
To estimate the parameters of P0, P1, P2 and P3 of a functional response equation by MLE?
this is equation of functional response following polynomial logistic regression with binomial distribution Na=N (exp p0 + p1 N + p2 N^2 + p3 N^3 ) /(1 + exp (p0 + p1 N + p2 N^2 + p3 N^3)). I would ...
2
votes
0
answers
29
views
Confidence interval for the quantiles given interval for parameter
Let us say we have a random sample $X_1,...,X_n\sim F(\theta)$ and we do inference over $\theta$ and give a maximum likelihood estimate $\hat{\theta}$ and a confidence interval at the $\alpha$% given ...
- 268
1
vote
0
answers
13
views
I suspect that the MLE of the scale parameter in scale families of distributions is scale equivariant. Any derivative-free proof? [closed]
Suppose $f(x)=c^{-1}f_0(x/c)$, $c,x>0$. Then the MLE $\hat c_n(X_1,...,X_n)$, satisfies$$\hat c_n(aX_1,...,aX_n)=a\hat c_n(X_1,...,X_n),$$ for all $a>0.$
- 11
0
votes
1
answer
74
views
Estimate parameters in Brownian Motion with drift, $dX_t = \mu dt + \sigma dW_t$
Consider a Brownian Motion with drift, $X$, on the interval $[0; T]$ given by
$$dX_t = \mu dt + \sigma dW_t.$$
Suppose that the interval is split into $n$ pieces of equal size to define $\Delta:=T/n$ ...
- 103
7
votes
2
answers
448
views
Why does the MAP differ from the MLE for the uniform prior in Laplace's Rule?
Laplace's Rule of Succession produces an estimate for the probability $p$ of a Bernoulli distribution. It starts with a $Beta(1,1)$ prior (equivalent to a uniform distribution prior on $(0,1)$), and ...
- 217
2
votes
0
answers
59
views
Estimation of parameters for a random effect model
I want to fit a simple one-way random effect model, i.e.
\begin{equation}
y_{ij} = \mu + \alpha_i + \epsilon_{ij},
\end{equation}
where
$\mu$ denotes the mean
$\alpha_i$ the effect of the $i$-th ...
- 21
0
votes
1
answer
56
views
Find the maximum likelihood estimator of $\theta$ with pdf $f(x)=2x/\theta^2$ [duplicate]
Let $(X_1,\dots, X_n)$ be a random sample from $X$ with pdf $f(x)=2x/\theta^2$ for $0\le x\le \theta$ where $\theta>0$. Find the maximum likelihood estimator of $\theta$.
The likelihood function ...
- 609
1
vote
1
answer
40
views
Estimating Markov chain transition probabilities from data
In a discrete time and space Markov chain, I know the formula to estimate the transition probabilities $$p_{ij} = \frac{n_{ij}}{\sum_{j \in S} n_{ij}}$$ I'm not sure however how you can find this is ...
- 13
0
votes
0
answers
46
views
MLE from a bivariate distribution
I have a sample of size $n$ from a bivariate distribution which have joint pdf with below form
$f_X(x)f_Y(y) \left[1+\lambda\left(2F_X(x)-1\right)\left(2F_Y(y)-1\...
- 343
0
votes
1
answer
42
views
Maximum likelihood fit of left truncated Weibull distribution
I want to fit some samples to the right tail of a Weibull distribution. To fix the notation:
the samples are $\{X_i\}_{i=1,\ldots,n}$,
all samples are greater than a fixed threshold $L$,
the $X_i$'...
- 195
1
vote
0
answers
24
views
Writing Likelihood in terms of Errors?
In the context of Survival Models (e.g. AFT https://en.wikipedia.org/wiki/Accelerated_failure_time_model), I have usually seen the Likelihood Function written as follows (https://hal-univ-pau.archives-...
- 7,048
0
votes
1
answer
91
views
Maximum Likelihood with heteroskedasticity
Suppose I regress $y$ on an explanatory variable $x$, so $y= a+bx + e$ where $e \sim N(0,g(x))$. This means the errors are normally distributed but show heteroskedasticity.
In my model, $x$ is a ...
- 55
0
votes
1
answer
43
views
Maximum likelihood estimate for mixture with components using both cartesian and polar coordinates
I have a set of points (x,y) that were generated from a mixture of two components: one component uses Cartesian coordinates, and the other polar coordinates.
For example, with probability $\gamma$ I ...
- 15
0
votes
0
answers
28
views
What is the influence of multicolinearity on the likelihood ratio test?
I learned the two concepts separately and I try to find out how these concepts relate to each other.
To illustrate the question introduce three models. The models establish a relation between age (AGE)...
- 201
0
votes
0
answers
29
views
Estimation of MLE - request for clarification
I am refering to an earlier post in Maximum Likelihood estimate of cell radius from observed cross section radii
I am not sure if I understood this question. What exactly is the meaning of ...
1
vote
0
answers
62
views
Solving the Neyman-Scott problem via Conditional MLE
In section 2.4 of the book Essential Statistical Inference by Boos and Stefanski, the authors discuss the idea conditional likelihoods and illustrate their usefulness by describing how they can be ...
- 1,358
1
vote
1
answer
53
views
Estimating mle of parameter of exponential distribution [closed]
I have a machine component whose life time is distributed as exponential with parameter $\lambda$
I switch on $n$ such components at time 0, then observe their performances during time $t$ and $t+z$ ...
- 343
3
votes
1
answer
118
views
A proof for existence of MLE
Suppose that for $\theta ∈ (0, 1)$, $X$ is a continuous random variable
with density
$f_\theta(x) =
\frac{3(1 − \theta)}
{4\delta^3(\theta)} [\delta^2(\theta) − (x − \theta)^2]\mathbb{1}(|x − \theta| ≤...
- 231
1
vote
0
answers
25
views
MLE of p in Bernoulli(p) where the range is specified
Came across this where b is binomial (the distribution is Bernoulli(p)). There's only one sample here, and I found that the MLE of p is X itself (by equating the first derivative to 0). I think I'm ...
- 207
0
votes
0
answers
14
views
Mixed logit and predicted shares
In Train 2009, pg. 323, he writes:
... maximum likelihood estimation
of a standard logit model with alternative specific constants for
each product in each market necessarily gives predicted shares ...
2
votes
1
answer
46
views
Likelihood for a log Gaussian Cox process (LGCP)
Suppose I have a log Gaussian Cox process (LGCP) $X$ with log intensity function $\lambda(x)=S(x)$ where $S$ follows a Gaussian process. Since LGCP still falls under the umbrella of inhomogeneous ...
- 81
0
votes
1
answer
49
views
Average log likelihood is maximized by a constant
Say I have the following data-generating process for a binary variable $y\in (0,1)$
$$\mathbb{P}(y=1\mid X) = \frac{1}{1+e^{-\beta X}}$$
where $\beta = (1,0.5,-1)$ and the ith variable $X_i \sim N(0,1)...
0
votes
1
answer
38
views
Least squares estimator for parameter p in binomial distribution
I am trying to find the least square estimator for the parameter $p$ in $Bin(n,p)$ but is it even possible? Isn't it the same as finding it using MLE?
- 1
0
votes
1
answer
40
views
MLE for Bernoulli parameter with constraint
We have $X \sim \text{bernoulli} \left(p \right), p \geq0.50$. Goal is to to estimate the MLE of $p$
With unconstraint case, I can calculate the MLE is the sample mean.
For constraint case, there is ...
- 343
0
votes
1
answer
37
views
Multinomial logit: why likelihood for one observation uses probabilities of all classes?
When dealing with non-binary discrete-choice outcomes, one common way of modeling such problems as a function of some covariates is through a multinomial logit/logistic model, in which there is one ...
- 537
4
votes
1
answer
126
views
(Why) is a logistic regression maximum likelihood estimator consistent?
A nice property of maximum likelihood estimators is that, while they can be biased, they are consistent for $iid$ observations.
In a logistic regression, unless the conditional distributions all have ...
- 47.1k
0
votes
0
answers
105
views
Hessian of the probit model
I was trying to compute the Hessian of the probit model.
Recall that the log likelihood is
$$\log𝐿=∑_{i=1}^{n}[𝑦_i \logΦ(𝐱′_𝐢𝛽)+(1−𝑦_𝑖)\log(1−Φ(𝐱′_i𝛽))]$$
Then, the Hessian is
$$H = E\left(\...
- 55
4
votes
2
answers
175
views
Marginal distribution of an autoregressive process of order one AR(1)
I'm reading "Econometric Modelling with Time Series" by V. L. Martin, A. S. Hurn and D. Harris ( https://www.researchgate.net/file.PostFileLoader.html?id=56bccdaa6225ff0de28b45a6&...
- 45