Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
1 vote
0 answers
22 views

Estimating parameters when drawn from known distribution

I'm trying to estimate a model where the data can be partitioned into subsets that each depend on a single parameter value, where the set of these parameters follow a known distribution. Because the ...
  • 11
2 votes
2 answers
50 views

Find the MLE say $\hat{\theta}$ and the asymptotic distribution of $\hat{\theta}$

Let $X_1,\dots, X_n$ be iid sample from $X\sim f(x;\theta)=(\theta+1)x^\theta$ for $\theta>0$. Find the MLE say $\hat{\theta}$ and the asymptotic distribution of $\hat{\theta}$. My work: The log-...
  • 609
0 votes
0 answers
3 views

Need Help in Deriving CRLB of GNSS Direct Position Estimation

I'm trying to code for the CRLB of position for GNSS Direct Position Estimation, as derived by Closas et al. in "Cramér–Rao Bound Analysis of Positioning Approaches in GNSS Receivers" in ...
0 votes
1 answer
25 views

Implementing GARCH(2,2) QMLE: where does the data (squared returns) come into play?

I am trying to implement a QMLE estimation of GARCH(2,2) model as a side project. We can represent GARCH(2,2) as follows: \begin{aligned} r_{t} &= \mu_{t} + \epsilon_{t}, \\ \mu_{t} &= 0, \\ \...
  • 157
0 votes
0 answers
18 views

Custom Maximum Likelihood [closed]

Suppose I have some data on $Y$ and $X$, I believe the following function form (plus a normal error term) $$ Y = X^{-(Y-\gamma)/\omega} $$ holds, and I want to get the estimates for the two parameters ...
  • 269
0 votes
0 answers
11 views

Gamma Distribution MLE After New Condition

I found the MLE of $\theta$ where $(x_1,...,x_n)$ is a sample from a $Gamma(\alpha, \theta)$ distribution where $\alpha>0$ is known to be $\frac{\alpha}{\bar{x}}$. However, suppose it is now known ...
  • 155
2 votes
1 answer
16 views

Likelihood estimation with 2 samples from a geometric distribution

Context Given there are 2 groups that can be modelled as a geometric distribution as follows: \begin{align*} f(x_i;p_1) &= p_1(1-p_1)^{x_i - 1} \; x_i = 1,2,... \; 0<p_1 <1 \\ f(y_i;p_2) &...
1 vote
1 answer
80 views

Maximum of a distribution not invariant in different coordinate systems?

Let's say we have a probability distribution in x,y space: $$ p(x, y)=\frac{1}{4 \pi} \sqrt{x^2+y^2} \exp \left(- \sqrt{x^2+y^2}\right) $$ This can be converted into polar coordinates as: $$ p(r, \...
0 votes
1 answer
69 views

MLE phi derivation

Let dataset $D = \{(x_1,y_1),...,(x_n,y_n)\}$ where $x\in\mathbb{R}^d$ and $y_i\in\{0,1\}$ There are 2 mean vectors $\mu_0,\mu_1\in\mathbb{R}^d$ that represent the means of each feature split by label....
  • 13
1 vote
0 answers
30 views

Asymptotic normality of penalized MLE?

Let $L(\theta;X)$ denote the log-likelihood of a model and I maximize the following to estimate $\theta$, $$ \arg\max_{\theta}L(\theta;X)+\lambda\theta^2 $$ If $\lambda$ is 0, then the asymptotic ...
0 votes
1 answer
37 views

Use the delta method to find confidence intervals

Given $X_1 ... X_n \sim \textrm{Exp}(\lambda)$, I found the MLE : $$\hat{\lambda} = \frac{1}{\bar{X}}$$ Now I need to find confidence intervals for: $$\eta = \lambda \cdot \log(\lambda)$$ To do so, I ...
  • 413
1 vote
0 answers
44 views

The confusing derivation in the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy

In the section 15.5 of the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy, it discusses the Gaussian Process Latent Variable Model. The log-likelihood objective function is ...
0 votes
0 answers
15 views

How to solve non-identifiability problem in point estimation

I am working with a normal model $X \sim N(0, \sigma^2(\theta))$, where $\sigma^2(\theta) = \frac{1}{e}\cos^2(\theta)+e\sin^2(\theta)$. My goal is to estimate $\theta$ within the range $[0, 2\pi]$. My ...
  • 63
0 votes
0 answers
31 views

Maximum simulated likelihood of binary logit model implementation in python

I am working through the "Microeconometrics and MATLAB" book, translating the code to Python. Here is my implementation for estimating the parameters of a binary logit model (Colab link). I ...
  • 237
2 votes
1 answer
50 views

Help Deriving Likelihood Term When the Target is Known Probabilistically

I am trying to model data $\{Y_t,Q_t\}_{t=1}^T$, where the model is parameterized by $\theta$. $Y_t$ is a quantity where the model prediction can be solved in closed form, $\hat{Y}_t(\theta)$, where ...
2 votes
0 answers
62 views

Problem with the Fisher information matrix in case of N measurements of two observables

Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model ...
  • 21
10 votes
2 answers
480 views

Is quantile regression a maximum likelihood method?

Quantile regression allows to estimate a conditional quantile for y (like e.g. the median of y,...) from data x. I do not see any distributional assumptions about y being made. This seems in contrast ...
  • 431
0 votes
0 answers
27 views

Is the EM algorithm guaranteed to converge if the log likelihood is concave

As the EM algorithm is guaranteed to increase the log likelihood at each iteration. If the log likelihood is concave is it guaranteed to converge to the maximum of the likelihood, that is will we get ...
1 vote
0 answers
14 views

If the most likely value is that which minimizes squared-error, what are the possible distributions?

Gauss uniquely characterised the 1D normal distribution by asking for a distribution that: is symmetric is decreasing on either side of some center point $\mu$ has the data likelihood maximized by ...
  • 121
-1 votes
0 answers
38 views

What are the good starting values in this question?

since when x = 1 the mean value of y is 200, so we have $\frac{\theta_1}{\theta_2+1} \approx 200$? and does $Y_i \sim N(\frac{\theta_1+x_i}{\theta_2 x_i},\sigma^2)$ mean var(y) is a good estimation of ...
1 vote
1 answer
125 views

How to calculate the conditional maximum likelihood of independent negative binomial variables conditioned on the likelihood of the sum?

In the paper Small-sample estimation of negative binomial dispersion, with applications to SAGE data, section 4.1 Conditional maximum likelihood: For RNA sequencing data, assume the counts for a ...
  • 63
0 votes
0 answers
30 views

Optimising Box-Cox lambda analytically

I'm taking a university course in statistics where the Box-Cox transform is being discussed. As I understand it, we assume that there is some $\lambda$ that makes the sample normally distributed after ...
  • 101
2 votes
1 answer
38 views

To estimate the parameters of P0, P1, P2 and P3 of a functional response equation by MLE?

this is equation of functional response following polynomial logistic regression with binomial distribution Na=N (exp p0 + p1 N + p2 N^2 + p3 N^3 ) /(1 + exp (p0 + p1 N + p2 N^2 + p3 N^3)). I would ...
2 votes
0 answers
29 views

Confidence interval for the quantiles given interval for parameter

Let us say we have a random sample $X_1,...,X_n\sim F(\theta)$ and we do inference over $\theta$ and give a maximum likelihood estimate $\hat{\theta}$ and a confidence interval at the $\alpha$% given ...
1 vote
0 answers
13 views

I suspect that the MLE of the scale parameter in scale families of distributions is scale equivariant. Any derivative-free proof? [closed]

Suppose $f(x)=c^{-1}f_0(x/c)$, $c,x>0$. Then the MLE $\hat c_n(X_1,...,X_n)$, satisfies$$\hat c_n(aX_1,...,aX_n)=a\hat c_n(X_1,...,X_n),$$ for all $a>0.$
  • 11
0 votes
1 answer
74 views

Estimate parameters in Brownian Motion with drift, $dX_t = \mu dt + \sigma dW_t$

Consider a Brownian Motion with drift, $X$, on the interval $[0; T]$ given by $$dX_t = \mu dt + \sigma dW_t.$$ Suppose that the interval is split into $n$ pieces of equal size to define $\Delta:=T/n$ ...
  • 103
7 votes
2 answers
448 views

Why does the MAP differ from the MLE for the uniform prior in Laplace's Rule?

Laplace's Rule of Succession produces an estimate for the probability $p$ of a Bernoulli distribution. It starts with a $Beta(1,1)$ prior (equivalent to a uniform distribution prior on $(0,1)$), and ...
  • 217
2 votes
0 answers
59 views

Estimation of parameters for a random effect model

I want to fit a simple one-way random effect model, i.e. \begin{equation} y_{ij} = \mu + \alpha_i + \epsilon_{ij}, \end{equation} where $\mu$ denotes the mean $\alpha_i$ the effect of the $i$-th ...
  • 21
0 votes
1 answer
56 views

Find the maximum likelihood estimator of $\theta$ with pdf $f(x)=2x/\theta^2$ [duplicate]

Let $(X_1,\dots, X_n)$ be a random sample from $X$ with pdf $f(x)=2x/\theta^2$ for $0\le x\le \theta$ where $\theta>0$. Find the maximum likelihood estimator of $\theta$. The likelihood function ...
  • 609
1 vote
1 answer
40 views

Estimating Markov chain transition probabilities from data

In a discrete time and space Markov chain, I know the formula to estimate the transition probabilities $$p_{ij} = \frac{n_{ij}}{\sum_{j \in S} n_{ij}}$$ I'm not sure however how you can find this is ...
0 votes
0 answers
46 views

MLE from a bivariate distribution

I have a sample of size $n$ from a bivariate distribution which have joint pdf with below form $f_X(x)f_Y(y) \left[1+\lambda\left(2F_X(x)-1\right)\left(2F_Y(y)-1\...
0 votes
1 answer
42 views

Maximum likelihood fit of left truncated Weibull distribution

I want to fit some samples to the right tail of a Weibull distribution. To fix the notation: the samples are $\{X_i\}_{i=1,\ldots,n}$, all samples are greater than a fixed threshold $L$, the $X_i$'...
  • 195
1 vote
0 answers
24 views

Writing Likelihood in terms of Errors?

In the context of Survival Models (e.g. AFT https://en.wikipedia.org/wiki/Accelerated_failure_time_model), I have usually seen the Likelihood Function written as follows (https://hal-univ-pau.archives-...
  • 7,048
0 votes
1 answer
91 views

Maximum Likelihood with heteroskedasticity

Suppose I regress $y$ on an explanatory variable $x$, so $y= a+bx + e$ where $e \sim N(0,g(x))$. This means the errors are normally distributed but show heteroskedasticity. In my model, $x$ is a ...
0 votes
1 answer
43 views

Maximum likelihood estimate for mixture with components using both cartesian and polar coordinates

I have a set of points (x,y) that were generated from a mixture of two components: one component uses Cartesian coordinates, and the other polar coordinates. For example, with probability $\gamma$ I ...
  • 15
0 votes
0 answers
28 views

What is the influence of multicolinearity on the likelihood ratio test?

I learned the two concepts separately and I try to find out how these concepts relate to each other. To illustrate the question introduce three models. The models establish a relation between age (AGE)...
  • 201
0 votes
0 answers
29 views

Estimation of MLE - request for clarification

I am refering to an earlier post in Maximum Likelihood estimate of cell radius from observed cross section radii I am not sure if I understood this question. What exactly is the meaning of ...
1 vote
0 answers
62 views

Solving the Neyman-Scott problem via Conditional MLE

In section 2.4 of the book Essential Statistical Inference by Boos and Stefanski, the authors discuss the idea conditional likelihoods and illustrate their usefulness by describing how they can be ...
  • 1,358
1 vote
1 answer
53 views

Estimating mle of parameter of exponential distribution [closed]

I have a machine component whose life time is distributed as exponential with parameter $\lambda$ I switch on $n$ such components at time 0, then observe their performances during time $t$ and $t+z$ ...
3 votes
1 answer
118 views

A proof for existence of MLE

Suppose that for $\theta ∈ (0, 1)$, $X$ is a continuous random variable with density $f_\theta(x) = \frac{3(1 − \theta)} {4\delta^3(\theta)} [\delta^2(\theta) − (x − \theta)^2]\mathbb{1}(|x − \theta| ≤...
1 vote
0 answers
25 views

MLE of p in Bernoulli(p) where the range is specified

Came across this where b is binomial (the distribution is Bernoulli(p)). There's only one sample here, and I found that the MLE of p is X itself (by equating the first derivative to 0). I think I'm ...
  • 207
0 votes
0 answers
14 views

Mixed logit and predicted shares

In Train 2009, pg. 323, he writes: ... maximum likelihood estimation of a standard logit model with alternative specific constants for each product in each market necessarily gives predicted shares ...
2 votes
1 answer
46 views

Likelihood for a log Gaussian Cox process (LGCP)

Suppose I have a log Gaussian Cox process (LGCP) $X$ with log intensity function $\lambda(x)=S(x)$ where $S$ follows a Gaussian process. Since LGCP still falls under the umbrella of inhomogeneous ...
0 votes
1 answer
49 views

Average log likelihood is maximized by a constant

Say I have the following data-generating process for a binary variable $y\in (0,1)$ $$\mathbb{P}(y=1\mid X) = \frac{1}{1+e^{-\beta X}}$$ where $\beta = (1,0.5,-1)$ and the ith variable $X_i \sim N(0,1)...
0 votes
1 answer
38 views

Least squares estimator for parameter p in binomial distribution

I am trying to find the least square estimator for the parameter $p$ in $Bin(n,p)$ but is it even possible? Isn't it the same as finding it using MLE?
  • 1
0 votes
1 answer
40 views

MLE for Bernoulli parameter with constraint

We have $X \sim \text{bernoulli} \left(p \right), p \geq0.50$. Goal is to to estimate the MLE of $p$ With unconstraint case, I can calculate the MLE is the sample mean. For constraint case, there is ...
0 votes
1 answer
37 views

Multinomial logit: why likelihood for one observation uses probabilities of all classes?

When dealing with non-binary discrete-choice outcomes, one common way of modeling such problems as a function of some covariates is through a multinomial logit/logistic model, in which there is one ...
4 votes
1 answer
126 views

(Why) is a logistic regression maximum likelihood estimator consistent?

A nice property of maximum likelihood estimators is that, while they can be biased, they are consistent for $iid$ observations. In a logistic regression, unless the conditional distributions all have ...
  • 47.1k
0 votes
0 answers
105 views

Hessian of the probit model

I was trying to compute the Hessian of the probit model. Recall that the log likelihood is $$\log𝐿=∑_{i=1}^{n}[𝑦_i \logΦ(𝐱′_𝐢𝛽)+(1−𝑦_𝑖)\log(1−Φ(𝐱′_i𝛽))]$$ Then, the Hessian is $$H = E\left(\...
4 votes
2 answers
175 views

Marginal distribution of an autoregressive process of order one AR(1)

I'm reading "Econometric Modelling with Time Series" by V. L. Martin, A. S. Hurn and D. Harris ( https://www.researchgate.net/file.PostFileLoader.html?id=56bccdaa6225ff0de28b45a6&...
  • 45

1
2 3 4 5
65