Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,211
questions
2
votes
1
answer
12
views
Likelihood estimation with 2 samples from a geometric distribution
Context
Given there are 2 groups that can be modelled as a geometric distribution as follows:
\begin{align*}
f(x_i;p_1) &= p_1(1-p_1)^{x_i - 1} \; x_i = 1,2,... \; 0<p_1 <1 \\
f(y_i;p_2) &...
- 23
1
vote
1
answer
78
views
Maximum of a distribution not invariant in different coordinate systems?
Let's say we have a probability distribution in x,y space:
$$
p(x, y)=\frac{1}{4 \pi} \sqrt{x^2+y^2} \exp \left(- \sqrt{x^2+y^2}\right)
$$
This can be converted into polar coordinates as:
$$
p(r, \...
- 11
0
votes
1
answer
68
views
MLE phi derivation
Let dataset $D = \{(x_1,y_1),...,(x_n,y_n)\}$ where $x\in\mathbb{R}^d$ and $y_i\in\{0,1\}$
There are 2 mean vectors $\mu_0,\mu_1\in\mathbb{R}^d$ that represent the means of each feature split by label....
- 13
1
vote
0
answers
26
views
Asymptotic normality of penalized MLE?
Let $L(\theta;X)$ denote the log-likelihood of a model and I maximize the following to estimate $\theta$,
$$
\arg\max_{\theta}L(\theta;X)+\lambda\theta^2
$$
If $\lambda$ is 0, then the asymptotic ...
0
votes
1
answer
36
views
Use the delta method to find confidence intervals
Given $X_1 ... X_n \sim \textrm{Exp}(\lambda)$, I found the MLE : $$\hat{\lambda} = \frac{1}{\bar{X}}$$
Now I need to find confidence intervals for: $$\eta = \lambda \cdot \log(\lambda)$$
To do so, I ...
- 413
1
vote
0
answers
43
views
The confusing derivation in the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy
In the section 15.5 of the book 'Machine Learning: A Probabilistic Perspective' by Kevin P. Murphy, it discusses the Gaussian Process Latent Variable Model. The log-likelihood objective function is ...
- 11
0
votes
0
answers
14
views
How to solve non-identifiability problem in point estimation
I am working with a normal model $X \sim N(0, \sigma^2(\theta))$, where $\sigma^2(\theta) = \frac{1}{e}\cos^2(\theta)+e\sin^2(\theta)$. My goal is to estimate $\theta$ within the range $[0, 2\pi]$.
My ...
- 63
0
votes
0
answers
30
views
Maximum simulated likelihood of binary logit model implementation in python
I am working through the "Microeconometrics and MATLAB" book, translating the code to Python.
Here is my implementation for estimating the parameters of a binary logit model (Colab link). I ...
- 237
2
votes
1
answer
50
views
Help Deriving Likelihood Term When the Target is Known Probabilistically
I am trying to model data $\{Y_t,Q_t\}_{t=1}^T$, where the model is parameterized by $\theta$. $Y_t$ is a quantity where the model prediction can be solved in closed form, $\hat{Y}_t(\theta)$, where ...
2
votes
0
answers
61
views
Problem with the Fisher information matrix in case of N measurements of two observables
Let consider two observables, $x$ and $y$. Suppose that $y$ depends on the independent variable $x$ through the model $m(x; \boldsymbol{\theta})$, where $\boldsymbol{\theta}$ is a vector of model ...
- 21
10
votes
2
answers
468
views
Is quantile regression a maximum likelihood method?
Quantile regression allows to estimate a conditional quantile for y (like e.g. the median of y,...) from data x.
I do not see any distributional assumptions about y being made. This seems in contrast ...
- 431
0
votes
0
answers
25
views
Is the EM algorithm guaranteed to converge if the log likelihood is concave
As the EM algorithm is guaranteed to increase the log likelihood at each iteration. If the log likelihood is concave is it guaranteed to converge to the maximum of the likelihood, that is will we get ...
- 55
1
vote
0
answers
13
views
If the most likely value is that which minimizes squared-error, what are the possible distributions?
Gauss uniquely characterised the 1D normal distribution by asking for a distribution that:
is symmetric
is decreasing on either side of some center point $\mu$
has the data likelihood maximized by ...
- 121
-1
votes
0
answers
38
views
What are the good starting values in this question?
since when x = 1 the mean value of y is 200, so we have $\frac{\theta_1}{\theta_2+1} \approx 200$?
and does $Y_i \sim N(\frac{\theta_1+x_i}{\theta_2 x_i},\sigma^2)$ mean var(y) is a good estimation of ...
- 411
1
vote
2
answers
118
views
How to calculate the conditional maximum likelihood of independent negative binomial variables conditioned on the likelihood of the sum?
In the paper Small-sample estimation of negative binomial dispersion, with applications to SAGE data, section 4.1 Conditional maximum likelihood:
For RNA sequencing data, assume the counts for a ...
- 53
0
votes
0
answers
27
views
Optimising Box-Cox lambda analytically
I'm taking a university course in statistics where the Box-Cox transform is being discussed. As I understand it, we assume that there is some $\lambda$ that makes the sample normally distributed after ...
- 101
2
votes
1
answer
34
views
To estimate the parameters of P0, P1, P2 and P3 of a functional response equation by MLE?
this is equation of functional response following polynomial logistic regression with binomial distribution Na=N (exp p0 + p1 N + p2 N^2 + p3 N^3 ) /(1 + exp (p0 + p1 N + p2 N^2 + p3 N^3)). I would ...
2
votes
0
answers
29
views
Confidence interval for the quantiles given interval for parameter
Let us say we have a random sample $X_1,...,X_n\sim F(\theta)$ and we do inference over $\theta$ and give a maximum likelihood estimate $\hat{\theta}$ and a confidence interval at the $\alpha$% given ...
- 268
1
vote
0
answers
13
views
I suspect that the MLE of the scale parameter in scale families of distributions is scale equivariant. Any derivative-free proof? [closed]
Suppose $f(x)=c^{-1}f_0(x/c)$, $c,x>0$. Then the MLE $\hat c_n(X_1,...,X_n)$, satisfies$$\hat c_n(aX_1,...,aX_n)=a\hat c_n(X_1,...,X_n),$$ for all $a>0.$
- 11
0
votes
1
answer
69
views
Estimate parameters in Brownian Motion with drift, $dX_t = \mu dt + \sigma dW_t$
Consider a Brownian Motion with drift, $X$, on the interval $[0; T]$ given by
$$dX_t = \mu dt + \sigma dW_t.$$
Suppose that the interval is split into $n$ pieces of equal size to define $\Delta:=T/n$ ...
- 103
7
votes
2
answers
444
views
Why does the MAP differ from the MLE for the uniform prior in Laplace's Rule?
Laplace's Rule of Succession produces an estimate for the probability $p$ of a Bernoulli distribution. It starts with a $Beta(1,1)$ prior (equivalent to a uniform distribution prior on $(0,1)$), and ...
- 217
2
votes
0
answers
58
views
Estimation of parameters for a random effect model
I want to fit a simple one-way random effect model, i.e.
\begin{equation}
y_{ij} = \mu + \alpha_i + \epsilon_{ij},
\end{equation}
where
$\mu$ denotes the mean
$\alpha_i$ the effect of the $i$-th ...
- 21
0
votes
1
answer
54
views
Find the maximum likelihood estimator of $\theta$ with pdf $f(x)=2x/\theta^2$ [duplicate]
Let $(X_1,\dots, X_n)$ be a random sample from $X$ with pdf $f(x)=2x/\theta^2$ for $0\le x\le \theta$ where $\theta>0$. Find the maximum likelihood estimator of $\theta$.
The likelihood function ...
- 557
1
vote
1
answer
40
views
Estimating Markov chain transition probabilities from data
In a discrete time and space Markov chain, I know the formula to estimate the transition probabilities $$p_{ij} = \frac{n_{ij}}{\sum_{j \in S} n_{ij}}$$ I'm not sure however how you can find this is ...
- 13
0
votes
0
answers
46
views
MLE from a bivariate distribution
I have a sample of size $n$ from a bivariate distribution which have joint pdf with below form
$f_X(x)f_Y(y) \left[1+\lambda\left(2F_X(x)-1\right)\left(2F_Y(y)-1\...
- 343
0
votes
1
answer
42
views
Maximum likelihood fit of left truncated Weibull distribution
I want to fit some samples to the right tail of a Weibull distribution. To fix the notation:
the samples are $\{X_i\}_{i=1,\ldots,n}$,
all samples are greater than a fixed threshold $L$,
the $X_i$'...
- 195
1
vote
0
answers
23
views
Writing Likelihood in terms of Errors?
In the context of Survival Models (e.g. AFT https://en.wikipedia.org/wiki/Accelerated_failure_time_model), I have usually seen the Likelihood Function written as follows (https://hal-univ-pau.archives-...
- 7,028
0
votes
1
answer
79
views
Maximum Likelihood with heteroskedasticity
Suppose I regress $y$ on an explanatory variable $x$, so $y= a+bx + e$ where $e \sim N(0,g(x))$. This means the errors are normally distributed but show heteroskedasticity.
In my model, $x$ is a ...
- 55
0
votes
1
answer
36
views
Maximum likelihood estimate for mixture with components using both cartesian and polar coordinates
I have a set of points (x,y) that were generated from a mixture of two components: one component uses Cartesian coordinates, and the other polar coordinates.
For example, with probability $\gamma$ I ...
- 15
0
votes
0
answers
26
views
What is the influence of multicolinearity on the likelihood ratio test?
I learned the two concepts separately and I try to find out how these concepts relate to each other.
To illustrate the question introduce three models. The models establish a relation between age (AGE)...
- 201
0
votes
0
answers
29
views
Estimation of MLE - request for clarification
I am refering to an earlier post in Maximum Likelihood estimate of cell radius from observed cross section radii
I am not sure if I understood this question. What exactly is the meaning of ...
1
vote
0
answers
55
views
Solving the Neyman-Scott problem via Conditional MLE
In section 2.4 of the book Essential Statistical Inference by Boos and Stefanski, the authors discuss the idea conditional likelihoods and illustrate their usefulness by describing how they can be ...
- 1,348
1
vote
1
answer
53
views
Estimating mle of parameter of exponential distribution [closed]
I have a machine component whose life time is distributed as exponential with parameter $\lambda$
I switch on $n$ such components at time 0, then observe their performances during time $t$ and $t+z$ ...
- 343
3
votes
1
answer
117
views
A proof for existence of MLE
Suppose that for $\theta ∈ (0, 1)$, $X$ is a continuous random variable
with density
$f_\theta(x) =
\frac{3(1 − \theta)}
{4\delta^3(\theta)} [\delta^2(\theta) − (x − \theta)^2]\mathbb{1}(|x − \theta| ≤...
- 231
1
vote
0
answers
24
views
MLE of p in Bernoulli(p) where the range is specified
Came across this where b is binomial (the distribution is Bernoulli(p)). There's only one sample here, and I found that the MLE of p is X itself (by equating the first derivative to 0). I think I'm ...
- 207
0
votes
0
answers
14
views
Mixed logit and predicted shares
In Train 2009, pg. 323, he writes:
... maximum likelihood estimation
of a standard logit model with alternative specific constants for
each product in each market necessarily gives predicted shares ...
2
votes
1
answer
45
views
Likelihood for a log Gaussian Cox process (LGCP)
Suppose I have a log Gaussian Cox process (LGCP) $X$ with log intensity function $\lambda(x)=S(x)$ where $S$ follows a Gaussian process. Since LGCP still falls under the umbrella of inhomogeneous ...
- 81
0
votes
1
answer
49
views
Average log likelihood is maximized by a constant
Say I have the following data-generating process for a binary variable $y\in (0,1)$
$$\mathbb{P}(y=1\mid X) = \frac{1}{1+e^{-\beta X}}$$
where $\beta = (1,0.5,-1)$ and the ith variable $X_i \sim N(0,1)...
0
votes
1
answer
35
views
Least squares estimator for parameter p in binomial distribution
I am trying to find the least square estimator for the parameter $p$ in $Bin(n,p)$ but is it even possible? Isn't it the same as finding it using MLE?
- 1
0
votes
1
answer
39
views
MLE for Bernoulli parameter with constraint
We have $X \sim \text{bernoulli} \left(p \right), p \geq0.50$. Goal is to to estimate the MLE of $p$
With unconstraint case, I can calculate the MLE is the sample mean.
For constraint case, there is ...
- 343
0
votes
1
answer
37
views
Multinomial logit: why likelihood for one observation uses probabilities of all classes?
When dealing with non-binary discrete-choice outcomes, one common way of modeling such problems as a function of some covariates is through a multinomial logit/logistic model, in which there is one ...
- 537
4
votes
1
answer
113
views
(Why) is a logistic regression maximum likelihood estimator consistent?
A nice property of maximum likelihood estimators is that, while they can be biased, they are consistent for $iid$ observations.
In a logistic regression, unless the conditional distributions all have ...
- 46.6k
0
votes
0
answers
98
views
Hessian of the probit model
I was trying to compute the Hessian of the probit model.
Recall that the log likelihood is
$$\log𝐿=∑_{i=1}^{n}[𝑦_i \logΦ(𝐱′_𝐢𝛽)+(1−𝑦_𝑖)\log(1−Φ(𝐱′_i𝛽))]$$
Then, the Hessian is
$$H = E\left(\...
- 55
4
votes
2
answers
171
views
Marginal distribution of an autoregressive process of order one AR(1)
I'm reading "Econometric Modelling with Time Series" by V. L. Martin, A. S. Hurn and D. Harris ( https://www.researchgate.net/file.PostFileLoader.html?id=56bccdaa6225ff0de28b45a6&...
- 45
0
votes
0
answers
21
views
Diagnostic Test needed for ARMAX Model Specification
I'm trying to fit an ARMAX model using forecast::Arima() which is using MLE (https://otexts.com/fpp3/arima-estimation.html). Thus, there is no need for normality test as required by OLS estimator. I ...
0
votes
0
answers
13
views
for linear regression, when is MLE the same as least squares? [duplicate]
Under what conditions is MLE the same as the least squares estimate for ordinary linear regression?
I have seen statements saying that these two methods are not entirely the same. But so far, using ...
- 8,457
2
votes
0
answers
57
views
Obtaining MLE of the parameter of exponential distribution
Let say I have a sample of size $n$ as $\{X_1,X_2,...,X_n\}$.
The sample points $X_i$ are integers, but each of them are actually integer ceiling of corresponding real number $\{Y_i\}$. For example if ...
- 841
1
vote
0
answers
41
views
In MAP, does maximizing the posterior minimize any divergence between distributions?
It's known that maximizing the log-likelihood is equivalent to minimizing the Kullback-Leibler divergence between the model $q(x \mid \theta)$ and the unknown true data generating distribution $p(x)$:
...
- 290
0
votes
0
answers
73
views
MLE of the Pareto parameter $\theta$ and unbiasedness
Warning
The question is the second part of this question. The third and last part is found here.
Exercise
Let $X \thicksim Pa(\lambda,\theta)$ with density function:
$ f(x; \theta, \lambda) = \frac{\...
- 73
0
votes
1
answer
21
views
Inferring individual measurements from multiple sums of random samples
Given a set of approximately 100 equations, one for each sample $i \in [1, 100]$:
$$
s_i = \sum_j c_{ij} p_j
$$
where $s_i$ is the known and positive sum for sample $i$, and $c_{ij}, j \in [1, 1000]$ ...
- 1