Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,378
questions
0
votes
1
answer
694
views
Max Likelihood Estimator of the quantile
Let $(X_1, \dots, X_n) \sim Exp(\theta)$ and so
$$f(x; \theta) = \theta e^{-\theta x}$$
where $x>0$ and $\theta > 0$.
I need to find the quantile $q_p$ as a function of $\theta$, the max-...
- 145
3
votes
1
answer
56
views
Calculate mean if I only know random predecates of each sample
I'm not super experienced in statistics so sorry if some terminology is off.
I'm trying to find the mean of some distribution, call it $P$. The problem is, the samples aren't directly visible. For ...
2
votes
1
answer
1k
views
How to correctly address "ALERT: Iterations finished, maximum likelihood not found" in poLCA?
I intend to use Latent Class Analysis on a large dataset with 12 response categories and approximately 50,000 observations. I am getting an "ALERT: Iterations finished, maximum likelihood not ...
- 81
1
vote
0
answers
382
views
Maximum Likelihood Estimator of lambda with constraints?
I know that the Maximum Likelihood estimator for the parameter lambda in a poisson distribution is the sample mean.
However my understanding is that this is only the case when the only constraint on ...
- 53
3
votes
0
answers
48
views
MLE for the number of samples given $k$ largest values
I have the views on the top 100 videos using a tag in TikTok and want to estimate the total number of videos in that tag. I know the distribution for other tags so I can make a guess as to what it is ...
- 2,608
8
votes
1
answer
2k
views
Sample correlation is also a MLE estimator
On page 599 of this book, the author states (without proving) that for random samples $(X_1, Y_1)$, ..., $(X_n, Y_n)$ from a bivariate normal distribution, the sample correlation coefficient
\begin{...
- 101
3
votes
1
answer
412
views
Deriving Logit Maximum Likelihood Estimator
According to Verbeek, we can obtain the logit model by simplifying the first order condition of the log-likelihood function. Where,
$$logL(\beta) = \Sigma^N_{i=1} y_i logF(x^{'}_i\beta)+ \Sigma^N_{i=...
7
votes
1
answer
291
views
Obtaining the correct Log-likelihood function
$X_1, ..., X_n$ is a random sample from a population with pdf given by
$$
f(x; \mu, \lambda) = \frac{\lambda}{2}\operatorname{exp}(- \lambda |x - \mu|)
$$
where $\mu \in \mathbb{R}$ is the location ...
- 559
2
votes
0
answers
50
views
Maximum Likelihood Estimator for a given density function
I have the following problem: Assume you observe $Y_1,...,Y_N$ independently from the distribution $f_y$:
$$
f_{Y}(y)=\frac{12}{12-\theta}\left\{\begin{array}{ll}-\theta(y-0.5)^{2}+1 & \text { if ...
- 121
3
votes
1
answer
661
views
Relation between OLS, MM and ML
What is the relation between OLS, MM (method of moments) and ML (maximum likelihood)? During my studies, the three concepts got taught completely separated from each other. However, they seem to be ...
- 1,109
5
votes
2
answers
932
views
Under what conditions are Maximum Likelihood Estimation and Empirical Risk Minimization equivalent
I've seen some places(such as these lecture notes from ETH Zurich) where they simply declare MLE=ERM, but so far I haven't been able to find any good explanations (or, actually, any explanations at ...
- 51
2
votes
1
answer
103
views
How to find exponential lambda parameters to maximise n parts of series of events likelihood?
I have a series of events. I think there are n periods that make up this series. (I do not know the bounds of the periods). I assume that the delay between the events of each of these periods follows ...
- 55
0
votes
0
answers
258
views
MLE for Poisson Process
I asked a similar question here regarding method of moments, and now I need to figure out how to solve using MLE.
Here is a repeat of the problem:
A School of Ornithology researcher wants to estimate ...
- 447
1
vote
0
answers
135
views
Maximum Likelihood Estimator for area of circle
I have been working on the following question:
The radius of a circle, $R$, is measured with an error of measurement which is is normally distributed with mean $0$ and variance $\sigma^2$, given $n$ ...
- 11
0
votes
0
answers
19
views
Can you find the maximum likelihood, from a probability not conditioned on a random variable?
I have the following distribution, $$p(Y_i=y_i\mid \nu_i, 1000) = \hbox{Poisson} (y_i \mid\lambda \cdot \frac{\nu_i}{1000}) \quad \hbox{ for } \quad i=1,...,n$$
I am wondering if I can find the ...
- 315
6
votes
1
answer
2k
views
Can I find MLE of probability of X greater than x [duplicate]
such as find MLE of $P(X>12)$ when $X_i$ ~ $N(\mu,\sigma^2) $
- 125
4
votes
1
answer
140
views
How to find MLE of p^2q^3 when X~B(n,p)
How to find MLE of $$p^2q^3 $$when $$X_1,X_2,...,X_m $$ are iid random variable from B(n,p)
- 125
1
vote
0
answers
77
views
Maximum Likelihood Estimator : special case of uniform which exclude the upper limit [duplicate]
I would like to understand why we can't get a MLE in this special case of PDF :
"Sometimes it is not so easy to find the maximum of the likelihood function as in the examples above and one might ...
user226073
1
vote
2
answers
59
views
Least squared error on two distributions vs Maximum Likelihood estimator
Is there any difference in applying MLE vs squared error of two distributions in the following example?: Assume we have a series of points. We can use KDE to create a distribution. Furthermore, we ...
- 15
2
votes
1
answer
53
views
Why do we take mean of errors in linear regression?
I was reading about the probabilistic interpretation of linear regression and the following formula is derived using maximum likelihood estimates :
$$
\begin{align*}
β=\underset{β}{\text{argmin}}\sum_{...
0
votes
0
answers
201
views
Why is variance a fixed constant when modelling Linear Regression through MLE?
I was following a derivation of Linear Regression through MLE.
Here we model
Then we proceed to derive the negative log-likelihood through a series of steps as shown.
Finally we maximize the above ...
- 101
1
vote
1
answer
68
views
Precision of parameter fits in computational models
I have a model that transforms input data $X$ to output data $Y$ with some model parameters $p_1, .., p_n$. I simulate $n$ datasets from my model and for each dataset I reconstruct the parameters via ...
- 509
0
votes
1
answer
89
views
Simple notation question: pdf for mle of uniform?
I have simple notation question related to pdf for mle of uniform $U(0,\theta)$.
Given following pdf $f(\hat{\theta}_{MLE}) = \frac{n \cdot \hat{\theta}_{MLE}^{(n-1)}}{\theta^n} $ , I'm confused ...
- 251
1
vote
1
answer
245
views
Existence of least squares and maximum likelihood estimators?
In statistical parameter estimation where there is a deterministic and stochastic component to the observation-generating model, do least squares and maximum likelihood estimators always exist? ...
- 859
1
vote
0
answers
36
views
maximum likelihood estimation of the variance [closed]
In what situations the maximum likelihood estimation of the variance of distribution can severely ruin the estimation?
- 149
0
votes
1
answer
139
views
MLE not knowing exactly the sample data
I am learning statistics for Machine Learning and have to answer to this basic question from the following extract:
"Imagine a machine learning class where the probability that a student gets an '...
5
votes
1
answer
3k
views
Prove that the MLE exists almost surely and is consistent
I need to show that given an i.i.d sample $X_1,\dots X_n$ arising from the model:
$$\{f(x,\theta)=\theta x^{\theta-1}exp\{-x^{\theta}\},x>0,\theta\in (0,\infty)\}$$
that the MLE exists with ...
- 153
11
votes
2
answers
2k
views
Can we derive cross entropy formula as maximum likelihood estimation for SOFT LABELS?
For hard integer labels {0,1}, the cross entropy simplifies to the log loss. In this case, it is easy to show that minimizing the cross entropy is equivalent to maximizing the log likelihood, see e.g. ...
- 450
0
votes
1
answer
48
views
Conditional expectation when more than certain number of heads are observed out of 'n' trials
Suppose a coin is tossed $n$ times and it is then observed that the numbers of head (the success event) appeared is at least $k<n$. I'm interested in knowing the conditional expectation of no. of ...
3
votes
1
answer
2k
views
Showing bias of MLE for exponential distribution is $\frac{\lambda}{n-1}$
I want to show that the bias of $\hat \lambda = \frac{N}{\sum\limits_{i=1}^N x_i}$ is $\lambda/(n-1)$. There's a good chance that I'm too mathematically illiterate to understand the answer here ...
4
votes
1
answer
157
views
When is cross validation necessary to estimate a parameter?
In 2013, @Donbeo asked whether there were any theoretical results supporting use of Cross Validation to choose the lasso penalty, and was scolded in the comments for asking "a pretty generic ...
- 5,377
2
votes
0
answers
73
views
The inverse of the observed Fisher information matrix at the MLEs as an estimate of the var-cov matrix? [duplicate]
This may be a lower level question, so thank you in advance to any patient person who has an answer for me. I develop non-linear mixed effects models of pharmaceutical kinetics. One tool I'm ...
- 21
1
vote
0
answers
68
views
Maximum likelihood estimate of rate of relative risk aversion in R fails [closed]
I try to estimate the relative rate of risk aversion gmma of a CRRA function using R. The likelihood maximisation either does not converge or produces an unrealistic estimate. In order to make the ...
- 153
2
votes
0
answers
286
views
More on Fisher Information matrix with constraint
I have a similar Maximum Likelihood problem setup and a follow-up question to the question asked here
My constraints involve vector parameters $\vec{w}=\{w_1,w_2,\cdots,w_K\}$ and $\vec{\mu} = \{\mu_1,...
- 21
0
votes
1
answer
860
views
maximum likelihood and OLS - true or false [closed]
I was wondering about these three accusations, whether they are true or false and why is that?
The maximum likelihood estimation maximizes the sum of squared residuals.
Maximum likelihood estimation ...
- 11
3
votes
1
answer
956
views
Why maximizing the expected value of log likelihood under the posterior distribution of latent variables maximize the observed data log-likelihood?
I am trying to understand the Expectation-Maximization algorithm and I am not able to get the intuition of a particular step. I am able to verify the mathematical derivation but I want to understand ...
2
votes
0
answers
62
views
In practice how well does asymptotic normality of the MLE hold?
There is a lot of theory about asymptotic normality of the MLE and many people use the result to generate confidence intervals given finite sample data. But a key question here is how large a sample ...
1
vote
0
answers
50
views
GARCH estimation for subsamples
I would like to apply a GARCH(1,1) model for subsamples at time intervals length $k\delta$ on a stock return time series $\big(r(i\delta,(i+1)\delta)\big)_{i=0}^{kq-1}$ each element of which is the ...
- 995
1
vote
0
answers
46
views
Expected value of unbiased estimator of $\sigma$ in binomial sum
Suppose that $Y_1, Y_2, \dots, Y_r$ are random independent variable such as $Y_i \sim B(m_i, \pi)$, the idea first is to find $\hat{\pi}$ which is the maximum likelihood estimator an use it to find ...
- 121
1
vote
3
answers
515
views
What are the differences between the likelihood functions in Maximum Likelihood Estimation and in the Bayes' Theorem? [duplicate]
I am wondering the differences between the likelihood function in Maximum Likelihood Estimation and the likelihood function in Bayes' Theorem. To me, the likelihood function in Bayes' Theorem depends ...
- 3,465
5
votes
2
answers
2k
views
Why don't we estimate the prior in a Naive Bayes' classifier?
I'm currently studying the textbook Introduction to Machine Learning 4e (Ethem Alpaydin) the brush up on my ML basics and had a question regarding a part w.r.t. using the Naive Bayes' classifier in ...
- 3,917
1
vote
0
answers
41
views
Why are two likelihood functions equivalent if one is a positive multiple of another? [duplicate]
In page 298 of Rosenthal's Probability of statistics, it says:
We are only interested in likelihood ratios $\frac{L(\theta_1 | s)}{L(\theta_2 | s)}$ for $\theta_1, \theta_2 \in \Omega$ when it comes ...
- 131
1
vote
1
answer
53
views
How many sets do I have?
I have a jar with N objects that are grouped into sets of 3. If I randomly pull objects without replacement from this jar and my first five objects are from different sets while my sixth object is the ...
- 115
4
votes
0
answers
93
views
This mixed-type family of random variables has no dominating measure, so a likelihood function can't be defined?
Let
$$ X = \begin{cases}\theta & \text{with probability 1/2}\\ Z\sim N(0,1) & \text{with probability 1/2.} \end{cases}$$
Here, $\theta\in\mathbb{R}$ is the parameter to be estimated. It ...
- 141
1
vote
0
answers
515
views
The similarity between Mallows Cp and AIC?
It is possible to compute the log-likelihood used for AIC as $n /log(RSS/n) + const$ or even as $RSS/\sigma^2 + n\log(\sigma) + const$ considering the least-square or MLE scenario for linear and non-...
- 133
1
vote
0
answers
90
views
Can we choose any arbitrary vector to “shrink” towards for the James-Stein Estimator?
My understanding of the James-Stein estimator is that the choice of the origin as the point to shrink towards is more for neatness than anything else, and that the estimator still dominates MLE for ...
- 31
2
votes
1
answer
340
views
Log-likelihood of a exponential distribution
I have an exercise that I don't quite understand:
The life of 100 lamps has been measured. Each lamp has been used with a intensity between 0 and 1, where 0 is off and 1 is the maximum intensity. It ...
- 73
3
votes
1
answer
160
views
Coin flipping: Relationship between Bayesian and Frequentist's point estimates
I have a (biased) coin that has an unknown Head probability $p\in(0,1)$. To point estimate $p$, say that I'm going to use two approaches.
Approach 1. I can use the Bayesian inference technique. ...
- 413
4
votes
2
answers
495
views
How does Prior Variance Affect Discrepancy between MLE and Posterior Expectation
Suppose that $\theta\in R$ is a parameter of interest, $p(\theta)$ is our prior belief regarding $\theta$, and $\hat \theta$ is the MLE for theta derived from the data $x$. It is my understanding that ...
- 1,256
1
vote
0
answers
125
views
Unbiased estimator for a parameter from a transformed distribution
I am solving an exercise in which I have to show that a certain estimator is unbiased for a given parameter. However, after a couple lines of computation I got stuck in the following scenario:
$$
\...
- 37