Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
2,291
questions
0
votes
1answer
7 views
Fitting mixture model on data with duplicate values
What is the correct procedure to fit finite mixture models on data with many duplicate values using EM? Let's say I have N(0,1) distributed data and try to fit a 2 component mixture using EM. There ...
0
votes
2answers
57 views
Why can't regression via Maximum Likelihood shrink coefficients to zero?
Why can't regression via Maximum Likelihood shrink regression coefficients to zero as in LASSO? Does shrinking coefficients to zero not give higher L-likelihood? Does the answer to my question have to ...
0
votes
0answers
17 views
Does the newton-raphson either find the maximum of the loglikelihood function or estimate the MLE and likelihood function? [closed]
Does using newton-raphson method or some other optimization method used in nlme package or mixed effects models actually find the maximum of the log-likelihood function's height or does it simply find ...
0
votes
0answers
5 views
Indefinite Hessian in Negative Binomial log-likelihood estimation
I have the following problem: I am estimating a Negative Binomial regression. In order to estimate the model, I implemented the Newton method from the scratch. I am optimizing the dual of the log-...
0
votes
1answer
18 views
Maximum likelihood and Minimizing Kullback–Leibler divergence to the ecdf?! (i.e.: finite sample statement?)
There is a known relationship stating that finding MLE is asymptotically the same is minimizing Kullback–Leibler divergence (see wiki here), or just the cross entropy. I'm wondering if there is a ...
0
votes
0answers
32 views
Difference Between $L(θ | X=x)$ and $P(θ | X=x) $? [closed]
Although statistically different, I feel like they both say the same thing?
$L(θ | X=x)$ = the probability that a sample provides support for particular values of a parameter in a parametric model.
...
1
vote
0answers
16 views
How does likelihood differ from probability? [duplicate]
i.e. Tossing a fair coin
$p_H =$ probability of heads $= 0.5$
$L(p_H | HH) = P(X = HH | p_H) $
$= 0.25$
Okay, right-hand of equation I understand,
$P(X = HH | p_H) = 0.25 =$ probability of "$X = ...
4
votes
1answer
78 views
When Bayesian estimator coincides with MLE
Assume that $\theta$ is an unknown parameter with prior belief $p(\theta)$ and that $X_1,...,X_n$ is an iid sample from $p(x|\theta)$.
What does it say about our prior distribution $p(\theta)$ when ...
3
votes
0answers
24 views
AIC and model selection with multimodel
For a simple example, I am doing a multi-model regression/likelihood estimation.
The data is $(y_1,y_2,x_1,x_2,x_3,x_4,x_5)$. The first model (A) consists with two regressions:
$y_1=e^{a_1x_1}+e^{...
1
vote
0answers
18 views
why the sigma(volatility) is so small when useing the maximum likelihood estimation for vasicek model
This is my monthly data(in percentile)-2year japan government bond's yield to marturity from year 2001 to year 2019.And my LL function comes from "Maximum likelihood estimation using price data of the ...
0
votes
0answers
37 views
Assumption of error of logistic regression [duplicate]
I know there are debates about whether the error exists and its distribution in the case of logistic regression.
Suppose we assume that the error term follows the logistic distribution. Are we ...
0
votes
0answers
9 views
Manual calculation of logit regression [duplicate]
I have been working on Logit regression and I don't understand how to find coefficients by hand. I have seen that its possible to do it using maximum likelihood but I don't have much background in ...
1
vote
0answers
21 views
Fisher matrix with penalty function
I am fitting a parametric model to human tracking data. Because my data is in part corrupted (see below why), as such I had to introduce a penalty function to my optimization algorithm.
The Fisher ...
0
votes
0answers
51 views
how to set the parameter constraints in maxLik() in R [closed]
question: how to use maxLik() to set the constraints? For example,the beta and alphh should >0, and sigma should >=0.0001 .The following is my code, and it doesn't work in mle<-maxLik(...),because ...
0
votes
1answer
37 views
Constrained MLE
How to write R-code for obtaining MLEs of a ,b and $\theta$ for the density function
\begin{equation}
f(x)=\theta(a+bx)e^{-(ax+\frac{1}{2} b x^2)}\left(1-e^{-(ax+\frac{1}{2} b x^2)}\right)^{\theta-1}
\...
-1
votes
0answers
24 views
Re-writing the Score Test Statistic [closed]
We all know that the formula of the score test is: $$ S = \frac{\{l'(\theta_0)\}^2}{E\{-l''(\theta)\}\bigg|_{\theta_0}} $$ where $l'$ and $l''$ are $log$ $likelihoods$
What I need to do is represent ...
1
vote
0answers
54 views
Show that $E[u(X_1)\mid\sum^n_{i=1}X_i=z]=\Phi \left( \frac{\sqrt n(c-z/n)}{\sqrt{n-1}} \right)$ [duplicate]
Let $X_1, ..., X_n$ be a random sample from the $N(\mu,1)$ distribution. First show that $$E \left[ u(X_1)\, \Big|\, \sum^n_{i=1}X_i=z \right] = \Phi \left( \frac{\sqrt n(c-z/n)}{\sqrt{n-1}} \right)\,,...
2
votes
0answers
31 views
Over “specification” of a statistics model?
For a simple example, I am fitting data with a likelihood function generated by a normal distribution. The first model is the normal distribution with two-parameters. The second competing model is the ...
0
votes
0answers
10 views
Maximum likelihood estimator for difference of parameters
We're comparing two different clinical treatments. For each patient we observe the pair $(Y_i,R_i)$, where $Y_i$ denotes if the treatment was successful ($Y_i=1$) or not ($Y_i=0$) and $R_i$ indicates ...
1
vote
0answers
23 views
Identifying original elements in a permutation of a pattern
In the pattern above, the blue peaks are known and are labeled $p_1, p_2, ... p_N$. The red line is the same pattern of peaks, but altered slightly. Given a blue pattern and red pattern, I want to be ...
0
votes
0answers
11 views
Newton-Raphson method to solve for dof when performing MLE of a multivariate Student-t distribution using EM
I am reading the derivation of EM algorithm to estimate the maximum likelihood of a multivariate Student-t distribution $\mathcal{T}(\mathbf{x} \vert \pmb{\mu}, \pmb{\Sigma}, \nu)$ in Kevin Murphy's ...
1
vote
0answers
13 views
Maximum A Poseriori - Regression - Dependence between feature coefficients and features?
I was going through some lectures on regression when I came across these statements.
Wouldn't the expansions, respectively, be $P(w|y,X) = \frac {P(y|w,X)*P(w|x)} {P(y|X)} $ and $ P(y,w|X) = P(y|w,X)...
0
votes
0answers
15 views
Request for Explanation: Deriving Probability Density Function of a Maximum Likelihood Estimator of a Uniform Distribution [duplicate]
I am reviewing some practice problems and have both a question and its solution but am struggling to understand them and am hoping someone can help me.
I am struggling to follow the logic for ...
2
votes
2answers
39 views
Maximum Likelihood Estimator with exponential noise
So I need a little help with this please. I'm given N measurements of a signal $Y_{i} = A + v_{i}, i = 1,...,N$, where $v_{i}$ is measurement noise with the exponential pdf $f_{v}(v) = e^{-v}, v \geq ...
0
votes
1answer
19 views
Compare return levels of fitted GPD using MLE in different R packages
This question is related to this post: Different quantiles of a fitted GPD in different R packages?
I want to constraint "potvalues" data to be in a period of 6 years, this is, 16 observations per ...
1
vote
0answers
21 views
would it be possible to pick a different likelihood model achieve the same posterior estimation?
Take the coin flipping example. When we decide to use the Bernoulli distribution to model a coin flip, of course with and without a conjugate prior would make some difference for estimation.
Would it ...
0
votes
0answers
19 views
Deriving the $p$-dimensional maximum likeliihood estimation
So I'm currently following a statistic course and we are deriving the maksimum likelihood estimator for a $n\times p$-dimensional normal disitribution. So for $\mathbf{X}=\begin{pmatrix} \mathbf{X}_1^*...
0
votes
0answers
17 views
Logistic regression: Equation for marginal effect at the mean
I am estimating the following logistic regression (binomial family) by maximum likelihood:
$$ \ln\left(\frac{Y}{1-Y}\right) = \beta_{0} + \beta_{1}D + \beta_{2}X + \epsilon$$
where D is a dummy. ...
5
votes
0answers
32 views
What are some of the robustness checks for the likelihood ratio test?
In the application of statistical methods in social science, one usually does a lot of robustness checks. If I got some publishable findings using LRT test by discrimination two theoretical models, ...
0
votes
0answers
20 views
What does Maximum Likelihood Estimation mean in Machine Learning? [duplicate]
I am wondering what Maximum Likelihood Estimation means to Machine Learning in terms of training a predictive model. I understand Machine Learning uses Maximum Likelihood Estimations for model ...
1
vote
0answers
39 views
Who performed the first Maximum Likelihood Estimation?
I am very interested in the historical development of statistical theories. Here is the research I've done: I've tried to read two old papers of Fisher. I think the first theory paper on MLE should be ...
18
votes
5answers
1k views
What regression/estimation is not a MLE?
I just rigorously learned that OLS is a special case of MLE. It surprises me because the popular and "reliable" sources such as researchgate and this do not mention this most important connection ...
0
votes
0answers
23 views
Expected value and Maximum Likelihood Estimation
I'm doing this exercise about Poisson distribution and maximum likelihood estimation:
I have had no problem with points a and b, but I'm struggling with the correct answers of the C part.
From my ...
0
votes
0answers
21 views
Can I use maximum likelihood to optimize a scaling factor for a composite independent variable?
Basically, I am wondering if it is possible to use some kind of likelihood approach to optimize a scaling coefficient for an independent variable. I have the following standard linear model:
$y = B_0 ...
1
vote
0answers
22 views
Parameter estimation using data and model [duplicate]
For the equation $A=Bw+C(1-w)$, I would like to find the most likely value for $w$.
Each coefficient A, B an C has an x and y coordinate and uncertainty (x,y; sd). For example, say I have the ...
1
vote
0answers
5 views
Effect of heteroskedasticity in hierarchical (non-)linear models
Unlike linear models estimated via OLS where heteroskedasticity lead to inconsistency of the variance estimator but not the coefficient estimates, heteroskedasticity causes inconsistency of both ...
4
votes
0answers
25 views
When using L2 regularization outside of linear regression, do the same MAP estimation assumptions hold?
Some context is shared below, and my question is bolded at the end.
In the linear regression setting, we learn model weights $\hat{\mathbf{w}}$ to make predictions $\mathbf{\hat{y}}$ from new samples ...
1
vote
0answers
44 views
Statistical estimators distance for close empirical distributions
Is it valid to argue that two empirical distributions $ p_1, p_2 $ having small Wasserstein distance $W_r(\cdot)$ for an order $ r $ will yield close MLE estimators for a statistical model ...
2
votes
1answer
139 views
Maximum Likelihood Fit for Non-Linear Regression
I am reading the blog on Bayesian priors and overfitting and it mentions that assume that a data is generated by the following function:
$$y_t=\sin\left(\dfrac{x_t}{10}\right)+ \cos\left(z_t\right)+...
1
vote
0answers
29 views
Extend likelihood equation to P(Y>=y) in R
This question involves both math and coding in R. Apologies if this should be on Stack Overflow, but I decided statisticians would ultimately provide better support....
3
votes
1answer
42 views
Bayesian random variables VS. MLE fixed variables
"Bayesian inference treats model parameters as random variables whereas frequentist inference considers them to be estimates of fixed, ‘true’ quantities." (Ellison 2004)
What does it mean in the ...
1
vote
0answers
35 views
“Explanation power” of a parameter [closed]
Suppose there are two practically similar statistical models with similar "breath": $L(\theta)$ and $M(a,b)$, where $\theta, a, b$ are parameters. When fitting with data, if we assume that $b$ is ...
0
votes
1answer
31 views
MLE $\hat{h(\mu)} = h(\hat{\mu})$ of $h(\mu) = var(Y_1) = \mu^2$
Question: Suppose Y1, · · · , Yn follows an Exponential distribution with $\lambda = \frac{1}{\mu}$. Derive the MLE $\hat{h(\mu)} = h(\hat{µ})$ of $h(µ) = var(Y_1) = µ^2$, and show that $h(\mu)$ is ...
1
vote
0answers
21 views
Logistic regression fitting methods clarification
Each book I read propose a different fitting method for Logistic Regression.
The general idea is to maximize this expression.
$$
Pr\left(\beta|y,X,M\right) = \frac{Pr\left(y|\beta,X,M\right) Pr(\...
3
votes
0answers
77 views
Parameter estimation when the likelihood function does not exist
The observations $Z_1,Z_2\cdots$ are i.i.d. We have
$$Z_k = \sum_{i=1}^\infty \frac{X_{ki}}{2^k}.$$
where the $X_{ki}$'s are i.i.d. with a Bernouilli$(p)$ distribution. If $p=\frac{1}{2}$ then $Z_k$ ...
0
votes
0answers
21 views
MLE vs MAP hypothesis, performance
I have a question regarding the performance of Maximum a posteriori vs Maximum Likelihood Estimation hypothesis.
If you only consider the training data. Can you then state that the MAP hypothesis ...
2
votes
2answers
83 views
Can an improper prior distribution be informative?
I have just worked through an example where, with an improper prior, the bayesian estimator equals the maximum likelihood estimator, leading me to believe that improper priors are uninformative. But ...
3
votes
1answer
46 views
Invariance of maximum likelihood estimates to rearrangements of parameters/constants in the model?
I know that maximum likelihood estimates are invariant to re-parametrization (https://stats.stackexchange.com/a/335368/267430). Is the MLE also invariant to rearrangements of the constants and ...
4
votes
1answer
46 views
Question about Casella and Berger's proof of MLE invariance
In Casella and Berger, p. 320, they have a proof of the invariance of the MLE. Let $g: \theta \mapsto \eta$ be a function. They define the induced likelihood as
$$
L^*(\eta \mid X) = \sup_{\{\theta: ...
0
votes
0answers
20 views
Two-sided one-parameter likelihood ratio test
Consider a single-parameter distribution and a two-sided test for hypothesis $\theta_0$. Does the likelihood ratio test just reduce to:
$$
\frac{\max_{\theta \in \Theta_0} \mathcal{L}(\theta)}{\max_{\...
