Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,211
questions
1
vote
0
answers
39
views
How does maximum likelihood estimation from the Kalman filter work?
My understanding is
Step 1: You would run through the Kalman filter equations
with initial parameter values.
Step 2: After you run through the Kalman filter equations,
you will have innovations ...
- 205
1
vote
0
answers
67
views
Sufficiency of $S=\min\{X_1, ..., X_n\}$ when $X \thicksim Pa(\lambda,\theta)$
Warning
The question is the sequel to this question and it was divided into three parts.
The first is this, while the second part is found here.
Exercise
Let $X \thicksim Pa(\lambda, \theta$) with ...
- 73
2
votes
0
answers
32
views
Why likelihood is product of probabilities? [duplicate]
I understand following
when we say probability we mean probability that a random variable $X$ will have certain value $x_i$ given the parameter $\theta$ that defines underlying probability ...
- 151
2
votes
1
answer
91
views
Use of weights in choosing power parameter in Tweedie distribution
I'm looking at the implementation of the tweedie.profile function from the R package tweedie. I have a few questions.
When ...
- 1,251
1
vote
2
answers
90
views
Observed Fisher information for the binomial: How is $I(\hat{\theta}) = \frac{n}{\hat{\theta}(1 - \hat{\theta})}$ calculated?
I am currently studying the textbook In All Likelihood by Yudi Pawitan. Example 2.10 of chapter 2.5 Maximum and curvature of likelihood says the following:
Example 2.10: Based on $x$ from the ...
- 1,364
2
votes
1
answer
102
views
Justification of the fixed variational distribution in diffusion models
Diffusion models can be regarded as latent variable models (Ho et al., 2020; Section 2), with the latents being an hierarchical chain of random variables $z_T → \dots → z_t → z_{t-1} → \dots → z_1$ (...
- 155
1
vote
0
answers
25
views
How to estimate parameters of a nonlinear polynomial model by using Maximum Likelihood Estimation?
Assume that you are having an equation that looks like this. It's a Box-Jenkins model.
$$F(q)D(q)y[k] = B(q)D(q)u[k] + F(q)C(q)e[k]$$
Where $F(q), D(q), B(q), C(q)$ are polynomials such as:
$$B(q) = 1 ...
- 193
0
votes
0
answers
17
views
Selecting a forecasting model with AIC: some questions
I'm using maximum likelihood estimation to fit a model to time series data. I later want to use the model to do forecasting. Let $L$ be the likelihood function,
$$L_i = L(f_k(\theta), data_i).$$
It ...
- 21
2
votes
0
answers
20
views
Aikaike Information Criterion for model with fixed parameter
I am trying to fit an inhomogenious, reinforced Poisson process to time series data using maximum likelihood estimation. The inhomogenious rate is
$$ \lambda = \alpha \cdot f(t,\theta) \cdot (m +n).$$
...
- 21
0
votes
0
answers
22
views
Correct approach for multilateration planimetric survey
Not being a statistician I apologize in advance. With multilateration I mean a 2D geometric grid of distances measured with a laser-meter (affected by a random error), such as the one below:
I ...
- 101
1
vote
1
answer
34
views
Interpretation of weights in a GLM
I want to know whether my interpretation of GLM weights is correct.
On R documentation of GLM it says that
Non-NULL weights can be used to indicate that different observations have different ...
- 13
0
votes
0
answers
23
views
What is the data $p_{data}$ and probability of the model $p_{model}(\textbf{x}, \boldsymbol{\theta})$ in the deep learning book?
In the Maximum Likelihood section of the Deep Learning Book (Section 5.5), the dataset (examples) is denoted by $\mathbb{X} = \{\boldsymbol{x}^{(1)}, \cdots, \boldsymbol{x}^{(m)}\}$. Note here the ...
- 3
0
votes
0
answers
22
views
Likelihood ratio for partitioned regression
Normally we have $ y= X \beta + u $ and we find $ \hat{\beta}_{MLE} $
and $\hat{\sigma^2}_{MLE}$ from the function of:
$$
l(\theta,y) = -\dfrac {n}{2}\cdot \ln
(2 \pi) -\dfrac {n}{2} \cdot \ln
(\...
- 21
0
votes
0
answers
24
views
Maximum likelihood estimator - conditioning input on model parameters
Assume we have a dataset $D=\{(x_i, y_i)\}$ drawn from the joint distribution $(x_i, y_i) \sim P (x=x_i, y=y_i)$. We want to make predictions based on the dataset and for that we use a parametric ...
- 217
4
votes
1
answer
148
views
Seemingly contradiction: probability density function and maximum likelihood calculations for continuous random variable
Claim 1: For continuous random variable, $P(X=x)=0$, where $x$ is a particular number.
Claim 2: When we use maximum likelihood estimation, we plug-in mean, standard deviation and data point $x$ into ...
- 355
2
votes
4
answers
275
views
Use of Relative Likelihoods in Statistics?
I am reading this article here on Confidence Intervals for Binomial Proportions (https://www.scirp.org/pdf/ojs_2021101415023495.pdf).
In this article, the author lists several different methods (e.g. ...
- 7,028
0
votes
0
answers
35
views
Is this a correct way of doing maximum likelihood estimation of parameters of a variance analysis model?
I would like to ask you if my way of doing maximum likelihood estimation of the parameters of a variance analysis model with one factor with K categories knowing that the models verifies the ...
- 13
6
votes
0
answers
728
views
Why do we minimise a cost function instead of maximising an equivalent? [duplicate]
I don't really understand why we minimise a cost function for gradient descent. Why don't we try to have something like a gradient 'climb', where we maximise some function?
Is it due to convention, or ...
- 61
0
votes
0
answers
41
views
Finding the standard deviation of the Maximum Likelihood Estimate
I have posted essentially the same question on reddit, but it has been some time now, and I have not yet received an answers, so I am also posting it here.
I have five measurements of speed $v_i\pm\...
- 101
0
votes
2
answers
72
views
Heavy vs light tail distributions when modelling with outliers
I am reading this lecture notes on using the MLEs from other distributions (as Laplace) rather than a Gaussian when dealing with outliers. The lecture notes came from Oxford University: https://www.cs....
- 8,457
6
votes
1
answer
324
views
What does it mean by "maximum likelihood estimation (MLE) problem is unbounded"?
I saw the following statement in this paper and wanted to understand its meaning:
- 401
2
votes
1
answer
75
views
How to reject a distribution given a sample?
Let's assume that we have the following vary basic problem:
We have a set of $N$ real numbers sampled from an unknown distribution.
We have a distribution that we consider as a hypothesis.
We would ...
- 465
2
votes
1
answer
109
views
Maximum likelihood vs generalized method of moments
I am trying to understand how maximum likelihood (MLE) and generalized method of moments (GMM) are related to each other. In particular, I often see people saying that MLE can be written in terms of ...
- 83
0
votes
0
answers
31
views
Likelihood Function for the Two-Parameter Exponential Distribution with Interval-Censored Data
Suppose three similar items fail. For two of them we observe the exact time of failure: time 10 and time 12. For the third, all we know is that it failed between times 8 and 9, inclusive.
Suppose ...
- 278
0
votes
0
answers
38
views
How do I impose restrictions $ 0\leq \alpha \leq \beta <1$?
I want to restrict values s.t. I get $\theta = (\alpha, \beta) =g(\theta_1, \theta_2)$
with the following restrictions.
$
0\leq \alpha \leq \beta <1
$
I know the correct answer should be $\beta = (...
0
votes
0
answers
59
views
Smallest threshold for hypothesis test with asymptotic level alpha
Consider a distribution with parameter $\lambda$ that has density
$$f_\lambda(x)=\frac{x^4}{24\lambda^5}e^{\frac{-x}{\lambda}},x>0$$
Let $X_1,...,X_n$ be $n$ independent random variables drawn from ...
- 107
3
votes
1
answer
91
views
MLE in a logistic regression model
Assuming that the design matrix is of full rank, in non-degenerate cases of the logistic regression model, does the maximum likelihood estimator always exists and is always unique?
It would be really ...
- 107
2
votes
1
answer
54
views
Derive Variance of MLE [duplicate]
I want to calculate the variance of the MLE of an iid sample $X_1,\dots,X_n$ if
$$
f(x)=\alpha x^{\alpha-1}, 0 \leq x \leq 1, \alpha >0
$$
The MLE is fairly easy calculated as:
$$
\hat{\alpha}=-\...
- 1,347
1
vote
0
answers
24
views
Maximum Likelihood with a sign restriction
Suppose that we have a log-likelihood function of five parameters and an observed data sample $y=[y_1,\ldots,y_N]$:
$$\mathcal{l}(\beta_1, \beta_2, \beta_3, \beta_4, \beta_5;\;y) =\log f_Y(y;\;\beta_1,...
- 45
2
votes
0
answers
18
views
Can we find some examples for inference distribution that MoM would be better than MLE? [duplicate]
For sample size large enough, we know that the Maximum likelihood estimator (MLE) is asymptotic efficient. So when we have two classical methods (MLE and method of moment estimators) for inference of ...
- 557
1
vote
0
answers
26
views
MLE for censored multivariate normal data?
Suppose I have $q$ censored covariates, say $X_1, \ldots, X_q$ which are all left-censored with censoring values $\mathbf{d} = d_1, \ldots, d_q$. If I assume that the $X_i$ are from some multivariate ...
- 437
0
votes
1
answer
53
views
Maximum Likelihood Estimation of custom piecewise distribution
Recently I've been working with the distribution described by the pdf below:
$$
f(x) =
\begin{cases}
\frac{2}{3(b-a)}(1+\frac{x-a}{x-c}) & \quad a<x<c \\
\frac{2}{3(b-a)}(1+\frac{b-x}{b-c}) ...
- 1
0
votes
0
answers
52
views
How to take 1000 samples from distribution X and then use MLE to prove they came from distribution X?
I am trying to do:
find 1000 points that represent samples from distribution X with parameters $(a,b,c,\ldots, d)$
be guaranteed that the MLEs for those 1000 points are $(a,b,c,\ldots, d)$ with $99\%$...
- 129
0
votes
1
answer
44
views
Math behind likelihood function for logit/probit models?
Given a model
$\hat{y_i} = F(x_i \hat{\beta}) + \epsilon_i$,
where F is some mapping function that could be, for example, the sigmoid function, and x is a row vector of your features
I see that
...
- 11
0
votes
0
answers
22
views
nakagami distribution
How can i do the parameter's estimation with nakagami distribution with fisher scoring ? i've problem with derive the log likelihood for the variabel of m.(i attach the picture with screenshot file)
0
votes
1
answer
42
views
Asymptotic covariance matrix of an ML estimator and Fisher information
Let $(Y_i, X_i)_{1\leq i \leq n}$ be i.i.d. such that $Y_i = (Y_i^A, Y_i^B)' \in \mathbb{R}^2$ and $X_i = (X_i^A, X_i^B)' \in \mathbb{R}^2$. Suppose that
$$
Y_i^A = X_i^A \beta_A + \epsilon_i^A,
$$
$$
...
- 113
0
votes
0
answers
19
views
Does my solution to an incorrect MAP problem mean anything?
I was doing maximum a posteriori estimation of the variance $v\equiv \sigma^2$ using samples $\{x_n\}$ from a normally distributed random variable $X \sim \mathcal{N}(0,1)$, where the variance has an ...
- 290
0
votes
0
answers
21
views
Help me express a tractable complete data log likelihood of this problem with hidden variables
Imagine the following scenario: a couple of friends(total nr of people=7) will all play 5 games.
At each game the player will ask an horiscope if he should use one out of two bernoulli distributions. ...
- 1
2
votes
1
answer
126
views
Proving that MLE is equal to LSE
Assuming that $\epsilon \sim N(0, \sigma^2)$
I want to prove that the maximum likelihood estimator is equal to the least squares estimator. Now, I think I'm very close to making the proof, I just need ...
- 389
1
vote
1
answer
40
views
Do nice properties of MLE still hold in Classical Linear Regression Model?
For simplicity, let's assume that we have the true DGP
$$y_i= \beta_0 + \beta_1x_i + \epsilon_i$$ where $\epsilon_i \sim N(0, \sigma^2)$ $(i = 1,2,...,n)$
Assume that these following usual assumptions ...
- 135
1
vote
1
answer
77
views
Derive the ML-estimation for the parameter $\beta$
I think I know in general how to derive the maximum likelihood estimation for a parameter given a distribution. But I can't wrap my head around this one!
We have the observations: $(y_1, x_1),...,(y_n,...
- 75
2
votes
2
answers
103
views
Estimating the parameters of a Poisson times a constant
Suppose that we have a distribution $X = aY$ where the distribution of $Y$ is Poisson{$\lambda$}, what is the maximum likelihood estimator of $a$?
By the method of moments we can estimate $a$ as $\...
- 145
1
vote
1
answer
94
views
How understand "All the methods of estimation are invariant under linear transformations of the data"?
In this paper, the authors compare different methods of fitting generalized extreme value distribution (such as the maximum likelihood method). For example, the Gumbel distribution:
$$
F(x)=\exp\left(-...
- 557
0
votes
0
answers
29
views
Do I need a "likelihood Jacobian" for point estimation?
Scenario 1
Suppose that you work in a lumber yard and are given the following measurements of the weights of various pieces of lumber:
x = [10, 26, 28, 13, 16, 7]
...
- 221
0
votes
0
answers
22
views
Maximum likelihood estimation normal
I am looking for the maximum likelihood estimators of the random sample with normal distribution $N(\mu,\gamma\mu)$. My question is whether from the conventional estimators, $\bar{X} $ and $S^2$, it ...
1
vote
1
answer
68
views
Maximum likelihood estimate for mixture of different distributions
I'd like to estimate the parameters of a mixture model using MLE. The density is:
$$
f(x,y) = \mathcal{N}(x, y; \boldsymbol{\mu}, \Sigma) \cdot \alpha + \mathcal{N}(x; \mu, \sigma^2) \cdot \mathcal{U}...
- 15
0
votes
0
answers
13
views
Goodness-of-fit and confidence intervals
I am having trouble grasping the subtleties of determining confidence intervals on parameters from degrading the goodness-of-fit (ie shifting $\chi^2$ or $2\ln {\cal L}$ by 1).
For instance, around ...
- 378
1
vote
3
answers
200
views
Deriving the asymptotic distribution using delta method
I have the density function:
$P_Y(y) = \sqrt{\frac{1}{2\pi y^3}} \exp\left(-\frac{(y-\mu)^2}{2\mu^2y}\right)$
If we define $r := \mu^2$ what is its asymptotic distribution?
The right answer is $\sqrt{...
- 75
1
vote
1
answer
65
views
What does it mean in Deep Learning, that L2 loss or L2 regularization induce a gaussian prior?
A prior for what?
Let's take a toy example of a Deep Neural Network with weights w being fit on a dataset D with an L2/Mean Square Error loss.
I looked up Maximum Likelihood Estimation (MLE) and ...
- 51
4
votes
3
answers
110
views
Estimating the parameter $\beta$
The lifetime of computer monitors has a exponential distribution where the expected value can be written as:
$\mu(s) = \frac{\beta}{s}$
Where $s$ is how bright the monitor is and both $s$ and $\beta$ ...
- 75