Questions tagged [maximum-likelihood]
a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.
3,180
questions
0
votes
0
answers
9
views
Hessian of the probit model
I was trying to compute the Hessian of the probit model.
Recall that the log likelihood is
$$\log𝐿=∑_{i=1}^{n}[𝑦_i \logΦ(𝐱′_𝐢𝛽)+(1−𝑦_𝑖)\log(1−Φ(𝐱′_i𝛽))]$$
Then, the Hessian is
$$H = E\left(\...
- 1
4
votes
2
answers
135
views
Marginal distribution of an autoregressive process of order one AR(1)
I'm reading "Econometric Modelling with Time Series" by V. L. Martin, A. S. Hurn and D. Harris ( https://www.researchgate.net/file.PostFileLoader.html?id=56bccdaa6225ff0de28b45a6&...
- 43
0
votes
0
answers
17
views
Diagnostic Test needed for ARMAX Model Specification
I'm trying to fit an ARMAX model using forecast::Arima() which is using MLE (https://otexts.com/fpp3/arima-estimation.html). Thus, there is no need for normality test as required by OLS estimator. I ...
0
votes
0
answers
10
views
for linear regression, when is MLE the same as least squares? [duplicate]
Under what conditions is MLE the same as the least squares estimate for ordinary linear regression?
I have seen statements saying that these two methods are not entirely the same. But so far, using ...
- 8,327
2
votes
0
answers
46
views
Obtaining MLE of the parameter of exponential distribution
Let say I have a sample of size $n$ as $\{X_1,X_2,...,X_n\}$.
The sample points $X_i$ are integers, but each of them are actually integer ceiling of corresponding real number $\{Y_i\}$. For example if ...
- 777
1
vote
0
answers
24
views
Does maximizing the posterior minimize some kind of divergence?
It's known that maximizing the log-likelihood is equivalent to minimizing the Kullback-Leibler divergence between the model $q(x \mid \theta)$ and the unknown true data generating distribution $p(x)$:
...
- 290
0
votes
0
answers
28
views
MLE of the Pareto parameter $\theta$ and unbiasedness
Warning
The question is the second part of this question. The third and last part is found here.
Exercise
Let $X \thicksim Pa(\lambda,\theta)$ with density function:
$ f(x; \theta, \lambda) = \frac{\...
- 63
0
votes
1
answer
18
views
Inferring individual measurements from multiple sums of random samples
Given a set of approximately 100 equations, one for each sample $i \in [1, 100]$:
$$
s_i = \sum_j c_{ij} p_j
$$
where $s_i$ is the known and positive sum for sample $i$, and $c_{ij}, j \in [1, 1000]$ ...
- 1
1
vote
0
answers
30
views
How does maximum likelihood estimation from the Kalman filter work?
My understanding is
Step 1: You would run through the Kalman filter equations
with initial parameter values.
Step 2: After you run through the Kalman filter equations,
you will have innovations ...
- 105
1
vote
0
answers
55
views
Sufficiency of $S=\min\{X_1, ..., X_n\}$ when $X \thicksim Pa(\lambda,\theta)$
Warning
The question is the sequel to this question and it was divided into three parts.
The first is this, while the second part is found here.
Exercise
Let $X \thicksim Pa(\lambda, \theta$) with ...
- 63
2
votes
0
answers
28
views
Why likelihood is product of probabilities? [duplicate]
I understand following
when we say probability we mean probability that a random variable $X$ will have certain value $x_i$ given the parameter $\theta$ that defines underlying probability ...
- 151
2
votes
1
answer
70
views
Use of weights in choosing power parameter in Tweedie distribution
I'm looking at the implementation of the tweedie.profile function from the R package tweedie. I have a few questions.
When ...
- 1,251
0
votes
0
answers
23
views
Normal Distribution for two groups mean estimation [closed]
Two groups $i=1,2$ mean estimate for normal distribution.
$$ f(y_i | \mu, \sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left\{ \frac{(y_i - \mu_i)^2 }{2\sigma^2}\right\}$$
I would like to find the ...
- 111
1
vote
2
answers
63
views
Observed Fisher information for the binomial: How is $I(\hat{\theta}) = \frac{n}{\hat{\theta}(1 - \hat{\theta})}$ calculated?
I am currently studying the textbook In All Likelihood by Yudi Pawitan. Example 2.10 of chapter 2.5 Maximum and curvature of likelihood says the following:
Example 2.10: Based on $x$ from the ...
- 1,334
2
votes
1
answer
76
views
Justification of the fixed variational distribution in diffusion models
Diffusion models can be regarded as latent variable models (Ho et al., 2020; Section 2), with the latents being an hierarchical chain of random variables $z_T → \dots → z_t → z_{t-1} → \dots → z_1$ (...
- 155
1
vote
0
answers
20
views
How to estimate parameters of a nonlinear polynomial model by using Maximum Likelihood Estimation?
Assume that you are having an equation that looks like this. It's a Box-Jenkins model.
$$F(q)D(q)y[k] = B(q)D(q)u[k] + F(q)C(q)e[k]$$
Where $F(q), D(q), B(q), C(q)$ are polynomials such as:
$$B(q) = 1 ...
- 193
0
votes
0
answers
15
views
Selecting a forecasting model with AIC: some questions
I'm using maximum likelihood estimation to fit a model to time series data. I later want to use the model to do forecasting. Let $L$ be the likelihood function,
$$L_i = L(f_k(\theta), data_i).$$
It ...
- 21
2
votes
0
answers
19
views
Aikaike Information Criterion for model with fixed parameter
I am trying to fit an inhomogenious, reinforced Poisson process to time series data using maximum likelihood estimation. The inhomogenious rate is
$$ \lambda = \alpha \cdot f(t,\theta) \cdot (m +n).$$
...
- 21
0
votes
0
answers
17
views
Correct approach for multilateration planimetric survey
Not being a statistician I apologize in advance. With multilateration I mean a 2D geometric grid of distances measured with a laser-meter (affected by a random error), such as the one below:
I ...
- 101
1
vote
1
answer
30
views
Interpretation of weights in a GLM
I want to know whether my interpretation of GLM weights is correct.
On R documentation of GLM it says that
Non-NULL weights can be used to indicate that different observations have different ...
- 13
0
votes
0
answers
22
views
What is the data $p_{data}$ and probability of the model $p_{model}(\textbf{x}, \boldsymbol{\theta})$ in the deep learning book?
In the Maximum Likelihood section of the Deep Learning Book (Section 5.5), the dataset (examples) is denoted by $\mathbb{X} = \{\boldsymbol{x}^{(1)}, \cdots, \boldsymbol{x}^{(m)}\}$. Note here the ...
- 1
0
votes
0
answers
22
views
Likelihood ratio for partitioned regression
Normally we have $ y= X \beta + u $ and we find $ \hat{\beta}_{MLE} $
and $\hat{\sigma^2}_{MLE}$ from the function of:
$$
l(\theta,y) = -\dfrac {n}{2}\cdot \ln
(2 \pi) -\dfrac {n}{2} \cdot \ln
(\...
- 21
0
votes
0
answers
19
views
Maximum likelihood estimator - conditioning input on model parameters
Assume we have a dataset $D=\{(x_i, y_i)\}$ drawn from the joint distribution $(x_i, y_i) \sim P (x=x_i, y=y_i)$. We want to make predictions based on the dataset and for that we use a parametric ...
- 207
4
votes
1
answer
143
views
Seemingly contradiction: probability density function and maximum likelihood calculations for continuous random variable
Claim 1: For continuous random variable, $P(X=x)=0$, where $x$ is a particular number.
Claim 2: When we use maximum likelihood estimation, we plug-in mean, standard deviation and data point $x$ into ...
- 355
2
votes
4
answers
227
views
Use of Relative Likelihoods in Statistics?
I am reading this article here on Confidence Intervals for Binomial Proportions (https://www.scirp.org/pdf/ojs_2021101415023495.pdf).
In this article, the author lists several different methods (e.g. ...
- 6,916
0
votes
0
answers
19
views
Is this a correct way of doing maximum likelihood estimation of parameters of a variance analysis model?
I would like to ask you if my way of doing maximum likelihood estimation of the parameters of a variance analysis model with one factor with K categories knowing that the models verifies the ...
- 13
6
votes
0
answers
726
views
Why do we minimise a cost function instead of maximising an equivalent? [duplicate]
I don't really understand why we minimise a cost function for gradient descent. Why don't we try to have something like a gradient 'climb', where we maximise some function?
Is it due to convention, or ...
- 61
0
votes
0
answers
36
views
Finding the standard deviation of the Maximum Likelihood Estimate
I have posted essentially the same question on reddit, but it has been some time now, and I have not yet received an answers, so I am also posting it here.
I have five measurements of speed $v_i\pm\...
- 101
0
votes
2
answers
53
views
Heavy vs light tail distributions when modelling with outliers
I am reading this lecture notes on using the MLEs from other distributions (as Laplace) rather than a Gaussian when dealing with outliers. The lecture notes came from Oxford University: https://www.cs....
- 8,327
6
votes
1
answer
297
views
What does it mean by "maximum likelihood estimation (MLE) problem is unbounded"?
I saw the following statement in this paper and wanted to understand its meaning:
- 401
2
votes
1
answer
67
views
How to reject a distribution given a sample?
Let's assume that we have the following vary basic problem:
We have a set of $N$ real numbers sampled from an unknown distribution.
We have a distribution that we consider as a hypothesis.
We would ...
- 435
2
votes
0
answers
36
views
Maximum likelihood vs generalized method of moments
I am trying to understand how maximum likelihood (MLE) and generalized method of moments (GMM) are related to each other. In particular, I often see people saying that MLE can be written in terms of ...
- 83
0
votes
0
answers
23
views
Likelihood Function for the Two-Parameter Exponential Distribution with Interval-Censored Data
Suppose three similar items fail. For two of them we observe the exact time of failure: time 10 and time 12. For the third, all we know is that it failed between times 8 and 9, inclusive.
Suppose ...
- 278
0
votes
0
answers
38
views
How do I impose restrictions $ 0\leq \alpha \leq \beta <1$?
I want to restrict values s.t. I get $\theta = (\alpha, \beta) =g(\theta_1, \theta_2)$
with the following restrictions.
$
0\leq \alpha \leq \beta <1
$
I know the correct answer should be $\beta = (...
0
votes
0
answers
52
views
Smallest threshold for hypothesis test with asymptotic level alpha
Consider a distribution with parameter $\lambda$ that has density
$$f_\lambda(x)=\frac{x^4}{24\lambda^5}e^{\frac{-x}{\lambda}},x>0$$
Let $X_1,...,X_n$ be $n$ independent random variables drawn from ...
- 107
3
votes
1
answer
76
views
MLE in a logistic regression model
Assuming that the design matrix is of full rank, in non-degenerate cases of the logistic regression model, does the maximum likelihood estimator always exists and is always unique?
It would be really ...
- 107
0
votes
0
answers
14
views
Parameters of the Logit Model
I'm building a Logit model. The data that I have is as follows: $y_i$ (binary), $x_{i1}, x_{i2}, x_{i3}$ $i \in \{1, 2, ..., 50\}$. From what I could gather, it is necessary to maximise the likelihood ...
2
votes
1
answer
49
views
Derive Variance of MLE [duplicate]
I want to calculate the variance of the MLE of an iid sample $X_1,\dots,X_n$ if
$$
f(x)=\alpha x^{\alpha-1}, 0 \leq x \leq 1, \alpha >0
$$
The MLE is fairly easy calculated as:
$$
\hat{\alpha}=-\...
- 1,347
1
vote
0
answers
23
views
Maximum Likelihood with a sign restriction
Suppose that we have a log-likelihood function of five parameters and an observed data sample $y=[y_1,\ldots,y_N]$:
$$\mathcal{l}(\beta_1, \beta_2, \beta_3, \beta_4, \beta_5;\;y) =\log f_Y(y;\;\beta_1,...
- 31
2
votes
0
answers
16
views
Can we find some examples for inference distribution that MoM would be better than MLE? [duplicate]
For sample size large enough, we know that the Maximum likelihood estimator (MLE) is asymptotic efficient. So when we have two classical methods (MLE and method of moment estimators) for inference of ...
- 419
1
vote
0
answers
20
views
MLE for censored multivariate normal data?
Suppose I have $q$ censored covariates, say $X_1, \ldots, X_q$ which are all left-censored with censoring values $\mathbf{d} = d_1, \ldots, d_q$. If I assume that the $X_i$ are from some multivariate ...
- 391
0
votes
1
answer
44
views
Maximum Likelihood Estimation of custom piecewise distribution
Recently I've been working with the distribution described by the pdf below:
$$
f(x) =
\begin{cases}
\frac{2}{3(b-a)}(1+\frac{x-a}{x-c}) & \quad a<x<c \\
\frac{2}{3(b-a)}(1+\frac{b-x}{b-c}) ...
- 1
0
votes
0
answers
48
views
How to take 1000 samples from distribution X and then use MLE to prove they came from distribution X?
I am trying to do:
find 1000 points that represent samples from distribution X with parameters $(a,b,c,\ldots, d)$
be guaranteed that the MLEs for those 1000 points are $(a,b,c,\ldots, d)$ with $99\%$...
- 129
0
votes
1
answer
34
views
Math behind likelihood function for logit/probit models?
Given a model
$\hat{y_i} = F(x_i \hat{\beta}) + \epsilon_i$,
where F is some mapping function that could be, for example, the sigmoid function, and x is a row vector of your features
I see that
...
- 11
0
votes
0
answers
18
views
nakagami distribution
How can i do the parameter's estimation with nakagami distribution with fisher scoring ? i've problem with derive the log likelihood for the variabel of m.(i attach the picture with screenshot file)
0
votes
1
answer
26
views
Asymptotic covariance matrix of an ML estimator and Fisher information
Let $(Y_i, X_i)_{1\leq i \leq n}$ be i.i.d. such that $Y_i = (Y_i^A, Y_i^B)' \in \mathbb{R}^2$ and $X_i = (X_i^A, X_i^B)' \in \mathbb{R}^2$. Suppose that
$$
Y_i^A = X_i^A \beta_A + \epsilon_i^A,
$$
$$
...
- 113
0
votes
0
answers
19
views
Does my solution to an incorrect MAP problem mean anything?
I was doing maximum a posteriori estimation of the variance $v\equiv \sigma^2$ using samples $\{x_n\}$ from a normally distributed random variable $X \sim \mathcal{N}(0,1)$, where the variance has an ...
- 290
0
votes
0
answers
20
views
Help me express a tractable complete data log likelihood of this problem with hidden variables
Imagine the following scenario: a couple of friends(total nr of people=7) will all play 5 games.
At each game the player will ask an horiscope if he should use one out of two bernoulli distributions. ...
- 1
2
votes
1
answer
96
views
Proving that MLE is equal to LSE
Assuming that $\epsilon \sim N(0, \sigma^2)$
I want to prove that the maximum likelihood estimator is equal to the least squares estimator. Now, I think I'm very close to making the proof, I just need ...
- 339
1
vote
1
answer
39
views
Do nice properties of MLE still hold in Classical Linear Regression Model?
For simplicity, let's assume that we have the true DGP
$$y_i= \beta_0 + \beta_1x_i + \epsilon_i$$ where $\epsilon_i \sim N(0, \sigma^2)$ $(i = 1,2,...,n)$
Assume that these following usual assumptions ...
- 135