Questions tagged [maximum-likelihood]

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Hessian of the probit model

I was trying to compute the Hessian of the probit model. Recall that the log likelihood is $$\log𝐿=∑_{i=1}^{n}[𝑦_i \logΦ(𝐱′_𝐢𝛽)+(1−𝑦_𝑖)\log(1−Φ(𝐱′_i𝛽))]$$ Then, the Hessian is $$H = E\left(\...
4 votes
2 answers
135 views

Marginal distribution of an autoregressive process of order one AR(1)

I'm reading "Econometric Modelling with Time Series" by V. L. Martin, A. S. Hurn and D. Harris ( https://www.researchgate.net/file.PostFileLoader.html?id=56bccdaa6225ff0de28b45a6&...
  • 43
0 votes
0 answers
17 views

Diagnostic Test needed for ARMAX Model Specification

I'm trying to fit an ARMAX model using forecast::Arima() which is using MLE (https://otexts.com/fpp3/arima-estimation.html). Thus, there is no need for normality test as required by OLS estimator. I ...
0 votes
0 answers
10 views

for linear regression, when is MLE the same as least squares? [duplicate]

Under what conditions is MLE the same as the least squares estimate for ordinary linear regression? I have seen statements saying that these two methods are not entirely the same. But so far, using ...
  • 8,327
2 votes
0 answers
46 views

Obtaining MLE of the parameter of exponential distribution

Let say I have a sample of size $n$ as $\{X_1,X_2,...,X_n\}$. The sample points $X_i$ are integers, but each of them are actually integer ceiling of corresponding real number $\{Y_i\}$. For example if ...
  • 777
1 vote
0 answers
24 views

Does maximizing the posterior minimize some kind of divergence?

It's known that maximizing the log-likelihood is equivalent to minimizing the Kullback-Leibler divergence between the model $q(x \mid \theta)$ and the unknown true data generating distribution $p(x)$: ...
  • 290
0 votes
0 answers
28 views

MLE of the Pareto parameter $\theta$ and unbiasedness

Warning The question is the second part of this question. The third and last part is found here. Exercise Let $X \thicksim Pa(\lambda,\theta)$ with density function: $ f(x; \theta, \lambda) = \frac{\...
0 votes
1 answer
18 views

Inferring individual measurements from multiple sums of random samples

Given a set of approximately 100 equations, one for each sample $i \in [1, 100]$: $$ s_i = \sum_j c_{ij} p_j $$ where $s_i$ is the known and positive sum for sample $i$, and $c_{ij}, j \in [1, 1000]$ ...
  • 1
1 vote
0 answers
30 views

How does maximum likelihood estimation from the Kalman filter work?

My understanding is Step 1: You would run through the Kalman filter equations with initial parameter values. Step 2: After you run through the Kalman filter equations, you will have innovations ...
  • 105
1 vote
0 answers
55 views

Sufficiency of $S=\min\{X_1, ..., X_n\}$ when $X \thicksim Pa(\lambda,\theta)$

Warning The question is the sequel to this question and it was divided into three parts. The first is this, while the second part is found here. Exercise Let $X \thicksim Pa(\lambda, \theta$) with ...
2 votes
0 answers
28 views

Why likelihood is product of probabilities? [duplicate]

I understand following when we say probability we mean probability that a random variable $X$ will have certain value $x_i$ given the parameter $\theta$ that defines underlying probability ...
  • 151
2 votes
1 answer
70 views

Use of weights in choosing power parameter in Tweedie distribution

I'm looking at the implementation of the tweedie.profile function from the R package tweedie. I have a few questions. When ...
  • 1,251
0 votes
0 answers
23 views

Normal Distribution for two groups mean estimation [closed]

Two groups $i=1,2$ mean estimate for normal distribution. $$ f(y_i | \mu, \sigma^2) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left\{ \frac{(y_i - \mu_i)^2 }{2\sigma^2}\right\}$$ I would like to find the ...
  • 111
1 vote
2 answers
63 views

Observed Fisher information for the binomial: How is $I(\hat{\theta}) = \frac{n}{\hat{\theta}(1 - \hat{\theta})}$ calculated?

I am currently studying the textbook In All Likelihood by Yudi Pawitan. Example 2.10 of chapter 2.5 Maximum and curvature of likelihood says the following: Example 2.10: Based on $x$ from the ...
  • 1,334
2 votes
1 answer
76 views

Justification of the fixed variational distribution in diffusion models

Diffusion models can be regarded as latent variable models (Ho et al., 2020; Section 2), with the latents being an hierarchical chain of random variables $z_T → \dots → z_t → z_{t-1} → \dots → z_1$ (...
1 vote
0 answers
20 views

How to estimate parameters of a nonlinear polynomial model by using Maximum Likelihood Estimation?

Assume that you are having an equation that looks like this. It's a Box-Jenkins model. $$F(q)D(q)y[k] = B(q)D(q)u[k] + F(q)C(q)e[k]$$ Where $F(q), D(q), B(q), C(q)$ are polynomials such as: $$B(q) = 1 ...
  • 193
0 votes
0 answers
15 views

Selecting a forecasting model with AIC: some questions

I'm using maximum likelihood estimation to fit a model to time series data. I later want to use the model to do forecasting. Let $L$ be the likelihood function, $$L_i = L(f_k(\theta), data_i).$$ It ...
  • 21
2 votes
0 answers
19 views

Aikaike Information Criterion for model with fixed parameter

I am trying to fit an inhomogenious, reinforced Poisson process to time series data using maximum likelihood estimation. The inhomogenious rate is $$ \lambda = \alpha \cdot f(t,\theta) \cdot (m +n).$$ ...
  • 21
0 votes
0 answers
17 views

Correct approach for multilateration planimetric survey

Not being a statistician I apologize in advance. With multilateration I mean a 2D geometric grid of distances measured with a laser-meter (affected by a random error), such as the one below: I ...
  • 101
1 vote
1 answer
30 views

Interpretation of weights in a GLM

I want to know whether my interpretation of GLM weights is correct. On R documentation of GLM it says that Non-NULL weights can be used to indicate that different observations have different ...
0 votes
0 answers
22 views

What is the data $p_{data}$ and probability of the model $p_{model}(\textbf{x}, \boldsymbol{\theta})$ in the deep learning book?

In the Maximum Likelihood section of the Deep Learning Book (Section 5.5), the dataset (examples) is denoted by $\mathbb{X} = \{\boldsymbol{x}^{(1)}, \cdots, \boldsymbol{x}^{(m)}\}$. Note here the ...
0 votes
0 answers
22 views

Likelihood ratio for partitioned regression

Normally we have $ y= X \beta + u $ and we find $ \hat{\beta}_{MLE} $ and $\hat{\sigma^2}_{MLE}$ from the function of: $$ l(\theta,y) = -\dfrac {n}{2}\cdot \ln (2 \pi) -\dfrac {n}{2} \cdot \ln (\...
0 votes
0 answers
19 views

Maximum likelihood estimator - conditioning input on model parameters

Assume we have a dataset $D=\{(x_i, y_i)\}$ drawn from the joint distribution $(x_i, y_i) \sim P (x=x_i, y=y_i)$. We want to make predictions based on the dataset and for that we use a parametric ...
  • 207
4 votes
1 answer
143 views

Seemingly contradiction: probability density function and maximum likelihood calculations for continuous random variable

Claim 1: For continuous random variable, $P(X=x)=0$, where $x$ is a particular number. Claim 2: When we use maximum likelihood estimation, we plug-in mean, standard deviation and data point $x$ into ...
  • 355
2 votes
4 answers
227 views

Use of Relative Likelihoods in Statistics?

I am reading this article here on Confidence Intervals for Binomial Proportions (https://www.scirp.org/pdf/ojs_2021101415023495.pdf). In this article, the author lists several different methods (e.g. ...
  • 6,916
0 votes
0 answers
19 views

Is this a correct way of doing maximum likelihood estimation of parameters of a variance analysis model?

I would like to ask you if my way of doing maximum likelihood estimation of the parameters of a variance analysis model with one factor with K categories knowing that the models verifies the ...
6 votes
0 answers
726 views

Why do we minimise a cost function instead of maximising an equivalent? [duplicate]

I don't really understand why we minimise a cost function for gradient descent. Why don't we try to have something like a gradient 'climb', where we maximise some function? Is it due to convention, or ...
  • 61
0 votes
0 answers
36 views

Finding the standard deviation of the Maximum Likelihood Estimate

I have posted essentially the same question on reddit, but it has been some time now, and I have not yet received an answers, so I am also posting it here. I have five measurements of speed $v_i\pm\...
  • 101
0 votes
2 answers
53 views

Heavy vs light tail distributions when modelling with outliers

I am reading this lecture notes on using the MLEs from other distributions (as Laplace) rather than a Gaussian when dealing with outliers. The lecture notes came from Oxford University: https://www.cs....
  • 8,327
6 votes
1 answer
297 views

What does it mean by "maximum likelihood estimation (MLE) problem is unbounded"?

I saw the following statement in this paper and wanted to understand its meaning:
  • 401
2 votes
1 answer
67 views

How to reject a distribution given a sample?

Let's assume that we have the following vary basic problem: We have a set of $N$ real numbers sampled from an unknown distribution. We have a distribution that we consider as a hypothesis. We would ...
  • 435
2 votes
0 answers
36 views

Maximum likelihood vs generalized method of moments

I am trying to understand how maximum likelihood (MLE) and generalized method of moments (GMM) are related to each other. In particular, I often see people saying that MLE can be written in terms of ...
  • 83
0 votes
0 answers
23 views

Likelihood Function for the Two-Parameter Exponential Distribution with Interval-Censored Data

Suppose three similar items fail. For two of them we observe the exact time of failure: time 10 and time 12. For the third, all we know is that it failed between times 8 and 9, inclusive. Suppose ...
0 votes
0 answers
38 views

How do I impose restrictions $ 0\leq \alpha \leq \beta <1$?

I want to restrict values s.t. I get $\theta = (\alpha, \beta) =g(\theta_1, \theta_2)$ with the following restrictions. $ 0\leq \alpha \leq \beta <1 $ I know the correct answer should be $\beta = (...
0 votes
0 answers
52 views

Smallest threshold for hypothesis test with asymptotic level alpha

Consider a distribution with parameter $\lambda$ that has density $$f_\lambda(x)=\frac{x^4}{24\lambda^5}e^{\frac{-x}{\lambda}},x>0$$ Let $X_1,...,X_n$ be $n$ independent random variables drawn from ...
3 votes
1 answer
76 views

MLE in a logistic regression model

Assuming that the design matrix is of full rank, in non-degenerate cases of the logistic regression model, does the maximum likelihood estimator always exists and is always unique? It would be really ...
0 votes
0 answers
14 views

Parameters of the Logit Model

I'm building a Logit model. The data that I have is as follows: $y_i$ (binary), $x_{i1}, x_{i2}, x_{i3}$ $i \in \{1, 2, ..., 50\}$. From what I could gather, it is necessary to maximise the likelihood ...
2 votes
1 answer
49 views

Derive Variance of MLE [duplicate]

I want to calculate the variance of the MLE of an iid sample $X_1,\dots,X_n$ if $$ f(x)=\alpha x^{\alpha-1}, 0 \leq x \leq 1, \alpha >0 $$ The MLE is fairly easy calculated as: $$ \hat{\alpha}=-\...
  • 1,347
1 vote
0 answers
23 views

Maximum Likelihood with a sign restriction

Suppose that we have a log-likelihood function of five parameters and an observed data sample $y=[y_1,\ldots,y_N]$: $$\mathcal{l}(\beta_1, \beta_2, \beta_3, \beta_4, \beta_5;\;y) =\log f_Y(y;\;\beta_1,...
2 votes
0 answers
16 views

Can we find some examples for inference distribution that MoM would be better than MLE? [duplicate]

For sample size large enough, we know that the Maximum likelihood estimator (MLE) is asymptotic efficient. So when we have two classical methods (MLE and method of moment estimators) for inference of ...
  • 419
1 vote
0 answers
20 views

MLE for censored multivariate normal data?

Suppose I have $q$ censored covariates, say $X_1, \ldots, X_q$ which are all left-censored with censoring values $\mathbf{d} = d_1, \ldots, d_q$. If I assume that the $X_i$ are from some multivariate ...
0 votes
1 answer
44 views

Maximum Likelihood Estimation of custom piecewise distribution

Recently I've been working with the distribution described by the pdf below: $$ f(x) = \begin{cases} \frac{2}{3(b-a)}(1+\frac{x-a}{x-c}) & \quad a<x<c \\ \frac{2}{3(b-a)}(1+\frac{b-x}{b-c}) ...
  • 1
0 votes
0 answers
48 views

How to take 1000 samples from distribution X and then use MLE to prove they came from distribution X?

I am trying to do: find 1000 points that represent samples from distribution X with parameters $(a,b,c,\ldots, d)$ be guaranteed that the MLEs for those 1000 points are $(a,b,c,\ldots, d)$ with $99\%$...
0 votes
1 answer
34 views

Math behind likelihood function for logit/probit models?

Given a model $\hat{y_i} = F(x_i \hat{\beta}) + \epsilon_i$, where F is some mapping function that could be, for example, the sigmoid function, and x is a row vector of your features I see that ...
0 votes
0 answers
18 views

nakagami distribution

How can i do the parameter's estimation with nakagami distribution with fisher scoring ? i've problem with derive the log likelihood for the variabel of m.(i attach the picture with screenshot file)
0 votes
1 answer
26 views

Asymptotic covariance matrix of an ML estimator and Fisher information

Let $(Y_i, X_i)_{1\leq i \leq n}$ be i.i.d. such that $Y_i = (Y_i^A, Y_i^B)' \in \mathbb{R}^2$ and $X_i = (X_i^A, X_i^B)' \in \mathbb{R}^2$. Suppose that $$ Y_i^A = X_i^A \beta_A + \epsilon_i^A, $$ $$ ...
  • 113
0 votes
0 answers
19 views

Does my solution to an incorrect MAP problem mean anything?

I was doing maximum a posteriori estimation of the variance $v\equiv \sigma^2$ using samples $\{x_n\}$ from a normally distributed random variable $X \sim \mathcal{N}(0,1)$, where the variance has an ...
  • 290
0 votes
0 answers
20 views

Help me express a tractable complete data log likelihood of this problem with hidden variables

Imagine the following scenario: a couple of friends(total nr of people=7) will all play 5 games. At each game the player will ask an horiscope if he should use one out of two bernoulli distributions. ...
  • 1
2 votes
1 answer
96 views

Proving that MLE is equal to LSE

Assuming that $\epsilon \sim N(0, \sigma^2)$ I want to prove that the maximum likelihood estimator is equal to the least squares estimator. Now, I think I'm very close to making the proof, I just need ...
1 vote
1 answer
39 views

Do nice properties of MLE still hold in Classical Linear Regression Model?

For simplicity, let's assume that we have the true DGP $$y_i= \beta_0 + \beta_1x_i + \epsilon_i$$ where $\epsilon_i \sim N(0, \sigma^2)$ $(i = 1,2,...,n)$ Assume that these following usual assumptions ...
  • 135

1
2 3 4 5
64