a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

0
votes
0answers
3 views

Problem with generalized likelihood ratio test from samples from beta distribution

I was trying to resolve this exercise: This exercise is from the book "Statistical Inference, Second Edition" by Casella and Berger. Checking the solutions manual. I was understanding the solution ...
1
vote
1answer
23 views

What are the (philosophical) assumptions behind GMM and Maximum Likelihood Estimation?

As stated in the question. In particular, how does a researcher know when to apply which estimation method and are there any examples that can show when one case is more appropriate than the other? ...
2
votes
0answers
18 views

Finding MLE with ordered statistics?

Let Y1 < Y2 < ... < Yn be the order statistics of a random sample of size n from the uniform distribution of the continuous type over the closed interval: $$[\theta - \rho, \theta + \rho]$$ ...
0
votes
0answers
11 views

Is the application of the Frisch-Waugh-Lovell Theorem really necessary?

Suppose I have a model \begin{eqnarray} y = X_1 \beta + X_2 \gamma + \epsilon \\ X = Z \Pi + V \end{eqnarray} where $X_1$ is endogenous, Z are instruments, $X_2$ are exogenous. If I however include ...
1
vote
1answer
11 views

Finding covariance matrix for MLE of correlated outputs

I generate data using the following model: $\begin{pmatrix}Y_1\\Y_2\end{pmatrix} \sim \mathcal{N}\left( \begin{pmatrix}\mathbf{X}\beta_1\\\mathbf{X}\beta_2\end{pmatrix}, \mathbf{\Sigma} \right)$ I ...
1
vote
1answer
15 views

How to find $\arg\max$ of a neural network?

Let's say I have a neural network $f$ that takes input $\vec x \in \mathbb {R}^n$ and produces output $f(\vec x) \in \mathbb{R}$. How can I find $\hat x = \underset{\vec x}{\arg\max} \; f(\vec x)$?
1
vote
0answers
16 views

Deming Regression/Errors-in-Variables with replicates

I have a question about a model related to Deming regression and would appreciate some help and/or publications to further study this model. Statistical Model: \begin{align} ...
0
votes
2answers
31 views

Computationally efficient Gaussian MAP estimation algorithm in MATLAB

I have a MAP estimation model for a Gaussian prior and i.i.d Gaussian noise: $$y=x+n$$ where $x\sim\mathcal{N}(0,\Sigma)$ and $n\sim \mathcal{N}(0,\sigma^2I)$. The MAP estimate is given by $$ ...
1
vote
0answers
32 views

State and parameter estimation: Stuck in maximizing the log likelihood function [on hold]

The model is an FIR system, $y_n = \mathbf{h^Tz_n} + w_n$ where $\mathbf{z_n} = [z_1,\ldots,z_{n-p-1}]$ is the input to the system which is a non-linear deterministic one dimensional map, ...
4
votes
1answer
35 views

Distributed Datasets and MLE

Suppose I have a very large dataset of size N, evenly distributed over M computers so that each computer has N/M data points. Suppose I want to fit a model using MLE that requires an iterative method. ...
1
vote
1answer
39 views

f(y | x) or f(y,x) in regression and MLE

In $Y = aX + b + \epsilon$ where $\epsilon$ ~ $N(0,\sigma^2)$ and i.i.d regression setting If X is stochastic and $E(\epsilon\mid X) =0$, then which one is correct: (1) $f(x,y) = ...
0
votes
1answer
36 views

Is LIML consistent under heteroskedastic errors?

Please let the answer be yes. Suppose we have a model \begin{eqnarray} y= X \beta + \epsilon \\ X = Z \Pi + V \end{eqnarray} and we compute the LIML estimator \begin{eqnarray} \hat{\beta}_{LIML} = ...
0
votes
0answers
11 views

Is this R code of Rao score test for the Bernoulli data model correct? [migrated]

I am a complete statistical noob and new to R, hence the question. I've tried to find an implementation of the Rao score for the particular case when one's data is ...
0
votes
0answers
16 views

Penalized ML estimation of non-linear probit

I have a model of the form $P(y_i=1) = \Phi(\frac{w_1^{\beta'x_i}-w_2^{\beta'x_i}}{\sigma' x_i})$ where $y_i$ is a binary response, $\Phi$ is the normal CDF, $w_1$ and $w_2$ are non-negative ...
0
votes
0answers
38 views

“All of Statistics” questions [closed]

I am not able to solve a couple of questions asked in the exercise 10.11 of book - All of Statistics. Question 1 Question 2 How do I start ? Any hints would be helpful ? I do not want complete ...
4
votes
1answer
28 views

Negative weights in maximum likelihood method?

In physics we like to use the maximum likelihood method to fit our models to our data. (I'm sure the first part of this post is review to you all, I just want to be complete so that you will know ...
0
votes
1answer
62 views

Optimizing parameter estimates by minimizing chi^2 in iterative procedure

I need to minimize my Chi^2 (bottom-left in figure 1) by adjusting parameter-values in a MLE-procedure (or something alike). The chi^2 (red) is a goodness-of-fit measure. It expresses how well the ...
0
votes
0answers
12 views

GARCH estimation, reduce outlier effect with t distribution

I am trying to estimate GARCH(1, 1) using MLE. As my data contains a lot of outliers, I think using t distribution to represent errors makes a lot of sense. But I am completely lost at how to ...
0
votes
1answer
25 views

Deriving the common LIML estimator from first principles

David Hendry (1976) comments that deriving the LIML estimator is hard. I tend to agree. Guido Imbens has a nice expression here which reads \begin{eqnarray} \hat{\beta}_{LIML} = (X'(I - \lambda M_Z) ...
4
votes
2answers
52 views

Expectation maxmisation algorithm increases true likelihood at each iteration

I've heard that the EM algorithm ensures that the true likelihood is non-decreasing at each iteration of the algorithm, but I'm not sure why this is the case. I've provided a basic plot which I ...
0
votes
1answer
28 views

MLE of Cauchy distribution in R

I am trying to compute the following "approximate Maximum Likelihood Estimate" in R. I am a little lost as to how to do this though: any hints would be appreciated, thanks
0
votes
0answers
42 views

maximum likelihood estimation with a programmed function

I am trying to do an exercise of MLE available in: http://people.missouristate.edu/songfengzheng/Teaching/MTH541/MLE-R.pdf it is the last exercise about gamma distribution, so far I have done the ...
0
votes
1answer
41 views

tobit likelihood functions with bivariate normal and unspecified error distributions

I have searched high and low for examples like this online but I cannot find it anywhere. This question is pulled from a PhD course in econometrics. Given $(y^*_1,y^*_2)$~ bivariate normal with the ...
0
votes
0answers
16 views

Approximate MLE of Cauchy Distribution in R

I am trying to compute the following "Approximate Maximum Likelihood" in R: $$ (\hat{\Psi},\hat{\Phi},\hat{\gamma})={Argmax}_{\Phi,\Psi,\gamma} \sum^{T-s}_{t=r+1} ln (g(A;\gamma)) $$ Assuming r = 2, ...
2
votes
1answer
28 views

mle estimate for standard deviation of t-distribution

For a Student-t distribution, $t_{\nu}\left(\mu,s^2\right)$, let $\hat{s}$ be mle of scale and $\hat{\nu}$ be the mle of degrees of freedom. Functional invariance of mle implies that any linear or ...
0
votes
0answers
31 views

MCMC: The posteriors are too narrow

I have this very simple MCMC there I have ...
0
votes
0answers
51 views

Maximum likelihood estimate parameters estimation

In this tutorial on mixture models, page 2, how did the author arrive to the parameters for maximum likelihood in the fully observed case? This is the general setting (based on an excerpt from the ...
3
votes
1answer
21 views

What is the solution to this minimization problem?

I'm encountering the following minimization problem in my research: $$\hat b = \underset{b}{\arg\min} \sum_i^n \left( \log \frac{a_i}{b} \right)^2$$ I could iteratively optimize, but I think that ...
0
votes
0answers
39 views

MLE of a linear function

So I have a function $$f(x\mid\theta) = \frac{1+x\theta}{2}$$ For the interval $-1\leq x\leq 1$. I need to find the mle of $\theta$ but the only way I've learned how is to take the derivative which ...
5
votes
2answers
144 views

A Question on Elementary Statistical Inference

A box contains $5$ white and $2$ black balls. A coin with unknown $P(Head)=p$ is tossed once. If it lands HEADS then a white ball is added, else a black ball is added to the box. Then a ball is ...
0
votes
0answers
10 views

How to obtain the covariance matrix for the parameters estimated by maximizing this log-likelihood of logit/probit

I have this linear latent model : $y^*=\beta_0+\beta_1 X+\epsilon,$ $y^*$ is latent (non observed) variable and $\epsilon\sim iid (0,\sigma^2 I)$ Consider : $y=1$ if $\beta_0+\beta_1 X>0$ and ...
0
votes
1answer
32 views

Relationship between Poisson generation and generalized Kullback-Leibler divergence

I have read that, in the context of matrix factorization, performing maximum likelihood estimation under the assumption that the entries are Poisson generated is equivalent to minimizing the ...
0
votes
0answers
13 views

To obtain Pearson type III parameters and shift value

How to obtain Pearson type III parameters and shift value? I am using R and if you can give me an instruction, it would be helpful. I have used pearsonFitML function from PearsonDS package, but I can ...
0
votes
0answers
16 views

I do not understand what it means to maximise a parameter of a population using the method of maximum likelihood estimator? [duplicate]

I understand the procedure of the method of maximum likelihood. I also understand the method is used to estimate a statistic within a population. However I do not understand what it means to maximise ...
0
votes
1answer
13 views

How to combine MLEs for a point process

I am estimating parameters of a point process by doing an MLE calculation using event time data. However I in fact have several distinct sets of data from different locations. I can estimate the ...
1
vote
0answers
15 views

finding an MLE with samples from binomial distribution

A box contains 5 white balls and 2 black balls. A coin with unknown $P(\text{Head}) = p$ is tossed. A white ball is added to the box if the outcome is head; otherwise a black ball is added. Then a ...
2
votes
0answers
18 views

Incorporating population priors into MLE fits with few/limited samples

I am fitting Beta distributions to data resulting from each of many experiments using maximum likelihood. My goal is for each experiment, given iid data $y_{1:k}$, fit a Beta distribution, and then ...
2
votes
2answers
102 views

How to understand that MLE of Variance is biased in a Gaussian distribution?

I'm reading PRML and I don't understand the picture. Could you please give some hints to understand the picture and why the MLE of variance in a Gaussian distribution is biased? formula 1.55: $$ ...
1
vote
0answers
46 views

Maximum likelihood in the GJR-GARCH(1,1) model

In the standard GARCH(1,1) model with normal innovations $\sigma^2_t=\omega+\alpha\epsilon^2_{t-1}+\beta\sigma^2_{t-1} $ the likelihood of $m$ observations occurring in the order in which they are ...
6
votes
3answers
279 views

Derivation of MLE of linear regression: and now? Why is there discrepancy to lm in R?

I want to understand the ML Estimation of the linear model from top to bottom or vice versa ;-). I totally get the part of formulating the LogLikelihood function and how to get the derivatives of beta ...
1
vote
1answer
25 views

On the usage of numerical optimization technique to maximize log-likelihood

Literature and resources say that when the ML log-likelhood does not have a closed form expression, then we can use Newton-Raphson and other optimization techniques. My Question is: During estimation ...
2
votes
0answers
61 views

Finding the MLE of the Pareto distribution and distributions

The Pareto distribution $P(a,c)$, with positive parameters $a$ and $c$, has density function $$ p(x;a,c) = \frac{ac^a}{x^{a+1}} $$ for $x \geq c$. Then, if $X_1, \dots, X_n$ is a random sample from ...
3
votes
0answers
17 views

Inferences about a distribution given running maximum values

Here is a question inspired by this question from StackOverflow. Suppose you have observations of a variable which is measured once a minute, but the values are only recorded if they are greater than ...
0
votes
1answer
26 views

Holdout set for image task

I need to validate whether one or two templates/shapes are present in an image. Fitting two templates has a better maximum likelihood then fitting one template which is a clear symptom of overfitting. ...
2
votes
0answers
43 views

Likelihood maximization: MCEM algorithm versus MCMC algorithm

Hello Everyone this is my first question. I am a particle physicist and I am doing some empirical studiues on parameters estimation using different methods (this might give me some handle to study on ...
0
votes
0answers
8 views

F-test for nested models fitted over two curves with shared parameters

I am currently doing a numerical minimization routine to simultaneously fit two curves (with shared parameters) to two datasets. I've managed to show that, assuming the likelihood of the combined ...
0
votes
0answers
25 views

Maximum likelihood method

I try to calibration the parameters $\theta$ of my probabilistic model from available (limited) information at some point $\mathbf{x}_i, i=(1,...,n)$ on a 3D space defined by $[0, M]^3 \subset ...
1
vote
0answers
17 views

Existence and Uniqueness of an Estimator

The object to be observed consists of B cubes $(b_{1},\ldots,b_{B})$. The detector consists of $D$ parts namely $(d_{1},\ldots,d_{D})$. Let $p(b_{i},d_{j})$ denote the probability of detecting a ...
0
votes
1answer
80 views

Logistic regression with +1/-1 labels

I am trying to implement logistic regression where the label space is {-1,+1} instead of the usual {0,1}. I know that I can model this as a 0-1 problem but nevertheless I wanted to see if I can derive ...
2
votes
0answers
38 views

Bayesian inference when the data are distorted in an unknown manner

Say I make observations of a spatial distribution on a 3D grid. Due to unknown combination of errors, the data on the grid is non-uniformly blurred, and so we can't consider each grid point to be ...