a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

0
votes
1answer
33 views

Is LIML consistent under heteroskedastic errors?

Please let the answer be yes. Suppose we have a model \begin{eqnarray} y= X \beta + \epsilon \\ X = Z \Pi + V \end{eqnarray} and we compute the LIML estimator \begin{eqnarray} \hat{\beta}_{LIML} = ...
0
votes
0answers
11 views

Is this R code of Rao score test for the Bernoulli data model correct? [migrated]

I am a complete statistical noob and new to R, hence the question. I've tried to find an implementation of the Rao score for the particular case when one's data is ...
0
votes
0answers
16 views

Penalized ML estimation of non-linear probit

I have a model of the form $P(y_i=1) = \Phi(\frac{w_1^{\beta'x_i}-w_2^{\beta'x_i}}{\sigma' x_i})$ where $y_i$ is a binary response, $\Phi$ is the normal CDF, $w_1$ and $w_2$ are non-negative ...
0
votes
0answers
37 views

“All of Statistics” questions [on hold]

I am not able to solve a couple of questions asked in the exercise 10.11 of book - All of Statistics. Question 1 Question 2 How do I start ? Any hints would be helpful ? I do not want complete ...
3
votes
1answer
24 views

Negative weights in maximum likelihood method?

In physics we like to use the maximum likelihood method to fit our models to our data. (I'm sure the first part of this post is review to you all, I just want to be complete so that you will know ...
0
votes
1answer
58 views

Optimizing parameter estimates by minimizing chi^2 in iterative procedure

I need to minimize my Chi^2 (bottom-left in figure 1) by adjusting parameter-values in a MLE-procedure (or something alike). The chi^2 (red) is a goodness-of-fit measure. It expresses how well the ...
0
votes
0answers
10 views

GARCH estimation, reduce outlier effect with t distribution

I am trying to estimate GARCH(1, 1) using MLE. As my data contains a lot of outliers, I think using t distribution to represent errors makes a lot of sense. But I am completely lost at how to ...
0
votes
1answer
19 views

Deriving the common LIML estimator from first principles

David Hendry (1976) comments that deriving the LIML estimator is hard. I tend to agree. Guido Imbens has a nice expression here which reads \begin{eqnarray} \hat{\beta}_{LIML} = (X'(I - \lambda M_Z) ...
4
votes
2answers
46 views

Expectation maxmisation algorithm increases true likelihood at each iteration

I've heard that the EM algorithm ensures that the true likelihood is non-decreasing at each iteration of the algorithm, but I'm not sure why this is the case. I've provided a basic plot which I ...
0
votes
1answer
27 views

MLE of Cauchy distribution in R

I am trying to compute the following "approximate Maximum Likelihood Estimate" in R. I am a little lost as to how to do this though: any hints would be appreciated, thanks
0
votes
0answers
39 views

maximum likelihood estimation with a programmed function

I am trying to do an exercise of MLE available in: http://people.missouristate.edu/songfengzheng/Teaching/MTH541/MLE-R.pdf it is the last exercise about gamma distribution, so far I have done the ...
0
votes
1answer
38 views

tobit likelihood functions with bivariate normal and unspecified error distributions

I have searched high and low for examples like this online but I cannot find it anywhere. This question is pulled from a PhD course in econometrics. Given $(y^*_1,y^*_2)$~ bivariate normal with the ...
0
votes
0answers
16 views

Approximate MLE of Cauchy Distribution in R

I am trying to compute the following "Approximate Maximum Likelihood" in R: $$ (\hat{\Psi},\hat{\Phi},\hat{\gamma})={Argmax}_{\Phi,\Psi,\gamma} \sum^{T-s}_{t=r+1} ln (g(A;\gamma)) $$ Assuming r = 2, ...
2
votes
1answer
28 views

mle estimate for standard deviation of t-distribution

For a Student-t distribution, $t_{\nu}\left(\mu,s^2\right)$, let $\hat{s}$ be mle of scale and $\hat{\nu}$ be the mle of degrees of freedom. Functional invariance of mle implies that any linear or ...
0
votes
0answers
29 views

MCMC: The posteriors are too narrow

I have this very simple MCMC there I have ...
0
votes
0answers
50 views

Maximum likelihood estimate parameters estimation

In this tutorial on mixture models, page 2, how did the author arrive to the parameters for maximum likelihood in the fully observed case? This is the general setting (based on an excerpt from the ...
3
votes
1answer
21 views

What is the solution to this minimization problem?

I'm encountering the following minimization problem in my research: $$\hat b = \underset{b}{\arg\min} \sum_i^n \left( \log \frac{a_i}{b} \right)^2$$ I could iteratively optimize, but I think that ...
0
votes
0answers
36 views

MLE of a linear function

So I have a function $$f(x\mid\theta) = \frac{1+x\theta}{2}$$ For the interval $-1\leq x\leq 1$. I need to find the mle of $\theta$ but the only way I've learned how is to take the derivative which ...
5
votes
2answers
127 views

A Question on Elementary Statistical Inference

A box contains $5$ white and $2$ black balls. A coin with unknown $P(Head)=p$ is tossed once. If it lands HEADS then a white ball is added, else a black ball is added to the box. Then a ball is ...
0
votes
0answers
10 views

How to obtain the covariance matrix for the parameters estimated by maximizing this log-likelihood of logit/probit

I have this linear latent model : $y^*=\beta_0+\beta_1 X+\epsilon,$ $y^*$ is latent (non observed) variable and $\epsilon\sim iid (0,\sigma^2 I)$ Consider : $y=1$ if $\beta_0+\beta_1 X>0$ and ...
0
votes
1answer
29 views

Relationship between Poisson generation and generalized Kullback-Leibler divergence

I have read that, in the context of matrix factorization, performing maximum likelihood estimation under the assumption that the entries are Poisson generated is equivalent to minimizing the ...
0
votes
0answers
12 views

To obtain Pearson type III parameters and shift value

How to obtain Pearson type III parameters and shift value? I am using R and if you can give me an instruction, it would be helpful. I have used pearsonFitML function from PearsonDS package, but I can ...
0
votes
0answers
16 views

I do not understand what it means to maximise a parameter of a population using the method of maximum likelihood estimator? [duplicate]

I understand the procedure of the method of maximum likelihood. I also understand the method is used to estimate a statistic within a population. However I do not understand what it means to maximise ...
0
votes
1answer
13 views

How to combine MLEs for a point process

I am estimating parameters of a point process by doing an MLE calculation using event time data. However I in fact have several distinct sets of data from different locations. I can estimate the ...
1
vote
0answers
15 views

finding an MLE with samples from binomial distribution

A box contains 5 white balls and 2 black balls. A coin with unknown $P(\text{Head}) = p$ is tossed. A white ball is added to the box if the outcome is head; otherwise a black ball is added. Then a ...
2
votes
0answers
17 views

Incorporating population priors into MLE fits with few/limited samples

I am fitting Beta distributions to data resulting from each of many experiments using maximum likelihood. My goal is for each experiment, given iid data $y_{1:k}$, fit a Beta distribution, and then ...
2
votes
2answers
97 views

How to understand that MLE of Variance is biased in a Gaussian distribution?

I'm reading PRML and I don't understand the picture. Could you please give some hints to understand the picture and why the MLE of variance in a Gaussian distribution is biased? formula 1.55: $$ ...
1
vote
0answers
40 views

Maximum likelihood in the GJR-GARCH(1,1) model

In the standard GARCH(1,1) model with normal innovations $\sigma^2_t=\omega+\alpha\epsilon^2_{t-1}+\beta\sigma^2_{t-1} $ the likelihood of $m$ observations occurring in the order in which they are ...
6
votes
3answers
272 views

Derivation of MLE of linear regression: and now? Why is there discrepancy to lm in R?

I want to understand the ML Estimation of the linear model from top to bottom or vice versa ;-). I totally get the part of formulating the LogLikelihood function and how to get the derivatives of beta ...
1
vote
1answer
21 views

On the usage of numerical optimization technique to maximize log-likelihood

Literature and resources say that when the ML log-likelhood does not have a closed form expression, then we can use Newton-Raphson and other optimization techniques. My Question is: During estimation ...
2
votes
0answers
58 views

Finding the MLE of the Pareto distribution and distributions

The Pareto distribution $P(a,c)$, with positive parameters $a$ and $c$, has density function $$ p(x;a,c) = \frac{ac^a}{x^{a+1}} $$ for $x \geq c$. Then, if $X_1, \dots, X_n$ is a random sample from ...
3
votes
0answers
17 views

Inferences about a distribution given running maximum values

Here is a question inspired by this question from StackOverflow. Suppose you have observations of a variable which is measured once a minute, but the values are only recorded if they are greater than ...
0
votes
1answer
24 views

Holdout set for image task

I need to validate whether one or two templates/shapes are present in an image. Fitting two templates has a better maximum likelihood then fitting one template which is a clear symptom of overfitting. ...
2
votes
0answers
42 views

Likelihood maximization: MCEM algorithm versus MCMC algorithm

Hello Everyone this is my first question. I am a particle physicist and I am doing some empirical studiues on parameters estimation using different methods (this might give me some handle to study on ...
0
votes
0answers
8 views

F-test for nested models fitted over two curves with shared parameters

I am currently doing a numerical minimization routine to simultaneously fit two curves (with shared parameters) to two datasets. I've managed to show that, assuming the likelihood of the combined ...
0
votes
0answers
25 views

Maximum likelihood method

I try to calibration the parameters $\theta$ of my probabilistic model from available (limited) information at some point $\mathbf{x}_i, i=(1,...,n)$ on a 3D space defined by $[0, M]^3 \subset ...
1
vote
0answers
16 views

Existence and Uniqueness of an Estimator

The object to be observed consists of B cubes $(b_{1},\ldots,b_{B})$. The detector consists of $D$ parts namely $(d_{1},\ldots,d_{D})$. Let $p(b_{i},d_{j})$ denote the probability of detecting a ...
0
votes
1answer
78 views

Logistic regression with +1/-1 labels

I am trying to implement logistic regression where the label space is {-1,+1} instead of the usual {0,1}. I know that I can model this as a 0-1 problem but nevertheless I wanted to see if I can derive ...
2
votes
0answers
38 views

Bayesian inference when the data are distorted in an unknown manner

Say I make observations of a spatial distribution on a 3D grid. Due to unknown combination of errors, the data on the grid is non-uniformly blurred, and so we can't consider each grid point to be ...
2
votes
2answers
161 views

Determine Maximum Likelihood Estimate (MLE) of loglogistic distribution

I am given two data sets containing dates and losses (in some currency). I have to determine the maximum likelihood estimates of the parameters of loglogistic distribution. I googled and found a ...
5
votes
0answers
42 views

Does using bootstrapped samples improve parameter estimates for a fitted distribution?

The R package retimes has a function for fitting an ex-Gaussian distribution to a set of observations. The method involves taking multiple bootstrapped samples of the observations, and fitting the ...
1
vote
1answer
33 views

Find the sampling distribution of the MLE of the uniform distribution [duplicate]

The MLE is $ \theta = max [x1,...,xn] $ And $ P(max [Xi] < t) = P(Xi < t)^n = P(t/\theta) $ But the question asks me to show that $ P(max[Xi]< t) = (min[\theta, t]/ \theta)^n * I[t>0] $ ...
-1
votes
1answer
25 views

Estimator $\gamma = \sum a_i\times x_i$ , where $X_i \sim \exp(t_i \theta )$ Show $\gamma$ is unbiased if $\sum a_i/t_i = 1$

I'm getting really confused with the estimators in this question! $X_i \sim \exp(t_i \theta x)$ where $t_i$ are positive constants. The MLE for $\theta = \frac n{\sum t_i x_i}$ And $\phi = ...
5
votes
3answers
96 views

Computing the Variance of an MLE

Suppose we have i.i.d. $n$ observations $(X_1,X_2,...X_n)$ from a population with density $$f_\theta(x)=\begin{cases}\theta x^{\theta-1}&\text{ if }0\leq x\leq ...
2
votes
1answer
59 views

numerical difference between sum of squared residuals and likelihood

I previously asked a question that got labelled as duplicated because I did not explain it correctly. I should not have used the regression model as an example because I can see how, by using that as ...
1
vote
0answers
35 views

Theorem A3 in Asymptotic properties of MLE for the i.n.i.d. case

Can someone explain to me why in the theorem below (case in $R^1$) $$\lim \inf I(A\cap B(u))|X_k(u)| \leq I(A\cap B)|X_k|$$ Full text of proof: ...
1
vote
0answers
25 views

A simple approach to maximum likelihood estimation for a model with no closed-form solution

I would like to estimate the best fitting parameters of a parametric model, $f(\theta)$, that does not have a closed-form solution. There are $n$ i.i.d. environmental observations and the aim is to ...
2
votes
1answer
65 views

ML estimate of exponential distribution (with censored data)

In Survival Analysis, you assume the survival time of a r.v. $X_i$ to be exponentially distributed. Considering now that I have $x_1,\dots,x_n$ "outcomes" of i.i.d r.v.'s $X_i$. Only some proportion ...
2
votes
2answers
69 views

Which one is better maximum likelihood or marginal likelihood and why?

While performing regression if we go by the definition from: What is the difference between a partial likelihood, profile likelihood and marginal likelihood? that, Maximum Likelihood Find β and θ ...
0
votes
0answers
55 views

Why is the most probable assignment for all variables in MRFs called MAP assignment?

I am new to graphical model, especially Markov Random Fields. I have a question about MAP assignment. Let say we have the graph structure and all the potential functions. MAP estimation is finding ...