a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

learn more… | top users | synonyms (1)

2
votes
0answers
17 views

Does maximum likelihood minimize a kind of generalized “0-1 loss”?

A very good point was raised here about how the optimal betting strategy under 0-1 loss was to bet on the mode, while under MSE loss the optimal strategy was to bet on the mean. Maximum likelihood ...
1
vote
1answer
50 views

Likelihood ratio tests using ML vs. REML

I am using Mixed effects models (nlme package in R) to choose the model with the best random and fixed effects. I am following the procedure of Zurr et al. (2009) ...
0
votes
0answers
15 views

maximum likelihood for count data [on hold]

Can maximum likelihood estimation method of parameter estimation be used in categorical data? I was looking for some examples of it. What other parameter estimation techniques work for this kind of ...
5
votes
1answer
199 views

Degeneracy paradox

Say I have a highly biased coin that lands heads with $p_h=0.01$ and tails with $p_t=0.99$, and I flip it $98$ times. The probability of zero heads is ${p_t}^{98} \approx 0.373$. The probability of ...
0
votes
0answers
40 views

Maximum likelihood method vs. least squares method

What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ? Why can't we use MLE for predicting $y$ values in linear regression and vice versa? Any ...
2
votes
1answer
38 views

Motivation for gradient descent method over canonical method (for OLS/MLE) for simple linear regression?

I am beginner in machine learning and I am currently trying to find the motivation for gradient descent method. I am confused why we want to employ gradient descent method for linear regression? I see ...
1
vote
0answers
18 views

Bayesian estimates for Deming regression coinciding with least-squares estimates

Consider the following Deming model with independent replicates : $$x_{i,j} \mid \theta_{i} \sim {\cal N}(\theta_{i}, \gamma_X^2), \quad y_{i,j} \mid \theta_{i} \sim {\cal N}(\alpha+\beta\theta_{i}, ...
0
votes
0answers
19 views

Box-Cox transformation for the ordered outcome model

I wonder if there is someone out there who had the following problem. Namely, I am trying to fit an ordered logit model (-ologit-) in Stata but before that I would ...
0
votes
0answers
14 views

Information matrix at Wald statistic

AFAIK, the Wald statistic of ML estimator uses the limit normality of MLE, and it looks as: assume to test $H_0 : \theta = \theta_0$ $T = (\hat{\theta} - \theta_0)' (I(\theta_0))^{-1} (\hat{\theta} ...
1
vote
0answers
44 views

Fitting GLM with Quasi-Newton method

I'm trying to code my own quasi-Newton algorithm for fitting GLMs in R. My results do not match up with glm and I've been over my code many, many times so I'm ...
2
votes
0answers
33 views
+50

Finding quality of fit for two discrete variables with low statistics

I have data from an experiment which I am trying to explain using a model. I do not have an analytic formula for the prediction of the model but instead I got its prediction through a simulation. The ...
0
votes
1answer
15 views

log multivariate normal differentiation (MLE)

I've come across a lot of explanations of how to differentiate the multivariate normal, but they all appear to skip the step that I'm stuck on. Here's what I've got so far. By logging and removing ...
1
vote
0answers
29 views

Convergence from the EM Algorithm with bivariate mixture distribution

I have a mixture model which I want to find the maximum likelihood estimator of given a set of data $x$ and a set of partially observed data $z$. I have implemented both the E-step (calculating the ...
2
votes
0answers
43 views

Maximizing combined log likelihood from different dataset using r

I have observed data $y$. And I have a function that gives me estimated $\hat{y} = f(x,\hat{P})$ where P is the parameter I want to estimate. I was able to optim() command in R to get maximized ...
2
votes
1answer
22 views

On FIML assumptions

In Hayashi's Econometrics, page 529, he states one of the assumptions we need for the FIML estimator. My doubt is in the third line of point 1). He says that the vector ...
0
votes
1answer
40 views

Weibull distribution

I need to find a distribution that fail regularity condition.Maybe weibull distribution can be but I did not find why, please help me.
0
votes
0answers
13 views

Practical application of Cramer-Rao lower bound to calculate the variance of estimator

I would like to use the Cramer-Rao lower bound to help me estimate the variance of my maximum likelihood estimator, across a range of different samples of data. My question is, how do I do this ...
3
votes
2answers
100 views

What to do when your likelihood function has a double product with small values near zero - log transform doesn't work?

I currently have a likelihood function defined as the following: $$ L=\prod_{i=1}^{N}\left[\prod_{s=1}^{S_i}L_{is}(y\space|\space \rho_A)\times\phi + \prod_{s=1}^{S_i}L_{is}(y\space|\space ...
0
votes
0answers
57 views

How can the R-matrix in a mixed model be estimated?

In Henderson's Mixed Model equation: $y = X\beta + Zv + \epsilon$ where the joint variance of v and the error term is: $Var\begin{bmatrix} v \\ \epsilon \end{bmatrix} = \begin{bmatrix} G & ...
0
votes
0answers
11 views

How to estimate Mean and Variance of a Gaussian dataset of 20 numbers using ML,MAP and Bayesian Inference?

I have generated 20 random numbers from a Gaussian distribution with mean 5, and standard deviation 1. I have a question that has asked me to estimate the mean and variance using the above methods. ...
1
vote
0answers
19 views

On the computational efficiency of some matlab code when finding the loglikelyhood

I am looking at some code on GAS models found here (number 7). The code finds the log-likelihood of the joint conditional distribution of each observation by summing up the single log-likelihood of ...
3
votes
1answer
56 views

Connection between MLE (Maximum Likelihood Estimation) and introductory Inferential Statistics?

The first thing that one learns in statistics is to use the sample mean, $\hat{X}$, as an unbiased estimate of the population mean, $\mu$; and pretty much the same would be true for the variance, ...
0
votes
1answer
36 views

How to estimate parameters of a log-normal distribution?

I am using income data from the Current Population Survey for a small undergrad economics paper. In economics, there is evidence that the income of 97%–99% of the population is distributed ...
2
votes
1answer
118 views

Conceptual question on log-likelihood value

I am trying to implement the log-likelihood expression Eq(7) from the paper, Parameter Estimation for Linear Dynamical Systems (1996). Re-writing, For the model, $h(t) = \mathbf{A^T} h(t-1) + ...
5
votes
1answer
75 views

Do I get the nice asymptotic properties of MLE when I restrict the parameter space?

I would like to know if the MLE is still consistent, asymptotically normal, and efficient when I put restrictions on the parameter space. I think my confusion stems from the definition of the ...
2
votes
0answers
35 views

MLE estimate of normal distribution

I am quoting this from Greene's econometrics book: The occasional statement that the properties of the MLE are only optimal in large samples is not true, however. It can be shown that when ...
1
vote
1answer
39 views

How to combine heckman selection and binary endogenous variable in a two-step way?

I want to fit a probit model with a binary endogenous variable and heckman sample selection problem, it's something like ...
0
votes
0answers
20 views

MLE of sum of two binomials

Let's $X_1 \sim Binomial(n_1, p_1), X_2 \sim Binomial(n_2, p_2)$. I want to find MLE of $\psi = p_1 - p_2$. There is a property of MLE: if $\hat{\theta}$ is MLE of $\theta$, then $g(\hat{\theta})$ is ...
4
votes
0answers
20 views

Can I penalize an arbitrary regression model and get Elastic-Net-esque results?

Consider an arbitrary-ish regression model with the unpenalized likelihood $$ \log \mathcal{L} = \sum_i f\left(y_i\,|\,g(\beta_0 + \beta x_i)\right) $$ with $\beta = \left(\beta_1, \dots, ...
2
votes
1answer
38 views

For Maximum-likelihood estimation (MLE), must one assume a probability distribution of the dependent value, the error term, or both?

So to estimate the parameters of a model using MLE one must write the likelihood function of having observed the data sample at hand by assuming that it came from a particular distribution. In order ...
0
votes
0answers
12 views

Fitting gamma distribution to data set with one zero observation

I am using maximum likelihood estimation to fit a gamma distribution to shelf life data. Specifically, the data I have is the time (in days) between the day a product was sold and the day the first ...
4
votes
2answers
378 views

I am wondering why we use negative (log) likelihood sometimes?

This question has puzzled me for a long time. I understand the use of 'log' in maximizing the likelihood so I am not asking about 'log'. My question is, since maximizing log likelihood is equivalent ...
0
votes
0answers
8 views

Model Selection In small clusters

I have a question please. Is ok to make model selection with MLE in small cluster in order to allow for comparison, and after getting the final model then fit the final model with REML? Since REML is ...
2
votes
0answers
20 views

KL-divergence as a negative log likelihood for exponential families

I am reading Distributed Estimation, Information Loss and Exponential Families, where the authors consider and compare two estimators for $\theta$ in the parametric model $p(x\mid\theta)$: the ...
3
votes
1answer
53 views

Self Study: ML Parameter Estimates — do I need numerical maximization?

I have a particular PDF with two parameters, specified as: $$\alpha \beta e^{-\beta x}(1 - e^{-\beta x})^{\alpha - 1}, \alpha > 0, \beta > 0, x_i > 0$$ Given a random iid sample $(x_1, ...
1
vote
0answers
13 views

Fit a conditional Bernoulli-gamma distribution using maximum likelihood

I am trying to model payment amounts for a collections agency. I am struggling with formulating the function I need to optimize, however. I have researched various distributions and stumbled-across ...
0
votes
0answers
16 views

Maximum likelihood estimation of the log-normal distribution using R [migrated]

I'm trying to estimate a linear model with a log-normal distributed error term. I already have working code for a linear model with normally distributed errors: ...
0
votes
1answer
33 views

Problem with generalized likelihood ratio test from samples from beta distribution

I was trying to resolve this exercise: This exercise is from the book "Statistical Inference, Second Edition" by Casella and Berger. Checking the solutions manual, I was understanding the solution ...
1
vote
1answer
38 views

What are the (philosophical) assumptions behind GMM and Maximum Likelihood Estimation?

As stated in the question. In particular, how does a researcher know when to apply which estimation method and are there any examples that can show when one case is more appropriate than the other? ...
2
votes
0answers
33 views

Finding MLE with ordered statistics?

Let Y1 < Y2 < ... < Yn be the order statistics of a random sample of size n from the uniform distribution of the continuous type over the closed interval: $$[\theta - \rho, \theta + \rho]$$ ...
1
vote
0answers
18 views

Is the application of the Frisch-Waugh-Lovell Theorem really necessary?

Suppose I have a model \begin{eqnarray} y = X_1 \beta + X_2 \gamma + \epsilon \\ X = Z \Pi + V \end{eqnarray} where $X_1$ is endogenous, Z are instruments, $X_2$ are exogenous. If I however include ...
1
vote
1answer
13 views

Finding covariance matrix for MLE of correlated outputs

I generate data using the following model: $\begin{pmatrix}Y_1\\Y_2\end{pmatrix} \sim \mathcal{N}\left( \begin{pmatrix}\mathbf{X}\beta_1\\\mathbf{X}\beta_2\end{pmatrix}, \mathbf{\Sigma} \right)$ I ...
1
vote
1answer
19 views

How to find $\arg\max$ of a neural network?

Let's say I have a neural network $f$ that takes input $\vec x \in \mathbb {R}^n$ and produces output $f(\vec x) \in \mathbb{R}$. How can I find $\hat x = \underset{\vec x}{\arg\max} \; f(\vec x)$?
1
vote
0answers
28 views

Deming Regression/Errors-in-Variables with replicates

I have a question about a model related to Deming regression and would appreciate some help and/or publications to further study this model. Statistical Model: \begin{align} ...
0
votes
2answers
40 views

Computationally efficient Gaussian MAP estimation algorithm in MATLAB

I have a MAP estimation model for a Gaussian prior and i.i.d Gaussian noise: $$y=x+n$$ where $x\sim\mathcal{N}(0,\Sigma)$ and $n\sim \mathcal{N}(0,\sigma^2I)$. The MAP estimate is given by $$ ...
4
votes
1answer
38 views

Distributed Datasets and MLE

Suppose I have a very large dataset of size N, evenly distributed over M computers so that each computer has N/M data points. Suppose I want to fit a model using MLE that requires an iterative method. ...
1
vote
1answer
43 views

f(y | x) or f(y,x) in regression and MLE

In $Y = aX + b + \epsilon$ where $\epsilon$ ~ $N(0,\sigma^2)$ and i.i.d regression setting If X is stochastic and $E(\epsilon\mid X) =0$, then which one is correct: (1) $f(x,y) = ...
0
votes
1answer
39 views

Is LIML consistent under heteroskedastic errors?

Please let the answer be yes. Suppose we have a model \begin{eqnarray} y= X \beta + \epsilon \\ X = Z \Pi + V \end{eqnarray} and we compute the LIML estimator \begin{eqnarray} \hat{\beta}_{LIML} = ...
0
votes
0answers
21 views

Penalized ML estimation of non-linear probit

I have a model of the form $P(y_i=1) = \Phi(\frac{w_1^{\beta'x_i}-w_2^{\beta'x_i}}{\sigma' x_i})$ where $y_i$ is a binary response, $\Phi$ is the normal CDF, $w_1$ and $w_2$ are non-negative ...
4
votes
1answer
32 views

Negative weights in maximum likelihood method?

In physics we like to use the maximum likelihood method to fit our models to our data. (I'm sure the first part of this post is review to you all, I just want to be complete so that you will know ...