# Tagged Questions

a method of estimating parameters of a statistical model by choosing the parameter value that optimizes the probability of observing the given sample.

978 views

### Intuition behind why Stein's paradox only applies in dimensions $\ge 3$

Stein's Example shows that the maximum likelihood estimate of $n$ normally distributed variables with means $\mu_1,\ldots,\mu_n$ and variances $1$ is inadmissible (under a square loss function) iff ...
2k views

There are several threads on this site for book recommendations on introductory statistics and machine learning but I am looking for a text on advanced statistics including, in order of priority: ...
7k views

### Using lmer for prediction

Hello I have two problems that sound like natural candidates for multilevel/mixed models, which I have never used. The simpler, and one that I hope to try as an introduction, is as follows: The data ...
607 views

### What are some illustrative applications of empirical likelihood?

I have heard of Owen's empirical likelihood, but until recently paid it no heed until I came across it in a paper of interest (Mengersen et al. 2012). In my efforts to understand it, I have gleaned ...
3k views

### What is the difference between a partial likelihood, profile likelihood and marginal likelihood?

I see these terms being used and I keep getting them mixed up. Is there a simple explanation of the differences between them?
1k views

### How to ensure properties of covariance matrix when fitting multivariate normal model using maximum likelihood?

Suppose I have the following model $$y_i=f(x_i,\theta)+\varepsilon_i$$ where $y_i\in \mathbb{R}^K$ , $x_i$ is a vector of explanatory variables, $\theta$ is the parameters of non-linear function ...
468 views

### Hessian of profile likelihood used for standard error estimation

This question is motivated by this one. I looked up two sources and this is what I found. A. van der Vaart, Assymptotic Statistics: It is rarely possible to compute a profile likelihood ...
405 views

### Which distributions have closed-form solutions for maximum likelihood estimation?

Which distributions have closed-form solutions for the maximum likelihood estimates of the parameters from a sample of independent observations?
519 views

### How to do estimation, when only summary statistics are available?

This is in part motivated by the following question and the discussion following it. Suppose the iid sample is observed, $X_i\sim F(x,\theta)$. The goal is to estimate $\theta$. But original sample ...
540 views

### Can the empirical Hessian of an M-estimator be indefinite?

Jeffrey Wooldridge in his Econometric Analysis of Cross Section and Panel Data (page 357) says that the empirical Hessian "is not guaranteed to be positive definite, or even positive semidefinite, for ...
592 views

### What are the disadvantages of the profile likelihood?

Consider a vector of parameters $(\theta_1, \theta_2)$, with $\theta_1$ the parameter of interest, and $\theta_2$ a nuisance parameter. If $L(\theta_1, \theta_2 ; x)$ is the likelihood constructed ...
3k views

### When should I *not* use R's nlm function for MLE?

I've run across a couple guides suggesting that I use R's nlm for maximum likelihood estimation. But none of them (including R's documentation) gives much theoretical guidance for when to use or not ...
850 views

### Maximum likelihood estimators for a truncated distribution

Consider $N$ independent samples $S$ obtained from a random variable $X$ that is assumed to follow a truncated distribution (e.g. a truncated normal distribution) of known (finite) minimum and maximum ...
451 views

### Properties of logistic regressions

We're working with some logistic regressions and we have realized that the average estimated probability always equals the proportion of ones in the sample; that is, the average of fitted values ...
389 views

### The trinity of tests in maximum likelihood: what to do when faced with contradicting conclusions?

The Wald, Likelihood Ratio and Lagrange Multiplier tests in the context of maximum likelihood estimation are asymptotically equivalent. However, for small samples, they tend to diverge quite a bit, ...
270 views

### Hypothesis testing on the inverse covariance matrix

Suppose I observe i.i.d. $x_i \sim \mathcal{N}\left(\mu,\Sigma\right)$, and wish to test $H_0: A\$vech$\left(\Sigma^{-1}\right) = a$ for a conformable matrix $A$ and vector $a$. Is there known work ...
356 views

### Observed information matrix is a consistent estimator of the expected information matrix?

I am trying to prove that the observed information matrix evaluated at the weakly consistent maximum likelihood estimator (MLE), is a weakly consistent estimator of the expected information matrix. ...
696 views

### Is there always a maximizer for any MLE problem?

I wonder if there is always a maximizer for any maximum (log-)likelihood estimation problem? In other words, is there some distribution and some of its parameters, for which the MLE problem does not ...
1k views

### How large should a sample be for a given estimation technique and parameters?

Is there a rule-of thumb or even any way at all to tell how large a sample should be in order to estimate a model with a given number of parameters? So, for example, if I want to estimate a ...
563 views

### If you use a point estimate that maximizes $P(x | \theta)$, what does that say about your philosophy? (frequentist or Bayesian or something else?)

If somebody said "That method uses the MLE the point estimate for the parameter which maximizes $\mathrm{P}(x|\theta)$, therefore it is frequentist; and further it is not Bayesian." would you ...
1k views

### What is the 'fundamental' idea of machine learning for estimating parameters?

The 'fundamental' idea of statistics for estimating parameters is maximum likelihood. I am wondering what is the corresponding idea in machine learning. Qn 1. Would it be fair to say that the ...
271 views

### How much calculus is necessary to understand maximum likelihood estimation?

I am trying to plan out a study plan for learning MLE. In order to do this I am trying to figure out what is the minimum level of calculus that is necessary to understand MLE. Is it sufficient to ...
1k views

### How do I know which method of parameter estimation to choose?

There are quite a few methods for parameter estimation out there. MLE, UMVUE, MoM, decision-theoretic, and others all seem like they have a fairly logical case for why they are useful for parameter ...
517 views

### How can I estimate 95% confidence intervals using profiling for parameters estimated by maximising a log-likelihood function using optim in R?

How can I estimate 95% confidence intervals using profiling for parameters estimated by maximising a log-likelihood function using optim in R? I know I can asymptotically estimate the covariance ...
1k views

### Weibull distribution parameters $k$ and $c$ for wind speed data

I have 5 years worth of mean wind speed meteorological data. I want to compare the observed wind speed data against a Weibull distribution on a graph of the probability density distribution. I have ...
193 views

### MLE/Likelihood of lognormally distributed interval

I have a variable set of responses that are expressed as an interval such as the sample below. ...
277 views

### Distribution of reciprocal of regression coefficient

Suppose that we have a linear model $y_i = \beta_0 + \beta_1 x_i + \epsilon_i$ that meets all the standard regression (Gauss-Markov) assumptions. We are interested in $\theta = 1/\beta_1$. Question ...
162 views

### For what distribution is a trimmed mean the maximum likelihood estimator?

The sample mean is the maximum likelihood estimator of $\mu$ for a normal distribution $\text{Normal}(\mu,\sigma)$. The sample median is the maximum likelihood estimator of $m$ for a Laplace ...
128 views

### Variance-covariance matrix of the parameter estimates wrongly calculated?

I fitted an hyperbolic distribution to my data with the hyperbFit(mydata,hessian=TRUE) command (package HyperbolicDist). The hessian looks like: ...
187 views

### Multinomial choice with binary observations

Is there a standard name for a multinomial choice model where the observations are in the form of binary questions such as "do you prefer A to B" and "do you prefer B to D"? This seems like a common ...
147 views

### How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative if they are different costs?

How to incorporate costs (into logit model) of false positive, false negative, true positive, true negative responses, if they are different costs ? Is it possible to do that on the level of ...
3k views

### In R, given an output from optim with a hessian matrix, how to calculate parameter confidence intervals using the hessian matrix?

Given an output from optim with a hessian matrix, how to calculate parameter confidence intervals using the hessian matrix? ...
455 views

### Finding the MLE for a univariate exponential Hawkes process

The univariate exponential Hawkes process is a self-exciting point process with an event arrival rate of: $\lambda(t) = \mu + \sum\limits_{t_i<t}{\alpha e^{-\beta(t-t_i)}}$ where $t_1,..t_n$ ...
335 views

### A good book with equal stress on theory and math

I have had enough courses on statistics during my school years and at the university. I have a fair understanding of the concepts, such as, CI, p-values, interpreting statistical significance, ...
390 views

### Estimating parameters of Student's t-distribution

What are the maximum-likelihood estimators for the parameters of Student's t-distribution? Do they exist in closed form? A quick Google search didn't give me any results. Today I am interested in the ...
855 views

### Variance-gamma distribution: parameter estimation

I have quite a general question about Variance-gamma distribution. I am interested in how to estimate it's parameters given a set of training points? I tried to find the answer in the internet, but ...
252 views

### Minimum-Distance estimation of mixed/mixture distributions

Please note: I posted this first on Mathoverflow. Someone there advised me that on stats.stackexchange the question might fit better here. This is the link to the original post. I currently have to ...
1k views

### Full information maximum likelihood for missing data in R

Context: Hierarchical regression with some missing data. Question: How do I use full information maximum likelihood (FIML) estimation to address missing data in R? Is there a package you would ...
313 views

### Testing for a significant difference between ML estimates: Likelihood ratio or Wald test?

I am trying to test whether or not there is a significant difference between maximum likelihood estimates of two genetic parameters (selection and dominance) across two environments with genotype data ...
3k views

### Estimating Lambda for Box Cox transformation for ANOVA

Assumptions: In an ANOVA where the normality assumptions are violated, the Box-Cox transformation can be applied to the response variable. The lambda can be ...
188 views

### Maximum Likelihood estimator - confidence interval

How can I construct an asymptotic confidence interval for a real parameter, starting from the MLE for that parameter?
217 views

### Integrating kernel density estimator in 2D

I'm coming from this question in case anybody wants to follow the trail. Basically I have a data set $\Omega$ composed of $N$ objects where each object has a given number of measured values attached ...
555 views

### How can I work around “lumpiness” in simulated maximum likelihood estimation?

I am attempting to estimate a model of the following form: W = alphaH * H + alphaM * M + alphaL * L + X * beta where H, M, L ...
2k views

### Fitting a generalized least squares model with correlated data; use ML or REML?

Reading the Linear Mixed Model (LMM) literature I am aware that fitting a model using REML provides better estimates of variance parameters than fitting via ML. However, we should not compare nested ...
179 views

### Maximum likelihood estimate: Is this possible to solve?

I have the following problem: Formulate the likelihood function, the log-likelihood function, and the maximum-likelihood estimate as well as the Fisher information and the observed Fisher information ...
96 views

### The vcov function cannot be applied?

I originally asked a question about the delta-method in the context of the hyperbolic distribution. I got an answer there, which is useful, except that it says I should apply the ...
74 views

### Why does the log likelihood need to go to minus infinity when the parameter approaches the boundary of the parameter space?

In a recent lecture I was told that, in order for the maximum likelihood estimate to be valid, the log likelihood needs to go to minus infinity as the parameter goes to the boundary of the parameter ...
222 views

### What is the UMVUE for $\sigma^2$ in $\mathcal N(0, \sigma^2 )$?

By using the exponential class factorization theorem, I came up with $Y = \sum (x_i)^2$ to be the complete and sufficient statistics for $\sigma^2$ . Using this sufficient statistic as a condition, ...
I'm looking for the right statistical terminology to describe the following problem. I want to characterize an electronics device that has a linear response $Y = \beta_0 + \beta_1 X + \epsilon$ ...