Questions tagged [estimation]

This tag is too general; please provide a more specific tag. For questions about the properties of specific estimators, use [estimators] tag instead.

Filter by
Sorted by
Tagged with
63
votes
3answers
62k views

Why is sample standard deviation a biased estimator of $\sigma$?

According to the Wikipedia article on unbiased estimation of standard deviation the sample SD $$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$ is a biased estimator of the SD of the ...
82
votes
7answers
137k views

Calculating the parameters of a Beta distribution using the mean and variance

How can I calculate the $\alpha$ and $\beta$ parameters for a Beta distribution if I know the mean and variance that I want the distribution to have? Examples of an R command to do this would be most ...
15
votes
5answers
3k views

Can the empirical Hessian of an M-estimator be indefinite?

Jeffrey Wooldridge in his Econometric Analysis of Cross Section and Panel Data (page 357) says that the empirical Hessian "is not guaranteed to be positive definite, or even positive semidefinite, for ...
6
votes
1answer
245 views

Trying to Estimate Disease Prevalence from Fragmentary Test Results

In response to the spread of COVID-19 disease, all Californians were ordered on 19 March 2020 to stay at home, except for such necessary errands as trips to grocery stores, pharmacies, etc. On 21 ...
27
votes
3answers
20k views

What is a Highest Density Region (HDR)?

In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book. One similar term is the Highest Posterior ...
58
votes
6answers
56k views

What is the difference between estimation and prediction?

For example, I have historical loss data and I am calculating extreme quantiles (Value-at-Risk or Probable Maximum Loss). The results obtained is for estimating the loss or predicting them? Where can ...
65
votes
15answers
9k views

Why would parametric statistics ever be preferred over nonparametric?

Can someone explain to me why would anyone choose a parametric over a nonparametric statistical method for hypothesis testing or regression analysis? In my mind, it's like going for rafting and ...
11
votes
1answer
1k views

Estimating parameters for a binomial

First of all I'd like to precise that I'm not an expert of the subject. Suppose to have two random variables $X$ and $Y$ that are binomial, respectively $X\sim B(n_1,p)$ and $Y\sim B(n_2,p),$ note ...
30
votes
4answers
13k views

Internal vs external cross-validation and model selection

My understanding is that with cross validation and model selection we try to address two things: P1. Estimate the expected loss on the population when training with our sample P2. Measure and report ...
54
votes
3answers
34k views

Standard deviation of standard deviation

What is an estimator of standard deviation of standard deviation if normality of data can be assumed?
21
votes
4answers
37k views

What is the relation between estimator and estimate?

What is the relation between estimator and estimate?
21
votes
4answers
17k views

Calculating required sample size, precision of variance estimate?

Background I have a variable with an unknown distribution. I have 500 samples, but I would like demonstrate the precision with which I can calculate variance, e.g. to argue that a sample size of 500 ...
4
votes
1answer
3k views

ARMA/GARCH estimation in sequence

I have a time series that shows a nonstationary seasonal autoregressive component as well as known heteroshedasticity. In order to model the series, I have fit a seasonal ARIMA model for the mean with ...
7
votes
7answers
3k views

Is there a GLM bible?

Is there consensus in the field of statistics that one book is the absolute best source and completely covering every aspect of GLM - detailing everything from estimation to inference?
9
votes
2answers
2k views

Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?

I have been taught that we can produce a parameter estimate in the form of a confidence interval after sampling from a population. For example, 95% confidence intervals, with no violated assumptions, ...
9
votes
2answers
10k views

How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
15
votes
1answer
3k views

What is an unbiased estimate of population R-square?

I am interested in getting an unbiased estimate of $R^2$ in a multiple linear regression. On reflection, I can think of two different values that an unbiased estimate of $R^2$ might be trying to ...
36
votes
6answers
5k views

If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?

I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless: Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
30
votes
1answer
14k views

Maximum likelihood estimators for a truncated distribution

Consider $N$ independent samples $S$ obtained from a random variable $X$ that is assumed to follow a truncated distribution (e.g. a truncated normal distribution) of known (finite) minimum and maximum ...
11
votes
1answer
5k views

Are estimates of regression coefficients uncorrelated?

Consider a simple regression (normality not assumed): $$Y_i = a + b X_i + e_i,$$ where $e_i$ is with mean $0$ and standard deviation $\sigma$. Are the Least Square Estimates of $a$ and $b$ ...
35
votes
4answers
45k views

Why squared residuals instead of absolute residuals in OLS estimation? [duplicate]

Why are we using the squared residuals instead of the absolute residuals in OLS estimation? My idea was that we use the square of the error values, so that residuals below the fitted line (which are ...
12
votes
2answers
12k views

Kullback-Leibler Divergence for two samples

I tried to implement a numerical estimate of the Kullback-Leibler Divergence for two samples. To debug the implementation draw the samples from two normal distributions $\mathcal N (0,1)$ and $\...
13
votes
4answers
786 views

Is any quantitative property of the population a “parameter”?

I'm relatively familiar with the distinction between the terms statistic and parameter. I see a statistic as the value obtained from applying a function to the sample data. However, most examples of ...
15
votes
3answers
6k views

Can I use Kolmogorov-Smirnov test and estimate distribution parameters?

I've read that Kolmogorov-Smirnov test should not be used to test the goodness of fit of a distribution whose parameters have been estimated from the sample. Does make sense to split my sample in two ...
15
votes
4answers
23k views

How to keep time invariant variables in a fixed effects model

I have data on a large Italian firm's employees over ten years and I would like to see how the gender gap in male-female earnings has changed over time. For this purpose I run pooled OLS: $$ y_{it} = ...
7
votes
1answer
2k views

Choosing the number of bootstrap resamples

Say we have the following sample and we are trying to estimate the variance of the sample mean of the population. X = [0, -1, 2, 10, -3] If I take an increasing ...
34
votes
3answers
5k views

How to find confidence intervals for ratings?

Evan Miller's "How Not to Sort by Average Rating" proposes using the lower bound of a confidence interval to get a sensible aggregate "score" for rated items. However, it's working with a Bernoulli ...
28
votes
4answers
18k views

Estimating parameters of Student's t-distribution

What are the maximum-likelihood estimators for the parameters of Student's t-distribution? Do they exist in closed form? A quick Google search didn't give me any results. Today I am interested in the ...
24
votes
3answers
2k views

Unbiased estimation of covariance matrix for multiply censored data

Chemical analyses of environmental samples are often censored below at reporting limits or various detection/quantitation limits. The latter can vary, usually in proportion to the values of other ...
15
votes
3answers
2k views

Computationally efficient estimation of multivariate mode

Short version: What's the most computationally efficient method of estimating the mode of a multidimensional data set, sampled from a continuous distribution? Long version: I've got a data set that I ...
33
votes
4answers
2k views

Are inconsistent estimators ever preferable?

Consistency is obviously a natural and important property of estimators, but are there situations where it may be better to use an inconsistent estimator rather than a consistent one? More ...
11
votes
2answers
5k views

Reference for $\mathrm{Var}[s^2]=\sigma^4 \left(\frac{2}{n-1} + \frac{\kappa}{n}\right)$?

In his answer to my previous question, @Erik P. gives the expression $$ \mathrm{Var}[s^2]=\sigma^4 \left(\frac{2}{n-1} + \frac{\kappa}{n}\right) \>, $$ where $\kappa$ is the excess kurtosis of the ...
15
votes
0answers
841 views

Practical thoughts on explanatory vs predictive modeling [duplicate]

Possible Duplicate: Practical thoughts on explanatory vs. predictive modeling This question has been bugging me for some time, and I was going to write a blog post about it. However, I think it ...
60
votes
2answers
81k views

Maximum likelihood method vs. least squares method

What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ? Why can't we use MLE for predicting $y$ values in linear regression and vice versa? Any ...
58
votes
8answers
9k views

Examples where method of moments can beat maximum likelihood in small samples?

Maximum likelihood estimators (MLE) are asymptotically efficient; we see the practical upshot in that they often do better than method of moments (MoM) estimates (when they differ), even at small ...
27
votes
4answers
113k views

How to derive the likelihood function for binomial distribution for parameter estimation?

According to Miller and Freund's Probability and Statistics for Engineers, 8ed (pp.217-218), the likelihood function to be maximised for binomial distribution (Bernoulli trials) is given as $L(p) = \...
6
votes
2answers
246 views

What is the relationship between minimizing prediction error versus parameter estimation error?

With the advent of statistical learning techniques, people are talking a lot about prediction error, while in classical statistics, one is focusing on parameter estimation error. What is the ...
6
votes
1answer
10k views

Variance of the reciprocal II

Background I've recently read the paper Leo A. Goodman, On the Exact Variance of Products Journal of the American Statistical Association Vol. 55, No. 292 (Dec., 1960), pp. 708-713 from where I ...
4
votes
1answer
3k views

What is the distribution of the variance of a sample from an unknown distribution?

I am sampling from a parameter with unknown distribution. I would like to calculate a 95% CI for the standard deviation of the sample. @cardinal provides a nice general solution for calculating a CI ...
1
vote
2answers
876 views

Seasonal adjustment for a series that has already been adjusted

A dataset I am working with (from the OECD), for harmonised unemployment seems to be seasonally adjusted: The unemployment rates shown here are calculated as the number of unemployed persons as a ...
2
votes
1answer
746 views

Estimate normal distribution from small sample with rankings

If I have n samples from an $N(\mu,\sigma)$ distribution, how can I estimate the distribution from a subset of m of these n samples where the order within those n is known. For example, if n = 100 ...
27
votes
6answers
17k views

How to calculate Zipf's law coefficient from a set of top frequencies?

I have several query frequencies, and I need to estimate the coefficient of Zipf's law. These are the top frequencies: ...
12
votes
1answer
2k views

Is the mean (Bayesian) posterior estimate of $\theta$ a (Frequentist) unbiased estimator of $\theta$?

I am wondering about the different ways that Bayesian and Frequentist statistic connect with each other. I recalled that the Maximum Likelihood estimate of a parameter $\theta$ is not necessarily an ...
12
votes
2answers
9k views

What are complete sufficient statistics?

I have some trouble understanding complete sufficient statistics? Let $T=\Sigma x_i$ be a sufficient statistic. If $E[g(T)]=0$ with probability 1, for some function $g$, then it is a complete ...
11
votes
1answer
2k views

James-Stein Estimator with unequal variances

Every statement I find of the James-Stein estimator assumes that the random variables being estimated have the same (and unit) variance. But all of these examples also mention that the JS estimator ...
9
votes
1answer
647 views

Does adjusted R-square seek to estimate fixed score or random score population r-squared?

Population r-square $\rho^2$ can be defined assuming fixed scores or random scores: Fixed scores: The sample size and the particular values of the predictors are held fixed. Thus, $\rho^2_f$ is the ...
3
votes
1answer
1k views

Relation Between Bayesian Estimation and Maximum a posteriori estimation

Is maximum a posteriori estimation some kind of Bayesian Estimation? If yes, can you point out other Bayesian estimators? Edit: So I've come to know the following (don't know if they are correct): ...
19
votes
3answers
1k views

How to do estimation, when only summary statistics are available?

This is in part motivated by the following question and the discussion following it. Suppose the iid sample is observed, $X_i\sim F(x,\theta)$. The goal is to estimate $\theta$. But original sample ...
6
votes
3answers
15k views

Standard error of a ratio

I have a linear regression and two estimates, say A and B and their standard errors. I need to find the standard error of a ratio A/B [or A/(1-B)]. I guess the main problem is that I don't know the ...
12
votes
4answers
10k views

How large should a sample be for a given estimation technique and parameters?

Is there a rule-of thumb or even any way at all to tell how large a sample should be in order to estimate a model with a given number of parameters? So, for example, if I want to estimate a least-...

1
2 3 4 5
9