Questions tagged [estimation]
This tag is too general; please provide a more specific tag. For questions about the properties of specific estimators, use [estimators] tag instead.
489 questions
83
votes
4
answers
89k
views
Why is sample standard deviation a biased estimator of $\sigma$?
According to the Wikipedia article on unbiased estimation of standard deviation the sample SD
$$s = \sqrt{\frac{1}{n-1} \sum_{i=1}^n (x_i - \overline{x})^2}$$
is a biased estimator of the SD of the ...
- 847
109
votes
7
answers
189k
views
Calculating the parameters of a Beta distribution using the mean and variance
How can I calculate the $\alpha$ and $\beta$ parameters for a Beta distribution if I know the mean and variance that I want the distribution to have? Examples of an R command to do this would be most ...
- 1,728
10
votes
1
answer
513
views
Trying to Estimate Disease Prevalence from Fragmentary Test Results
In response to the spread of COVID-19 disease, all Californians were ordered on 19 March 2020 to stay at home, except for such necessary errands as trips to grocery stores, pharmacies, etc. On 21 ...
- 57.6k
68
votes
6
answers
78k
views
What is the difference between estimation and prediction?
For example, I have historical loss data and I am calculating extreme quantiles (Value-at-Risk or Probable Maximum Loss). The results obtained is for estimating the loss or predicting them? Where can ...
- 681
16
votes
2
answers
3k
views
Estimating parameters of a binomial model
First of all I'd like to precise that I'm not an expert of the subject.
Suppose to have two random variables $X$ and $Y$ that are binomial, respectively $X\sim B(n_1,p)$ and $Y\sim B(n_2,p),$ note ...
- 261
12
votes
2
answers
4k
views
Can we reject a null hypothesis with confidence intervals produced via sampling rather than the null hypothesis?
I have been taught that we can produce a parameter estimate in the form of a confidence interval after sampling from a population. For example, 95% confidence intervals, with no violated assumptions, ...
- 321
42
votes
5
answers
39k
views
What is a Highest Density Region (HDR)?
In statistical inference, problem 9.6b, a "Highest Density Region (HDR)" is mentioned. However, I didn't find the definition of this term in the book.
One similar term is the Highest Posterior ...
- 1,122
77
votes
15
answers
13k
views
Why would parametric statistics ever be preferred over nonparametric?
Can someone explain to me why would anyone choose a parametric over a nonparametric statistical method for hypothesis testing or regression analysis?
In my mind, it's like going for rafting and ...
- 957
18
votes
5
answers
4k
views
Can the empirical Hessian of an M-estimator be indefinite?
Jeffrey Wooldridge in his Econometric Analysis of Cross Section and Panel Data (page 357) says that the empirical Hessian "is not guaranteed to be positive definite, or even positive semidefinite, for ...
21
votes
4
answers
21k
views
Calculating required sample size, precision of variance estimate?
Background
I have a variable with an unknown distribution.
I have 500 samples, but I would like demonstrate the precision with which I can calculate variance, e.g. to argue that a sample size of 500 ...
- 3,901
61
votes
3
answers
48k
views
Standard deviation of standard deviation
What is an estimator of standard deviation of standard deviation if normality of data can be assumed?
user88
41
votes
6
answers
7k
views
If a credible interval has a flat prior, is a 95% confidence interval equal to a 95% credible interval?
I'm very new to Bayesian statistics, and this may be a silly question. Nevertheless:
Consider a credible interval with a prior that specifies a uniform distribution. For example, from 0 to 1, where 0 ...
- 823
37
votes
4
answers
17k
views
Internal vs external cross-validation and model selection
My understanding is that with cross validation and model selection we try to address two things:
P1. Estimate the expected loss on the population when training with our sample
P2. Measure and report ...
- 19.7k
25
votes
5
answers
51k
views
What is the relation between estimator and estimate?
What is the relation between estimator and estimate?
user3412
43
votes
1
answer
19k
views
Maximum likelihood estimators for a truncated distribution
Consider $N$ independent samples $S$ obtained from a random variable $X$ that is assumed to follow a truncated distribution (e.g. a truncated normal distribution) of known (finite) minimum and maximum ...
- 707
12
votes
1
answer
8k
views
Are estimates of regression coefficients uncorrelated?
Consider a simple regression (normality not assumed): $$Y_i = a + b X_i + e_i,$$ where $e_i$ is with mean $0$ and standard deviation $\sigma$. Are the Least Square Estimates of $a$ and $b$ ...
- 605
7
votes
7
answers
4k
views
Is there a GLM bible?
Is there consensus in the field of statistics that one book is the absolute best source and completely covering every aspect of GLM - detailing everything from estimation to inference?
5
votes
1
answer
4k
views
ARMA/GARCH estimation in sequence
I have a time series that shows a nonstationary seasonal autoregressive component as well as known heteroshedasticity. In order to model the series, I have fit a seasonal ARIMA model for the mean with ...
- 3,712
3
votes
1
answer
621
views
How to find quantiles and likelihoods of mixture distributions?
My PDF:
M was estimated and found to be 5.
I need to work out the quartiles for the PDF above. In addition, I need to use different methods of estimation to estimate the parameters. So far I've ...
15
votes
2
answers
4k
views
What is an unbiased estimate of population R-square?
I am interested in getting an unbiased estimate of $R^2$ in a multiple linear regression.
On reflection, I can think of two different values that an unbiased estimate of $R^2$ might be trying to ...
- 45.7k
13
votes
3
answers
2k
views
Revisiting the Rule of Three
The rule of three is a method for calculating a 95% confidence interval when estimating $p$ from a set of $n$ IID Bernoulli trials with no successes.
My understanding from its derivation is that the ...
- 1,463
11
votes
2
answers
17k
views
How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?
I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
- 229
9
votes
2
answers
4k
views
Choosing the number of bootstrap resamples
Say we have the following sample and we are trying to estimate the variance of the sample mean of the population.
X = [0, -1, 2, 10, -3]
If I take an increasing ...
- 4,598
32
votes
4
answers
27k
views
Estimating parameters of Student's t-distribution
What are the maximum-likelihood estimators for the parameters of Student's t-distribution? Do they exist in closed form? A quick Google search didn't give me any results.
Today I am interested in the ...
- 755
22
votes
2
answers
24k
views
Kullback-Leibler Divergence for two samples
I tried to implement a numerical estimate of the Kullback-Leibler Divergence for two samples. To debug the implementation draw the samples from two normal distributions $\mathcal N (0,1)$ and $\...
- 355
19
votes
4
answers
33k
views
How to keep time invariant variables in a fixed effects model
I have data on a large Italian firm's employees over ten years and I would like to see how the gender gap in male-female earnings has changed over time. For this purpose I run pooled OLS:
$$
y_{it} = ...
- 193
14
votes
4
answers
2k
views
Is any quantitative property of the population a "parameter"?
I'm relatively familiar with the distinction between the terms statistic and parameter. I see a statistic as the value obtained from applying a function to the sample data. However, most examples of ...
- 45.7k
9
votes
2
answers
2k
views
What is the relationship between minimizing prediction error versus parameter estimation error?
With the advent of statistical learning techniques, people are talking a lot about prediction error, while in classical statistics, one is focusing on parameter estimation error. What is the ...
- 3,184
40
votes
5
answers
191k
views
How to derive the likelihood function for binomial distribution for parameter estimation?
According to Miller and Freund's Probability and Statistics for Engineers, 8ed (pp.217-218), the likelihood function to be maximised for binomial distribution (Bernoulli trials) is given as
$L(p) = \...
- 1,092
41
votes
4
answers
59k
views
Why squared residuals instead of absolute residuals in OLS estimation? [duplicate]
Why are we using the squared residuals instead of the absolute residuals in OLS estimation?
My idea was that we use the square of the error values, so that residuals below the fitted line (which are ...
- 2,399
29
votes
3
answers
41k
views
How to choose prior in Bayesian parameter estimation
I know 3 methods to do parameter estimation, ML, MAP and Bayes approach. And for MAP and Bayes approach, we need to pick priors for parameters, right?
Say I have this model $p(x|\alpha,\beta)$, in ...
- 3,653
15
votes
3
answers
8k
views
Can I use Kolmogorov-Smirnov test and estimate distribution parameters?
I've read that Kolmogorov-Smirnov test should not be used to test the goodness of fit of a distribution whose parameters have been estimated from the sample.
Does make sense to split my sample in two ...
- 251
4
votes
2
answers
1k
views
Estimating a normal distribution from three order statistics
I am interested in predicting a normal distribution, but not sure if this is possible.
I do not have information on the mean or standard deviation. However, I know the range of values, let's say ...
- 151
3
votes
1
answer
1k
views
Estimate normal distribution from small sample with rankings
If I have n samples from an $N(\mu,\sigma)$ distribution, how can I estimate the distribution from a subset of m of these n samples where the order within those n is known. For example, if n = 100 ...
- 33
80
votes
4
answers
142k
views
Maximum likelihood method vs. least squares method
What is the main difference between maximum likelihood estimation (MLE) vs. least squares estimaton (LSE) ?
Why can't we use MLE for predicting $y$ values in linear regression and vice versa?
Any ...
- 901
26
votes
3
answers
3k
views
Unbiased estimation of covariance matrix for multiply censored data
Chemical analyses of environmental samples are often censored below at reporting limits or various detection/quantitation limits. The latter can vary, usually in proportion to the values of other ...
- 334k
61
votes
8
answers
12k
views
Examples where method of moments can beat maximum likelihood in small samples?
Maximum likelihood estimators (MLE) are asymptotically efficient; we see the practical upshot in that they often do better than method of moments (MoM) estimates (when they differ), even at small ...
- 290k
16
votes
3
answers
3k
views
Computationally efficient estimation of multivariate mode
Short version: What's the most computationally efficient method of estimating the mode of a multidimensional data set, sampled from a continuous distribution?
Long version: I've got a data set that I ...
- 283
13
votes
2
answers
8k
views
Reference for $\mathrm{Var}[s^2]=\sigma^4 \left(\frac{2}{n-1} + \frac{\kappa}{n}\right)$?
In his answer to my previous question, @Erik P. gives the expression
$$
\mathrm{Var}[s^2]=\sigma^4 \left(\frac{2}{n-1} + \frac{\kappa}{n}\right) \>,
$$
where $\kappa$ is the excess kurtosis of the ...
- 3,901
3
votes
2
answers
2k
views
Regression and the CEF
I recently read in this page (https://www.timlrx.com/2018/02/26/notes-on-regression-approximation-of-the-conditional-expectation-function/#fn1) that:
"Regression offers a way of approximating ...
38
votes
4
answers
3k
views
Are inconsistent estimators ever preferable?
Consistency is obviously a natural and important property of estimators, but are there situations where it may be better to use an inconsistent estimator rather than a consistent one?
More ...
- 12.1k
36
votes
3
answers
6k
views
How to find confidence intervals for ratings?
Evan Miller's "How Not to Sort by Average Rating" proposes using the lower bound of a confidence interval to get a sensible aggregate "score" for rated items. However, it's working with a Bernoulli ...
- 413
23
votes
4
answers
7k
views
How can I estimate unique occurrence counts from a random sampling of data?
Let's say I have a large set of $S$ values which sometimes repeat. I wish to estimate the total number of unique values in the large set.
If I take a random sample of $T$ values, and determine that ...
- 390
19
votes
1
answer
6k
views
What is "Targeted Maximum Likelihood Expectation"?
I'm trying to understand some papers by Mark van der Laan. He's a theoretical statistician at Berkeley working on problems overlap significantly with machine learning. One problem for me (besides ...
- 311
15
votes
0
answers
869
views
Practical thoughts on explanatory vs predictive modeling [duplicate]
Possible Duplicate:
Practical thoughts on explanatory vs. predictive modeling
This question has been bugging me for some time, and I was going to write a blog post about it. However, I think it ...
- 151
10
votes
1
answer
13k
views
Variance of the reciprocal II
Background
I've recently read the paper
Leo A. Goodman, On the Exact Variance of Products
Journal of the American Statistical Association
Vol. 55, No. 292 (Dec., 1960), pp. 708-713
from where I ...
- 537
7
votes
2
answers
2k
views
Comparing estimators of location of the Cauchy distribution
I'm comparing the following 4 estimators of location of the Cauchy distribution:
Let $x_{1},..x_{n}$ be observations and $l$ be the log likelihood function.
$x=median(x_{1},..x_{n})$, $y=x+\frac{l'(x)...
- 351
1
vote
1
answer
1k
views
Finding maximum likelihood estimator, symmetric uniform distribution
Let $X_1, ...X_n$ be IID random variables with uniform$[ -\theta , \theta ]$ . I need to find the Maximum Likelihood estimator (MLE) of $\theta$.
My work is as follows,
The likelihood function is ,
...
- 348
30
votes
7
answers
21k
views
How to calculate Zipf's law coefficient from a set of top frequencies?
I have several query frequencies, and I need to estimate the coefficient of Zipf's law. These are the top frequencies:
...
- 319
20
votes
4
answers
1k
views
Under which conditions do Bayesian and frequentist point estimators coincide?
With a flat prior, the ML (frequentist -- maximum likelihood) and the MAP (Bayesian -- maximum a posteriori) estimators coincide.
More generally, however, I'm talking about point estimators derived as ...
- 852