As of May 31, 2023, we have updated our Code of Conduct.

# Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

745 questions
Filter by
Sorted by
Tagged with
193 votes
3 answers
232k views

### What is the difference between a consistent estimator and an unbiased estimator?

What is the difference between a consistent estimator and an unbiased estimator? The precise technical definitions of these terms are fairly complicated, and it's difficult to get an intuitive feel ...
• 2,680
85 votes
5 answers
50k views

### How exactly did statisticians agree to using (n-1) as the unbiased estimator for population variance without simulation?

The formula for computing variance has $(n-1)$ in the denominator: $s^2 = \frac{\sum_{i=1}^N (x_i - \bar{x})^2}{n-1}$ I've always wondered why. However, reading and watching a few good videos about "...
• 14.2k
56 votes
2 answers
6k views

• 251
14 votes
4 answers
4k views

### How does one explain what an unbiased estimator is to a layperson?

Suppose $\hat{\theta}$ is an unbiased estimator for $\theta$. Then of course, $\mathbb{E}[\hat{\theta} \mid \theta] = \theta$. How does one explain this to a layperson? In the past, what I have said ...
• 4,213
14 votes
2 answers
1k views

### Unbiased estimator of exponential of measure of a set?

Suppose we have a (measurable and suitably well-behaved) set $S\subseteq B\subset\mathbb R^n$, where $B$ is compact. Moreover, suppose we can draw samples from the uniform distribution over $B$ wrt ...
14 votes
5 answers
488 views

### Different usage of the term "Bias" in stats/machine learning

I think I've seen about 4 different usages of the word "bias" in stats/ML, and all these usages seem to be non-related. I just wanted to get clarification that the usages are indeed non-...
14 votes
2 answers
1k views

### Model for population density estimation

A database of (population, area, shape) can be used to map population density by assigning a constant value of population/area to each shape (which is a polygon such as a Census block, tract, county, ...
• 309k
14 votes
1 answer
2k views

### Does a median-unbiased estimator minimize mean absolute deviance?

This is a follow-up but also a different question of my previous one. I read on Wikipedia that "A median-unbiased estimator minimizes the risk with respect to the absolute-deviation loss function, as ...
• 393
14 votes
1 answer
959 views

### Biased estimator for regression achieving better results than unbiased one in Error In Variables Model

I am working on some syntatic data for Error In Variable model for some research. Currently I have a single independent variable, and I am assuming I know the variance for the true value of the ...
• 191
13 votes
2 answers
25k views

### Correct equation for weighted unbiased sample covariance

I'm looking for the correct equation to compute the weighted unbiased sample covariance. Internet sources are quite rare on this theme and they all use different equations. The most likely equation I'...
• 717
13 votes
4 answers
1k views

### Unbiased estimator for the smaller of two random variables

Suppose $X \sim \mathcal{N}(\mu_x, \sigma^2_x)$ and $Y \sim \mathcal{N}(\mu_y, \sigma^2_y)$ I am interested in $z = \min(\mu_x, \mu_y)$. Is there an unbiased estimator for $z$? The simple estimator ...
• 131
12 votes
4 answers
1k views

### Why isn't this estimator unbiased?

Suppose we have a IID sample $X_1, X_2, \cdots, X_n$ with each $X_i$ distributed as $\mathcal{N}(\mu, \sigma^2)$. Now suppose we construct (a rather peculiar) estimator for the mean $\mu$: we only ...
• 603
12 votes
5 answers
3k views

### Does the biased estimator always have less variance than unbiased one?

Suppose I am estimating one of the parameter. Now if we plot the biased estimator of that and unbiased estimator of that can we say for sure that biased one has less variance than unbiased one always. ...
• 279
12 votes
2 answers
7k views

### Why is bias equal to zero for OLS estimator with respect to linear regression?

I understand the concept of bias-variance tradeoff. Bias based on my understanding, represents the error because of using a simple classifer(eg: linear) to capture a complex non-linear decision ...
• 5,493
11 votes
2 answers
17k views

### Using MSE instead of log-loss in logistic regression

Suppose we replace the loss function of the logistic regression (which is normally log-likelihood) with the MSE. That is, still have log odds ratio be a linear function of the parameters, but minimize ...
• 1,584
11 votes
1 answer
1k views

### Should the standard deviation be corrected in a Student's T test?

Using the Student's T test, T-Critical is calculated via: $t = \frac{\bar{X} - \mu_{0}}{s / \sqrt{n}}$ Looking at Wikipedia article on the unbiased Estimation of the standard deviation, there ...
• 390
11 votes
2 answers
4k views

### Bias / variance tradeoff math

I understand the matter in the underfitting / overfitting terms but I still struggle to grasp the exact math behind it. I've checked several sources (here, here, here, here and here) but I still don't ...
• 291
11 votes
2 answers
3k views

### Degrees of Freedom In Sample Variance

Recall the formula for sample variance $$s_{n - 1}^2 = \dfrac{1}{n -1} \sum_{i = 1}^n (\bar{x} - x_i)^2,$$ where $\bar{x}$ is the sample mean. There are many proofs for why $s_{n - 1}^2$ is an ...
11 votes
1 answer
3k views

### When is having an unbiased estimator important?

We have a few questions and answers about when one would prefer a biased estimate over a unbiased one, but I have not found anything on the reverse question: In what situations is it important to ...
• 34.9k
11 votes
2 answers
830 views

### Bayes estimator are immune to selection Bias

Are Bayes estimators immune to selection bias? Most papers that discuss estimation in high dimension, e.g., whole genome sequence data, will often raise the issue of selection bias. Selection bias ...