Questions tagged [unbiased-estimator]
Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.
745
questions
0
votes
0
answers
21
views
Is the estimator 0.5X1 + 0.5(n-1)^(-1) * the sum from i=2 to n of Xi an unbiased estimator? Is it consistent?
Let {Xi} from i=1 to n be an i.i.d. sample from a distribution f.
I suspect this is unbiased, but is it consistent? I'm not sure how to approach it as I think the variance converges to 0, but won't it ...
- 1
0
votes
1
answer
123
views
Derivation of unbiased MLE for Gaussian variance
I'm currently studying ML basics with the book Introduction to Machine Learning (Ethem Alpaydin) and had a question regarding checking whether the maximum likelihood estimators (MLE's) for a Gaussian ...
- 2,901
1
vote
0
answers
107
views
Unbiased estimator for a parameter from a transformed distribution
I am solving an exercise in which I have to show that a certain estimator is unbiased for a given parameter. However, after a couple lines of computation I got stuck in the following scenario:
$$
\...
- 37
5
votes
1
answer
371
views
(From van der Vaart's Asymptotic Statistics, page 161, U-statistic) Why we can always replace the function $h$ with a symmetric one?
I'm reading the following Chapter from van der Vaart's Asymptotic Statistics, Section 12.1 page 161 (see the screenshot below). For the $h$ function that it mentioned, I have two questions regarding ...
- 2,422
1
vote
0
answers
75
views
Using variance of sample to calculate unbiased estimate of population variance
I am trying to follow through the survey sampling chapter of Rice's statistics book.
Denote the sample values by $X_1, X_2, \ldots, X_n$ and the population values by $x_1, x_2, \ldots, x_N$ (so the ...
- 346
1
vote
1
answer
94
views
What is the distinction between bias in prediction and parameter estimation?
I am trying to understand the distinction between bias in prediction and parameter estimation. This example in Gelman, Bayesian Data Analysis, 2nd ed. 2004 pp. 255-256 is very confusing to me.
Why do ...
- 799
1
vote
1
answer
163
views
validation error and test error when limited data is available
In machine learning, to get an unbiased estimate of model performance, we split data 80:20 into train and test set. We use the training set for model training and model selection according to cross-...
- 749
2
votes
2
answers
151
views
"... because sample mean gets different values from sample to sample and it is a random variable with mean $\mu$ and variance $\frac{\sigma^2}{n}$."
This answer by user "sevenkul" says the following:
The sample mean $\overline{X}$ also deviates from $\mu$ with variance $\frac{\sigma^2}{n}$ because sample mean gets different values from ...
- 1,414
2
votes
1
answer
197
views
Citation: Sample mean as consistent and unbiased estimator of the expected value
A reviewer asked for a citation that the sample mean is a consistent and unbiased estimator of the expected value and therefore converges towards the expected value. I know I can easily do the ...
- 2,501
0
votes
1
answer
253
views
Intuition behind unbiased OLS estimator derivation
I was going through the derivation of unbiased OLS estimator
$$E(\hat{\beta_1}) = \beta_1 + (1/SST_x) \sum_{i=1}^n d_i E(u_i) = \beta_1 + (1/SST_x) \sum_{i=1}^n d_i\cdot 0 = \beta_1$$
My doubt is if $...
0
votes
1
answer
80
views
Unbiased estimatior for $\bar{x} $ from a Random Sample with unequal selection probability
I have the following population:
Where the left column is the age of our individuals and the right column is their weight (in kg).
The exercise tells us that we use Random Sampling with no ...
- 23
1
vote
0
answers
24
views
Negatively correlated estimators for the AR-1 process
I have the following question. Assume we have a stochastic process
\begin{equation}
y_t = \gamma + \phi y_{t-1} + \epsilon_t, \ \epsilon_t \sim \mathcal{N}(0, \sigma^2),
\end{equation}
where $|\phi| &...
- 156
0
votes
1
answer
715
views
mixed model variance-covariance matrix| parameter estimation
I am fairly new to LMM's and I am trying to undestand how the parameter estimation happens;
According to this:
Beta is obtained with equation 13.28. Beta is supposed to be the parameters for the ...
- 45
2
votes
0
answers
338
views
proof of Cramer-Rao lower bound
I am trying to understand the proof for this theorem from the book Casella and Berger (2nd ed.) page 336. $W(X)$ if any estimator for samples $X_1,\ldots,X_n$ on distribution $f(X|\theta)$. I notice ...
- 311
1
vote
0
answers
69
views
Evaluation of Limit involved in the proof of Asymptotic Unbiasedness
We know that $S^{2}$ is an unbiased estimator of $\sigma^{2}$ and $S$ is a biased estimator of $\sigma$. But if $n\rightarrow\infty$, then $S$ is an asymptotically unbiased estimator of $\sigma$. I ...
- 11
0
votes
0
answers
18
views
Variance estimator using mixture of scaled and unscaled data
Given two datasets:
$X_1, \dots, X_n \sim N(1, \sigma^2)$
and
$X_{n+1}, \dots, X_N \sim N(1, 2\sigma^2)$
My proposed estimator for $\sigma^2$ is simply a scaled combination of both classical ...
- 154
2
votes
0
answers
173
views
What is the difference between bias in a beta coefficient estimate and bias as a property of an estimator?
I was thinking about bias in the context of simulation studies - defined as the average of the difference between the estimated beta parameter estimate and the true parameter estimate across all ...
- 41
1
vote
2
answers
373
views
Is a convolutional neural network unbiased? Is it a regularized multilayer perceptron?
"Is a convolutional neural network biased?"
This came up in an interview I had a few years ago, and I’ve recently thought of it.
I think it’s a misguided question. Imagine this related ...
- 51k
5
votes
1
answer
250
views
Point estimator for product of independent RVs
Let $X$ and $Y$ be two independent random variables. Given an (iid) random sample of size $n$ of $X$ and a random sample of size $n$ of $Y$, what is a good way to estimate the mean of their product, $...
- 309
0
votes
0
answers
102
views
Calculating consistent estimators
Let $X_1, X_2,\dots$ be $iid$ random variables with density $f(x|p), 0<p<1$ being the unknown parameter. Suppose that there exists an unbiased estimator T of $p$ based on sample size 1, i.e. $E(...
- 104
1
vote
0
answers
591
views
Finding Best Linear Unbiased Estimator
I have the doubt that if Gauss Markov theorem is applicable here since the Variance is not constant in the model. Without Gauss Markov Theorem, how can we obtain BLUE?
- 43
0
votes
0
answers
245
views
Unbiased Estimator of AR(1) Models?
What are the options for unbiased estimators of AR(1) (or AR(p)) models?
Bias reduction techniques may also be included (jack knife would be one). I found one paper called "Bias correction using ...
- 323
0
votes
0
answers
23
views
How to show that the variance estimator of a gaussian is biased? [duplicate]
The sample variance of $m$ samples from a gaussian is $$ \hat{\sigma}^2_m=\frac{1}{m}\sum_{i=1}^m(x^{(i)}-\hat{\mu}_m)^2$$
How do i show that the sample variance $\hat{\sigma}_m^2$ is biased ?
I.e $$...
- 1,076
0
votes
0
answers
95
views
Based on the record X1 ,…, Xn what is the unbiased estimation of 1/p [duplicate]
If we investigate $n$ patients for SARS. The indicator of sequence of the trails is $X_i$ ($X_i=1$ is for success and $0$ is not success). And the sequence indicator is available for all n independent ...
1
vote
1
answer
201
views
In a weighted least squares regression, can we use the weight as a control variable?
I have found Weighted Least Squares with Endogenous Weights but the answers primarily tackle the question of when $w_i$ correlates with $\epsilon_i$. I would like to ask if we use $w_i$ as a control ...
- 113
0
votes
3
answers
1k
views
Show that the two estimators are unbiased for $\theta$ [closed]
$X_1$ and $X_2$, one accurate than the other, are subject to the standard deviations, $\sigma$ and 1.25$\sigma$
respectively. $X_1$ occurred 6 independent times, giving a mean of
$\bar{x}_1$ while $...
- 11
0
votes
1
answer
233
views
Prove bias/unbias-edness of mean/median estimators for lognormal
Looking at a problem where X is lognormally distributed from normal distribution Y, which asks me to prove that:
1) $e^{\bar{y}}$ is a biased estimator for the median of X
2) $e^{\bar{y} - \sigma^2 /...
- 125
1
vote
0
answers
673
views
Uniform distribution, estimates, MVUEs and Cramer Rao Lower Bound
As a revision exercise, I'm going through all of the distributions and deriving estimators. I've gotten to the $Uniform$. I've worked out the MLE and MOM estimators. The next step is to consider ...
2
votes
1
answer
347
views
Doubt on derivation of OLS estimators as unbiased estimators of Optimal Linear Predictors
I'm studying from C. Shalizi's lecture notes https://www.stat.cmu.edu/~cshalizi/ADAfaEPoV/ .
In the third chapter he introduces the optimal linear estimator of a random variable $Y$ conditioned to ...
- 151
0
votes
0
answers
408
views
Minimizing Mean Square Error
Suppose we have a random sample $\textbf{X}=(X_1,...,X_n)$ from a shifted exponential distribution with common density
$f(x|\theta)=\left\{\begin{matrix}
e^{-(x-\theta)} & x\geq \theta\\
0 & ...
2
votes
1
answer
257
views
Testing the equality of two multivariate mean vectors $μ_1$ and $μ_2$ based on independent random normal samples
Let $X_1,...,X_{n_1}$ be an i.i.d. sample from $N_p(\mu_1,\Sigma)$ and let $Y_1,...,Y_{n_2}$ be an independent sample from $N_p(\mu_2,\Sigma)$, for some $\mu_1,\mu_2 \in \mathbb{R}^p$ and some ...
user255658
2
votes
1
answer
353
views
MAE regression gives biased regression parameters for symmetric error?
Consider a linear model,
$$
y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \epsilon_i.
$$
From the Gauss-Markov theorem, I know that, under nice conditions, the $\hat{\beta}_{OLS}=(X^TX)^{-1}X^Ty$ ...
- 51k
0
votes
1
answer
693
views
Unbiased estimator and biased error
I'm having some trouble relating unbiased estimators and bias error. By bias error, I mean the bias error we talk about when analyzing "bias-variance tradeoffs." Is this bias error and an unbiased ...
- 281
0
votes
0
answers
362
views
Estimator of $\log \mathbb{E}[X]$
In many fields of statistic we are faced with quantities of type $\log \mathbb{E}[X]$ where $X$ is a generic random variable. However, I never came across any good estimator for this quantity. The ...
- 101
0
votes
0
answers
37
views
Does minimizing the mean of Varience of an unbiased estimator by selecting the values of ${\bf{r}}$, imply minimizing the mean of CRLB?
I can minimize the mean of variance of an unbiased estimator of a paramater $\theta$ by selecting the values of a set of parameters, ${\bf{r}}$. So i can minimeze ${\rm{E[Va}}{{\rm{r}}_{\hat \theta }}...
- 53
3
votes
1
answer
82
views
Combining importance sampling with optimization - does this yield an unbiased estimate?
I'm wondering if it is OK to combine importance sampling with optimization to choose the parameters for the substitute distribution.
I have a non-negative random variable $X$ on $\mathbb{R}^d$ with ...
- 6,433
4
votes
1
answer
215
views
Unbiased Estimator of Largest Mean of Two Normal Distributions
Given samples from two normal distributions:
$X_i \stackrel{iid}{\sim} \mathcal{N}(\mu_X, \sigma_X^2)$ for $i = 1,...,n$
$Y_i \stackrel{iid}{\sim} \mathcal{N}(\mu_Y, \sigma_Y^2)$ for $i = 1,...,n$
How ...
1
vote
1
answer
164
views
How can I find the BUE of $\theta$ in the simple linear relationship $Y_i=\theta x_i^2+\epsilon_i$?
Let $Y_1,...,Y_n$ be described by the relationship $Y_i=\theta x_i^2+\epsilon_i$, where $x_1,...,x_n$ are fixed constants and $\epsilon_1,...,\epsilon_n$ are iid $N(0,\sigma^2)$. How can I find the ...
- 1,799
7
votes
1
answer
531
views
How do I find the UMVUE of $\sqrt{\alpha}$ here?
new user here self-studying some mathematical statistics. I came across this problem and am stuck.
Problem: Suppose that for $i = 1, ... , n$, the positive random variables $X_i$ are independent and ...
- 391
4
votes
0
answers
62
views
The expected value of $\frac{1}{\sqrt{1-r}}$ where $r$ is Pearson correlation
I am looking to unbias the sample statistic $\frac{1}{\sqrt{1-r}}$ where $r$ is a Pearson correlation. The population is assumued binormal with equal variance $\sigma$ and with true correlation $\rho$....
- 170
0
votes
1
answer
48
views
Unbiased estimator of variation of median in spatial bins using bootstrap method
Say I have a satellite that's flying through the atmosphere, over multiple orbits, sampling its density at different altitudes, at say 1 measurement per second (specific numbers are irrelevant). The ...
- 59
0
votes
0
answers
45
views
Finding good estimators for a function of bernoulli parameter [duplicate]
Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ l'm interested in finding estimator of $(1-\theta)^{1 / k},$ when $k$ is a positive integer. I am considering the following ...
- 195
5
votes
2
answers
1k
views
Finding UMVUE for a function of a Bernoulli parameter
Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ I'm interested in finding the UMVUE of $(1-\theta)^{1/k}$, when $k$ is a positive integer. .
I know $\sum X_{i}$ is a ...
- 195
3
votes
1
answer
232
views
Unbiased estimator of standard deviation
I'm reading "Properties of range-based volatility estimators" where the authors talk about using the range of a distribution ($h$ - $l$) to estimate its volatility. Specifically, they say,
Daily ...
- 546
2
votes
1
answer
328
views
Are all estimators biased? Is the unbiasedness only a theoretical or approximation case?
The definition of unbiased estimator says that it's expected value has no difference comparing to a true value. So can we say that all estimators are biased (even slightly)? I thought that only in ...
- 299
0
votes
0
answers
39
views
Showing that a estimator is biased?
I am solving a exercise who asks me to show that one estimator is biased. Given the function
\begin{equation}
f(x|\theta) = \left( (1-\sigma) + \dfrac{\sigma}{2\sqrt{x}} \right)I_{[0,1]}(x), \sigma \...
- 1
3
votes
1
answer
968
views
Why is temporal difference learning biased in reinforcement learning?
When I learn reinforcement learning from David Silver's online video, I saw "the objective of TD learning, $r_t + \gamma V(s_{t+1})$ is a biased target for learning value function. " I know the ...
- 1,863
1
vote
2
answers
1k
views
How is the sample mean an unbiased estimator of the population mean via deeplearningbook.org?
So I know that the sample mean is a unbiased estimator of the population mean. Just wondering how the author gets from 5.33 to 5.34 in the below.
How do you get from $\mathbb{E}[\mu_m]$ to just $\mu$...
- 2,623
0
votes
1
answer
362
views
Does bias mean additional constant in any estimator? Can I say proportional estimator unbiased estimator?
I saw one question in which the sample mean was estimated as follows (I don't know why they divided by $n-1$ instead of $n$ here for estimation),
$$
\widehat{\mu} = \frac{\sum_{i=1}^n x_i}{n-1}
$$
...
- 155
0
votes
0
answers
193
views
Which statistic to choose - biased or unbiased?
In a book on introductory statistics, there is a question that goes like this:
If two statistics are available for estimating a population
characteristic, under what circumstances might you ...
- 157