Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

Filter by
Sorted by
Tagged with
5
votes
2answers
44 views

Unbiased estimator of $ 1 + \mu^{2}$ from a Normal population

Question: If $ x_{1}, x_{2}, x_{3},...x_{n}$ is a random sample from a $Normal$ $population$ $N(\mu,1)$ then what is the unbiased estimator of $ 1 + \mu^{2}$ ? I began finding the mean and variance ...
0
votes
0answers
53 views

Finding the M.V.U.E of n Bernoulli trials [duplicate]

Let $r$ be the observed number of successes in $n$ Bernoulli trials with probability $\pi$ of success. Then M.V.U.E (Minimum Variance Unbiased Estimator) of $\pi (1-\pi)$ is ? $n$ Bernoulli trials ...
7
votes
1answer
143 views

Finding unbiased estimator for Truncated Poisson Distribution

Let $X$ be a single observation from truncated Poisson distribution having probability mass function $P(X = x) =\frac{e^{-\theta} \theta^{x}}{x!(1-e^{-\theta})} ; x = 1,2,3,$ . The estimator $T = \...
1
vote
1answer
54 views

How does computing the bias of model parameters make sense?

I've been studying statistics recently and was thinking about the fact that computing the expectation of a random variable $E(X)$ only really makes sense if $X$ is a random variable defined over a ...
1
vote
1answer
21 views

Bias-variance trade-off in case of biased estimators: is the bias zero?

Consider a data generating process (DGP) that is AR(1): $y_t=\varphi_1 y_{t-1}+\varepsilon_t$ with $\varepsilon_t\sim i.i.D(0,\sigma^2)$ for some distribution $D$ with mean zero and variance $\sigma^2$...
0
votes
0answers
23 views

Simulations: why is "1 by 1" much more efficient than "many by 1"?

Note: I am not familiar with discussions of this particular issue of simulation studies, so I may use wrong terms or oversee obvious aspect. My apologies for that. I want to simulate a two-step ...
1
vote
0answers
25 views

Variance in variance-weighted variance estimate?

Apologies for the confusing title, but I couldn't resist. Much can and has been said about computing the unbiased variance using a sample of points, weighting by the variances of each point (for ...
1
vote
1answer
31 views

Fisher vs. Asymptotic Consistency - Example using a single observation as the population mean estimator

I am learning about Fisher Consistency and came across this section of a Wikipedia article (https://en.wikipedia.org/wiki/Fisher_consistency#Relationship_to_asymptotic_consistency_and_unbiasedness) ...
2
votes
1answer
67 views

Empirical Implications of Unbiased Estimators

I am familiar with the layperson explanation of an unbiased estimator as follows: if we repeat an experiment under identical conditions many times, the average value of the estimate will be close to ...
3
votes
2answers
76 views

confused about unbiasedness of sample mean

Recently, in a different thread, I was convinced by others ( after claiming that they were wrong ) that the sample mean is unbiased for the mean of its underlying distribution. But the case of the chi-...
3
votes
0answers
26 views

Computationally + Statistically Efficient Unbiased Estimation of Chebyshev Polynomials of Expectations

Let $T_n$ denote the $n^\text{th}$ Chebyshev polynomial, defined by the recursion \begin{align} T_0(x) &= 1,\\ T_1(x) &= x,\\ T_n(x) &= 2x \cdot T_{n-1} (x) - T_{n-2} (x). \end{align} Now, ...
0
votes
1answer
31 views

MLE of Variance of Normal Distribution Asymptotically Unbiased?

So the MLE of the variance of a normal distribution, $\sigma^2$, is just the mean squared error, i.e., $\frac{1}{N}\sum_{i=1}^{N} (\hat{y_i} - y_i)^2$. Clearly, this goes to $0$ as $n \rightarrow \...
1
vote
2answers
105 views

Definition of the bias of an estimator

I'm quite confused about the definition of the bias of an estimator. Suppose we have unknown distribution $P(x, \theta)$, and construct the estimator $\hat{\theta}$ that maps the observed data sample ...
9
votes
2answers
843 views

Is it wrong to say that a Riemann sum is an unbiased estimate of an integral?

Would it be wrong to say that a Riemann sum approximation of an integral \begin{align} \int_a^b f(t) \mathrm{d}t \approx \sum_{k=1}^{n_\text{samples}} f(t^{\ast}_k)\Delta t, \end{align} where $\...
0
votes
0answers
13 views

Weak orthogonality, consistency, and unbiasedness of the OLS estimator

My question is based on this question. Suppose we assume the sample is iid (so time series data is out) and $E[e_i X_i ] = 0$ but we're not sure about $E[e_i \mid X_i]$ = 0. Can you provide a ...
1
vote
1answer
68 views

Why does $T$ being an unbiased estimator for $g(\theta)$ imply that $g(\theta) = ET = \int T(\mathbf{y}) f_\theta(\mathbf{y}) \ d\mathbf{y}$?

I am currently studying the Cramer-Rao lower bound. My notes say the following: Theorem: Cramer-Rao lower bound Let $Y_1, \dots, Y_n$ have a joint distribution $f_\theta (\mathbf{y})$, where $f_\...
1
vote
2answers
73 views

Why do we prefer unbiased estimators instead of minimizing MSE?

I was thinking about why, usually, $\hat{\sigma}^2=\hat{p}(1-\hat{p})$ is used to estimate the variance in a Bernoulli population instead of $s^2=\hat{p}(1-\hat{p})\frac{n}{n-1}$. $s^2$ is unbiased, ...
0
votes
1answer
25 views

MSE Proof for an estimator

I am trying to figure out the following proof. The third line is not clear. We all know that (a+b)^2=a^2+2ab+b^2. The term 2ab should be 0, but I can't figure out why. I have found other proofs here ...
6
votes
1answer
136 views

Asymptotic bias of LASSO vs. none of SCAD

I am reading a paper which says that LASSO is asymptotically biased while SCAD is not. I take asymptotic (un)biasedness to concern the slope estimators from LASSO and SCAD as the sample size goes to ...
0
votes
0answers
19 views

Where did this expression come from? (estimation theory, chi-square distribution) [duplicate]

I was trying to understand the solution of this problem; Two samples $\{x[0], x[1]\}$ are independently observed from a $N(0,\sigma^2)$ distribution. The estimator $\hat{\sigma}^2 = \frac{1}{2}(x^2[0]...
3
votes
1answer
52 views

Why is REML default if it inflates MSE?

Within the mixed effects model world, REML has become the method of choice in order to correct for the downward bias in variance components. For years, I accepted this rationale without thinking about ...
1
vote
0answers
34 views

exponential parameter estimtion from the smallest k-th order statistics

Assume $X_1, X_2, X_3,\ldots,X_n$ are i.i.d. samples from Exp($\lambda$). Assume that the integer $k<n$, is it possible to find a an unbiased estimator for $\lambda$ from the k-th smallest ordered ...
0
votes
0answers
8 views

Understanding the conceputal difference between loss(and hence risk) and variance of an estimator?

If I have an unknown distribution with $n$ sample points, and if I want to infer the unknown, then I can proceed in one of the two ways: (a) Pick an estimator. Suppose we are lucky and always find a ...
5
votes
1answer
176 views

How do these results show that $T(\mathbf{X})$ is an unbiased estimator of $E_\varphi[T(\mathbf{X})]$ that achieves the Cramer-Rao lower bound?

Let's say that $X_1, \dots, X_n$ has the joint distribution $f_\varphi(\mathbf{x})$ that belongs to the one-parameter exponential family $$f_\varphi(\mathbf{x}) = \exp{\left\{ c(\varphi) T(\mathbf{x}) ...
5
votes
1answer
55 views

Does an estimator need to be unbiased in order to be sufficient?

I am reviewing some theoretical statistics content, and I was wondering if an estimator need to be unbiased in order to be sufficient? Is there any way to prove this? Thanks!
1
vote
0answers
17 views

is the principal components regression coefficient biased?

can anyone help me calculate the bias of the principal components regression ? it's clearly different than zero, but i am strugling to find a reference that proves it. and this attached wikipedia page ...
3
votes
1answer
48 views

Why the sample covariance estimator is unbiased, but the sample pearson correlation coeficient is not?

Why the sample covariance estimator is unbiased, but the sample Pearson correlation coefficient is not? I'm a bit confused because the sample Pearson coefficent was built using the sample covariance ...
0
votes
0answers
13 views

Unbiased estimators - simultaneous equations?

I am studying for university and there is something I do not really understand, I would be very thankful if you could help me! It is about simultaneous equations, I solved a) and b), they were easy, ...
0
votes
0answers
16 views

OLS and ARCH Error Term

As for an OLS equation, If the error term has ARCH, i.e. Is the OLS estimator still unbiased?
0
votes
0answers
44 views

An unbiased efficient estimator of a function of parameter is MLE

Let a random sample taken from a common density $f(x, \theta)$ where $\theta \in \Theta \subseteq R$. Let $\theta^o$ be the unique maximum likelihood estimator (MLE) of $\theta$; note that $\theta^o$ ...
1
vote
1answer
64 views

Unbiased estimation for constrained input-output data?

I am trying to understand the following. I have a series of measured ground true data $Y = (y_1,y_2,\ldots,y_m)$ and a series of estimated data $\hat Y = (\hat y_1, \hat y_2,\ldots,\hat y_m)$. Then, ...
1
vote
1answer
52 views

How to show that $var(\hat{\mu}) < var(\bar{X}) $for a stationary process ${X_t}$, where $X_t = \mu + Z_t + Z_{t-1} $?

If ${X_t}$ is a stationary time series with mean $\mu$ then the usual estimator for $\mu$ is the sample mean $\bar{X} = \frac{X_1+...+X_n}{n}$. Assume we have $X_t = \mu + Z_t + Z_{t-1}$, where ${Z_t}...
0
votes
1answer
26 views

Equivalence of the completeness of the order statistics and the uniqueness of symmetric unbiased estimators

I am reading A.J. Lee's 1990 book "U-statistics: Theory and Practice". There is an equation on page 6 that I cannot explain why it holds, and I hope somebody could help me. Here is the ...
0
votes
1answer
28 views

Unbiasing machine learning model features, is it valid?

I've been working on a RecSys model recently (using HRNNs), and when thinking about the features used for users and itens, I thought that many of them ended up being biased by the old system ...
0
votes
1answer
79 views

Is my calculation for the MLE correct? How do I check whether it's biased?

In this question, I explored the Rayleigh distribution, with PDF $$f_{\sigma}(x) = \dfrac{x}{\sigma^2} e^{-\dfrac{x^2}{2\sigma^2}},$$ where $x \ge 0$. I calculated that the MLE is $\hat{\sigma^2} = \...
0
votes
1answer
40 views

$\frac{1}{n - 1} E \left[ \sum_{i = 1}^n (Y_i - \bar{Y})^2 \right]$ to $\frac{1}{n - 1} E \left[ \sum_{i = 1}^n Y_i^2 - n \bar{Y}\right]$?

I have that $S^2 = \dfrac{1}{n - 1} \sum_{i = 1}^n (Y_i - \bar{Y})^2$, and I am trying to show that $S^2$ is an unbiased estimator. I get the following: $$E[S^2] = E \left[ \dfrac{1}{n - 1} \sum_{i = ...
2
votes
1answer
24 views

Does multiple imputation (MI) introduce bias in estimates?

I am trying to use MI to deal with missing values in my data set. If I understand correctly, MI is about simulating multiple data sets from a given initial data set and imputing possible values ...
0
votes
1answer
42 views

How can one show that $\bar{X}$ is the best unbiased estimator for $\lambda$ without using the Cramèr-Rao lower bound?

Assume we have the random sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\sigma^2 < \infty$. We have that $E[S^2] = \sigma^2$, where $S^2 = \sum_{i = 1}^n \dfrac{(X_i - \bar{X})^2}{n - 1}$ ...
0
votes
0answers
26 views

steady-state kalman filter: properties of the temporal autocovariance of the estimates

Suppose I have an equation like $q_t = \mathbb{E}[q_t|I_t]+\epsilon_t$, where $t$ is a discrete time index, $q_t$ is a Gaussian stationary process, $\epsilon_t$ is an error term, and $I$ is an ...
0
votes
0answers
20 views

show X~$N(\mu,\sigma^2)$ is not risk-unbiased under standardized square loss function

Let X follow the $N(\mu, \sigma^2)$ distribution with parameter $\theta=(\mu,\sigma^2)$. First, I try to show X is not risk-unbiased under the standardized square loss function $𝐿(\theta,\delta)= \...
2
votes
0answers
23 views

Are these statements about the maximum likelihood estimator and efficiency correct?

I'm trying to understand efficiency and its relation with maximum likelihood estimators so I need someone to confirm or correct these statements I deduced : 1/ If the maximum likelihood estimator ...
12
votes
4answers
1k views

Why isn't this estimator unbiased?

Suppose we have a IID sample $X_1, X_2, \cdots, X_n$ with each $X_i$ distributed as $\mathcal{N}(\mu, \sigma^2)$. Now suppose we construct (a rather peculiar) estimator for the mean $\mu$: we only ...
3
votes
1answer
42 views

Plot for unbiasedness

I was reading a paper and the authors describe a plot which they used to determine whether their estimator was unbiased. This plot is described as follows (verbatim): To assess if the probabilities ...
0
votes
0answers
18 views

unbiased estimation of the variance of $p$ (proportion) of a random sample without replacement

Given a random sample without replacement of size $n$ from population of size $N$ and $p$ is the estimator of the proportion $P$. How could one show that: \begin{equation*} \frac{N-n}{N(N-1)}pq \end{...
1
vote
0answers
20 views

Standard error of estimate of $\lambda^2$

In a problem, given $n$ observations from $Poisson(\lambda)$ , I have to get an unbiased estimator of $\lambda ^2 $ and the corresponding standard error. I used the efficiency test to get the unbiased ...
0
votes
0answers
45 views

OLS biasedness in AR(1) model [duplicate]

I am trying to show why the OLS estimator in time series models is not conditionally unbiased when using a zero-mean strong AR(1) model. From what I've read so far, this can be done through a Monte ...
0
votes
1answer
31 views

What Cramer-Rao bound should I use?

I have been researching about the Cramer-Rao bound and I have found two inequalities: $$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\...
1
vote
1answer
46 views

Find covariance of estimator and derivative of the log-likelihood function

Problem: Given and estimator $\hat k$. The estimation method is unknown (so, it can be max. likelihood, method of moments or another method), however, we know that $bias(\hat k) = 0$. Let $L$ be the ...
1
vote
1answer
22 views

Variance Bias Tradeoff

Let's consider the Mean Square Error of an approximation of a parameter $\theta$ by $\hat{\theta}$. $$\mathbb{E}(\theta-\hat{\theta})^2=Var(\hat{\theta})+(Bias(\hat{\theta}))^{2}$$ Usually, we say ...
2
votes
1answer
49 views

For iid $X_1, \dots, X_n \sim N(0,\sigma^2)$, get sufficient statistic $T = \sum_{i=1}^nX_i^2$, how to find unbiased estimator of $\sigma^a$

For $X_1, \dots, X_n \sim N(0,\sigma^2)$, we define a sufficient statistic $T = \sum_{i=1}^nX_i^2$. There is a positive number $a$. My question is how to find unbiased estimator of $\sigma^a$ using ...

1
2 3 4 5
14