Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

Does multiple imputation (MI) introduce bias in estimates?

I am trying to use MI to deal with missing values in my data set. If I understand correctly, MI is about simulating multiple data sets from a given initial data set and imputing possible values ...
0
votes
1answer
17 views

How can one show that $\bar{X}$ is the best unbiased estimator for $\lambda$ without using the Cramèr-Rao lower bound?

Assume we have the random sample $X_1, \dots, X_n$ with mean $\mu$ and variance $\sigma^2 < \infty$. We have that $E[S^2] = \sigma^2$, where $S^2 = \sum_{i = 1}^n \dfrac{(X_i - \bar{X})^2}{n - 1}$ ...
0
votes
0answers
13 views

steady-state kalman filter: properties of the temporal autocovariance of the estimates

Suppose I have an equation like $q_t = \mathbb{E}[q_t|I_t]+\epsilon_t$, where $t$ is a discrete time index, $q_t$ is a Gaussian stationary process, $\epsilon_t$ is an error term, and $I$ is an ...
0
votes
0answers
19 views

show X~$N(\mu,\sigma^2)$ is not risk-unbiased under standardized square loss function

Let X follow the $N(\mu, \sigma^2)$ distribution with parameter $\theta=(\mu,\sigma^2)$. First, I try to show X is not risk-unbiased under the standardized square loss function $𝐿(\theta,\delta)= \...
2
votes
0answers
18 views

Are these statements about the maximum likelihood estimator and efficiency correct?

I'm trying to understand efficiency and its relation with maximum likelihood estimators so I need someone to confirm or correct these statements I deduced : 1/ If the maximum likelihood estimator ...
11
votes
4answers
1k views

Why isn't this estimator unbiased?

Suppose we have a IID sample $X_1, X_2, \cdots, X_n$ with each $X_i$ distributed as $\mathcal{N}(\mu, \sigma^2)$. Now suppose we construct (a rather peculiar) estimator for the mean $\mu$: we only ...
2
votes
0answers
23 views

Efficiency and consistency of an estimator

I recently encountered the following question, which I am struggling with for two days: Let $X_1, \ldots, X_n$ be an i.i.d. sample from a $\text{Geometric}(1/(1 + \theta))$ distribution with pmf $$f(...
3
votes
1answer
39 views

Plot for unbiasedness

I was reading a paper and the authors describe a plot which they used to determine whether their estimator was unbiased. This plot is described as follows (verbatim): To assess if the probabilities ...
0
votes
0answers
17 views

unbiased estimation of the variance of $p$ (proportion) of a random sample without replacement

Given a random sample without replacement of size $n$ from population of size $N$ and $p$ is the estimator of the proportion $P$. How could one show that: \begin{equation*} \frac{N-n}{N(N-1)}pq \end{...
1
vote
0answers
19 views

Standard error of estimate of $\lambda^2$

In a problem, given $n$ observations from $Poisson(\lambda)$ , I have to get an unbiased estimator of $\lambda ^2 $ and the corresponding standard error. I used the efficiency test to get the unbiased ...
0
votes
0answers
28 views

OLS biasedness in AR(1) model [duplicate]

I am trying to show why the OLS estimator in time series models is not conditionally unbiased when using a zero-mean strong AR(1) model. From what I've read so far, this can be done through a Monte ...
0
votes
1answer
24 views

What Cramer-Rao bound should I use?

I have been researching about the Cramer-Rao bound and I have found two inequalities: $$\text{Var}\left(\hat{\theta}\right)\geq\frac{1}{\text{E}\left[\left[\frac{\partial}{\partial\theta}\ln f(X;\...
1
vote
1answer
40 views

Find covariance of estimator and derivative of the log-likelihood function

Problem: Given and estimator $\hat k$. The estimation method is unknown (so, it can be max. likelihood, method of moments or another method), however, we know that $bias(\hat k) = 0$. Let $L$ be the ...
1
vote
1answer
18 views

Variance Bias Tradeoff

Let's consider the Mean Square Error of an approximation of a parameter $\theta$ by $\hat{\theta}$. $$\mathbb{E}(\theta-\hat{\theta})^2=Var(\hat{\theta})+(Bias(\hat{\theta}))^{2}$$ Usually, we say ...
2
votes
1answer
43 views

For iid $X_1, \dots, X_n \sim N(0,\sigma^2)$, get sufficient statistic $T = \sum_{i=1}^nX_i^2$, how to find unbiased estimator of $\sigma^a$

For $X_1, \dots, X_n \sim N(0,\sigma^2)$, we define a sufficient statistic $T = \sum_{i=1}^nX_i^2$. There is a positive number $a$. My question is how to find unbiased estimator of $\sigma^a$ using ...
0
votes
1answer
32 views

Finding UMVUE of function of poisson parameter

I am to estimate $\exp(-\lambda)\lambda^2/2$ from the distribution $Exp(\lambda) \sim \frac{e^{-\lambda}\lambda^x}{x!}$ I used the indicator function $W=\mathbb I_{2}(X_1)$ as an initial unbiased ...
13
votes
5answers
214 views

Different usage of the term “Bias” in stats/machine learning

I think I've seen about 4 different usages of the word "bias" in stats/ML, and all these usages seem to be non-related. I just wanted to get clarification that the usages are indeed non-...
0
votes
0answers
20 views

Is the mundlak estimator equal to the within estimator?

Mundlak has proposed to estimate the following correlated random effects model: $$ y_{i t}=\boldsymbol{x}_{i t}^{\prime} \boldsymbol{\beta}+\overline{\boldsymbol{x}}_{i}^{\prime} \gamma+\omega_{i}+u_{...
0
votes
0answers
57 views

why is the nadaraya watson estimator unbiased?

Say I have the model $Y_{i} = m(x_{i}) + \epsilon_{i}$ and $Y_{i}$ and $X_{i}$ are two mutually independent i.i.d. sequences. Then, how can I show that the Nadaraya Watson estimator is unbiased for ...
0
votes
0answers
13 views

Prove that MLE for this distribution is a biased estimator

Let X1, . . . , Xn be a random sample from a normal distribution with mean µ and variance 1. It is known that µ ∈ (0, 1] ∪ [2, 3). Prove that the MLE of µ, if it exists, is a biased estimator of µ. Ok,...
0
votes
0answers
12 views

Accuracy estimator for scan sample of categorical time series

Suppose we are tracking the behavior of some agent over time, and this agent can only exist in two states (+, -). This agent is only active for 20 timesteps, but we are only able to observe the agent ...
0
votes
0answers
28 views

Unbiased estimator and getting estimate from estimator

I got a unbiased estimator but I don't know how to interpret it and adjust it to get estimate. The original problem is to find out the unbiased estimator for $\lambda$ in Zero-truncated Poisson ...
0
votes
0answers
12 views

Estimating Total Number of Customers from Observed Total + Non Observed Transactions

Lets say there is a business has 1 million customers that are tracked through their loyalty program. The business knows that these 1 million customers purchase, on average, 3 products per year. Their ...
1
vote
1answer
20 views

In simple linear regression, is the estimator of an individual response unbiased?

I am using linear regression. $Y_i = \beta_0 + \beta_1 x_i + \varepsilon_i$ $\varepsilon_i \overset{iid}{\sim} Normal(0, \sigma^2)$ At $X = x^\ast$, let's define the mean response as $\mu^\ast = \...
0
votes
1answer
21 views

Bias of difference-in-means estimator for experiments randomized using Bernoulli trials

Under potential outcomes framework (Neyman-Rubin causal model), it is straighforward to show that difference-in-means is an unbiased estimator of average treatment effect under completely randomized ...
0
votes
0answers
15 views

“Z-value” equivalent for sample variance

For a random variable $X$ (mean $\mu$, variance $\sigma^2$, kurtosis $\kappa$), I take $n$ i.i.d. samples $X_1,\dots,X_n$ and find their mean, $\hat \mu^{(n)}$. By linearity of expectation, I know it ...
1
vote
1answer
44 views

SGD unbiased estimator: 1 example vs larger minibatch for each iteration

Studying the SGD, I found that at each iteration the SGD turns out to be an unbiased estimator of the full gradients. The number of iterations (stochastic gradient estimation) depends on the variance. ...
1
vote
1answer
30 views

If a distribution contains a non-trivial unbiased statistic of zero, then this distribution does not have a complete statistic?

If a distribution contains a non-trivial unbiased statistic of zero, then this distribution does not have a complete statistic? Here is what it means Suppose we have a family of distribution $\...
0
votes
0answers
19 views

Understanding the difference between “different iid random variables” and “different instance of same random variable”

In the derivation of unbiased sample variance, it is considered that $X_i$ are iid random variables while $X_i$ actually represents a sample from a population. So my question is that shouldn't we ...
0
votes
0answers
18 views

Using residuals as an independent variable to get unbiased estimates of the other variables

Assuming that one works in a scientific discovering frame and is just interested in discovering relations between variables, and not to forecast anything, and since an underspecified regression model ...
1
vote
0answers
25 views

estimation of covariance of function of two i.i.d. data points

Given i.i.d. data: $X_1,\dots,X_n$ living in some space $\mathcal{X}$ and drawn according to distribution $P$, and symmetric functions $f,g: \mathcal{X} \times \mathcal{X} \to \mathbb{R}$, I want to ...
2
votes
0answers
27 views

Textbook default estimator of Bernoulli variance

Why do "most" (basically all) statistics text books use $\hat{\sigma}^2=\hat{p} (1-\hat{p})$ as an estimator for the variance of a Bernoulli process which we know is biased. Should the first ...
0
votes
0answers
33 views

unbiased estimator Bernoulli variance [duplicate]

Why do "most" (basically all) statistics text books use $\hat{\sigma}^2=\hat{p} (1-\hat{p})$ as an estimator for the variance of a Bernoulli process which we know is biased. Should the ...
4
votes
1answer
96 views

An unbiased estimator for the 2 parameters of the gamma distribution?

Nor Maximum Likelihood Estimators (MLE) neither the Moments Matching Estimators (MME) for the two parameters $\alpha, \beta$ (shape and rate respectively) are unbiased. Is there a closed formula to ...
0
votes
1answer
23 views

Is Bias Affected By Dataset Size?

I am trying to understand the concept of asymptotic unbiasedness. I understand that an estimator is said to be asymptotically unbiased if, when the size of our data increases to infinity, the bias of ...
1
vote
1answer
61 views

Should one apply bias correction for the standard deviation, for small sample sizes, as a matter of course?

If one is dealing with small sample sizes, let's say $8-16$ observations per sample, and we are interested in estimates of the standard-deviation (let us also assume Gaussian statistics), is there a ...
1
vote
1answer
50 views

Bias corrrection for MLE when dealing with normally distributed small samples

When estimating the standard-deviation for samples of normally distributed data, it is sometimes necessary to account for bias in whatever estimator one chooses -- which is usually related to the ...
2
votes
1answer
122 views

General recipe for finding unbiased or consistent estimator? [closed]

I am wondering whether there is a general recipe for finding unbiased and consistent estimators of some non-random quantity. For concreteness, I will discuss only discrete probability distributions ...
2
votes
2answers
68 views

Completeness of a statistic - Open ball

I was studying the slides of the course in statistics, but there is a theorem that is not clear for me. This chapter was about finding a complete statistic, and it explains that it can be found with ...
1
vote
1answer
202 views

How do I prove that the expectation of sample variance is equal to the population variance for any distribution? [duplicate]

In other words, is $S^2$ unbiased for $\sigma^2$ for any distribution? I know how to prove this for the normal distribution, but is it possible for me to prove this generally? Thank you
1
vote
0answers
63 views

Unbiasedness of estimators of conditional expectation with discrete dependent variable

I'm trying to figure out whether the basic formula for a conditional expectation with discrete conditioning variable (let's call it $X$). The basic argument can go something like the following: a ...
2
votes
1answer
87 views

Unbiased estimator for $\lambda$

We were asked to find the UMVUE for $\lambda$ using the Lehmann-Scheffe Theorem, given a random sample from $f_X(x;\lambda)=\frac{1}{\lambda}x^{\frac{1}{\lambda}-1}, 0<x<1, \lambda > 0$. I ...
1
vote
1answer
196 views

Rao Blackwell theorem on Bernoulli distribution

I need help with the following Problem: Let $X_1,...,X_n$ be a random sample of iid random variables, $X_i\sim Ber(p), p\in (0,1)$. We want to estimate $\theta = p^2$. It is known, that $\hat{\theta}(...
1
vote
0answers
33 views

Consistency of an estimator [closed]

I have an estimator for the coefficients of the model $$ y=X\beta+\varepsilon $$ with $y_{n\times1}$, $X_{n\times p}$, $\beta_{p\times1}$, $\varepsilon_{n\times1}$. The estimator is in the form $$ \...
2
votes
1answer
40 views

Estimating mean in the presence of serial correlation

Consider the following generating equation: \begin{equation} X_{d+1} = a X_d + b + {\cal E}_d \end{equation} where $a$ and $b$ are constants with $0 <a < 1$ and $b > 0$. Further let ${\...
1
vote
1answer
44 views

Fast Evaluation of a Double Sum

Let $q$ be a probability distribution on $\mathcal{X}$, $w$ be a nonnegative function from $\mathcal{X}$ to $\mathbf{R}$ which is bounded away from $0$ and $\infty$, and $s$ be a bounded function ...
1
vote
0answers
69 views

No unbiased estimators

Let $X_1, . . . , X_n$ be a sample from the Poisson distribution with the parameter $\theta$. How to prove that there are no unbiased estimators for $\theta^{−2}$? I have no idea that why there are no ...
2
votes
1answer
53 views

Making sense of MLE

Since we were taught MLE (Maximum Likelihood Estimation), a number of questions often bothered me. Why does Maximum Likelihood Estimation work ? Why does it always produce almost accurate results ...
1
vote
0answers
90 views

Variance-bias tradeoff problem and how Bayesian and non-Bayesian approaches perform in a big data setting

When it comes to deal with the variance-bias tradeoff issue, I assume that bias is automatically induced in the Bayesian approach just from using a prior, while non-Bayesian approaches use math to ...
0
votes
1answer
55 views

OLS on autoregressive models

Suppose I have a linear model with strongly correlated residuals. Suppose further that after adding one or more lags of the dependent variable, the residuals no longer appear to be autocorrelated ...

1
2 3 4 5
13