Questions tagged [unbiased-estimator]

Refers to an estimator of a population parameter that "hits the true value" on average. That is, a function of the observed data $\hat{\theta}$ is an unbiased estimator of a parameter $\theta$ if $E(\hat{\theta}) = \theta$. The simplest example of an unbiased estimator is the sample mean as an estimator of the population mean.

Filter by
Sorted by
Tagged with
1
vote
1answer
44 views

MAE regression gives biased regression parameters for symmetric error?

Consider a linear model, $$ y_i = \beta_0 + \beta_1x_{1i} + \beta_2x_{2i} + \epsilon_i. $$ From the Gauss-Markov theorem, I know that, under nice conditions, the $\hat{\beta}_{OLS}=(X^TX)^{-1}X^Ty$ ...
0
votes
1answer
32 views

Unbiased estimator and biased error

I'm having some trouble relating unbiased estimators and bias error. By bias error, I mean the bias error we talk about when analyzing "bias-variance tradeoffs." Is this bias error and an unbiased ...
0
votes
0answers
24 views

Error Propagation for Unbiased Means

I am reading through https://arxiv.org/pdf/1210.3781.pdf, and do not understand its derivation for propagation of errors with respect to means. According to the text, when trying to estimate a ...
0
votes
0answers
34 views

Estimator of $\log \mathbb{E}[X]$

In many fields of statistic we are faced with quantities of type $\log \mathbb{E}[X]$ where $X$ is a generic random variable. However, I never came across any good estimator for this quantity. The ...
0
votes
0answers
27 views

Does minimizing the mean of Varience of an unbiased estimator by selecting the values of ${\bf{r}}$, imply minimizing the mean of CRLB?

I can minimize the mean of variance of an unbiased estimator of a paramater $\theta$ by selecting the values of a set of parameters, ${\bf{r}}$. So i can minimeze ${\rm{E[Va}}{{\rm{r}}_{\hat \theta }}...
2
votes
1answer
25 views

Combining importance sampling with optimization - does this yield an unbiased estimate?

I'm wondering if it is OK to combine importance sampling with optimization to choose the parameters for the substitute distribution. I have a non-negative random variable $X$ on $\mathbb{R}^d$ with ...
4
votes
1answer
78 views

Unbiased Estimator of Largest Mean of Two Normal Distributions

Given samples from two normal distributions: $X_i \stackrel{iid}{\sim} \mathcal{N}(\mu_X, \sigma_X)$ for $i = 1,...,n$ $Y_i \stackrel{iid}{\sim} \mathcal{N}(\mu_Y, \sigma_Y)$ for $i = 1,...,n$ How ...
0
votes
1answer
34 views

How can I find the BUE of $\theta$ in the simple linear relationship $Y_i=\theta x_i^2+\epsilon_i$?

Let $Y_1,...,Y_n$ be described by the relationship $Y_i=\theta x_i^2+\epsilon_i$, where $x_1,...,x_n$ are fixed constants and $\epsilon_1,...,\epsilon_n$ are iid $N(0,\sigma^2)$. How can I find the ...
4
votes
1answer
64 views

How do I find the UMVUE of $\sqrt{\alpha}$ here?

new user here self-studying some mathematical statistics. I came across this problem and am stuck. Problem: Suppose that for $i = 1, ... , n$, the positive random variables $X_i$ are independent and ...
0
votes
0answers
21 views

Proof of consistency of OLS sample estimator

I am having a hard time understanding equation B1-3 below. Why does the maximum probability limit converge to variance of X? From what I understand, Var(X) = E[X^2] - E[X]^2. E[X]^2 seems to be 0 here ...
0
votes
0answers
28 views

Bootstrap estimate of the bias of a sample t-statistic

Given a collection of $n$ bootstrapped sample means as well as bootstrapped sample t-statistics, I have 2 questions: how would one go about calculating the bootstrap estimate of the bias of the ...
3
votes
0answers
44 views

The expected value of $\frac{1}{\sqrt{1-r}}$ where $r$ is Pearson correlation

I am looking to unbias the sample statistic $\frac{1}{\sqrt{1-r}}$ where $r$ is a Pearson correlation. The population is assumued binormal with equal variance $\sigma$ and with true correlation $\rho$....
0
votes
1answer
18 views

Unbiased estimator of variation of median in spatial bins using bootstrap method

Say I have a satellite that's flying through the atmosphere, over multiple orbits, sampling its density at different altitudes, at say 1 measurement per second (specific numbers are irrelevant). The ...
0
votes
0answers
23 views

Variance vs Accuracy of Estimators

My friend and I have been working on this problem (exercise 2.16 of Statistical Theory: A Concise Introduction by Felix Abramovich, Ya'acov Ritov): A large telephone company wants to estimate the ...
0
votes
0answers
34 views

Finding good estimators for a function of bernoulli parameter [duplicate]

Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ l'm interested in finding estimator of $(1-\theta)^{1 / k},$ when $k$ is a positive integer. I am considering the following ...
4
votes
2answers
223 views

Finding UMVUE for a function of a Bernoulli parameter

Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ I'm interested in finding the UMVUE of $(1-\theta)^{1/k}$, when $k$ is a positive integer. . I know $\sum X_{i}$ is a ...
0
votes
0answers
24 views

Correct for bias in biased incremental exponentially weighted variance

According to Finch, 2009 "Incremental calculation of weighted mean and variance" the biased exponential variance estimate for an additional value $x_{n}$ at iteration $n$ can be calculated in an ...
2
votes
1answer
36 views

Unbiased estimator of standard deviation

I'm reading "Properties of range-based volatility estimators" where the authors talk about using the range of a distribution ($h$ - $l$) to estimate its volatility. Specifically, they say, Daily ...
1
vote
1answer
29 views

Are all estimators biased? Is the unbiasedness only a theoretical or approximation case?

The definition of unbiased estimator says that it's expected value has no difference comparing to a true value. So can we say that all estimators are biased (even slightly)? I thought that only in ...
0
votes
0answers
7 views

How to proof the asymptotic properties of the penalized spline estimator using asymptotic notations?

Please could someone proof how the Average Mean Squared Error of penalized spline estimator is given as \begin{eqnarray} AMSE(\hat{l} )=\...
0
votes
0answers
31 views

Showing that a estimator is biased?

I am solving a exercise who asks me to show that one estimator is biased. Given the function \begin{equation} f(x|\theta) = \left( (1-\sigma) + \dfrac{\sigma}{2\sqrt{x}} \right)I_{[0,1]}(x), \sigma \...
0
votes
1answer
35 views

Why is temporal difference learning biased in reinforcement learning?

When I learn reinforcement learning from David Silver's online video, I saw "the objective of TD learning, $r_t + \gamma V(s_{t+1})$ is a biased target for learning value function. " I know the ...
0
votes
1answer
22 views

How is the sample mean an unbiased estimator of the population mean via deeplearningbook.org?

So I know that the sample mean is a unbiased estimator of the population mean. Just wondering how the author gets from 5.33 to 5.34 in the below. How do you get from $\mathbb{E}[\mu_m]$ to just $\mu$...
0
votes
0answers
18 views

Question about consistency and unbiasedness of least square estimators in linear regression

For a random variable $X$ and $Y$, the $MSE$ is defined as $MSE(b_0, b_1) = E((Y - b_0-b_1 X)^2)$ and is minimized when $b_1 = \beta_1 =\frac{Cov(X,Y)}{Var(X)}$ and $b_0 = \beta_0 = E[Y]-\beta_1 E[X]$...
0
votes
1answer
32 views

Does bias mean additional constant in any estimator? Can I say proportional estimator unbiased estimator?

I saw one question in which the sample mean was estimated as follows (I don't know why they divided by $n-1$ instead of $n$ here for estimation), $$ \widehat{\mu} = \frac{\sum_{i=1}^n x_i}{n-1} $$ ...
0
votes
0answers
25 views

Which statistic to choose - biased or unbiased?

In a book on introductory statistics, there is a question that goes like this: If two statistics are available for estimating a population characteristic, under what circumstances might you ...
1
vote
1answer
59 views

Expectation of MLE with a logarithm [duplicate]

Let $X_1,...,X_n$ be i.i.d. with common density $$f(x)=\theta x^{\theta -1}I\{x \in [0,1]\}$$ where $\theta >0$. e) Determine whether the MLE is unbiased for $\theta$. If not unbiased, could ...
-1
votes
2answers
55 views

Show that estimator $\bar{X}-1$ is unbiased estimator if $X_1, X_2, …, X_n$ are random samples from given distribution [closed]

So, the distribution for samples $X_1, X_2, ..., X_n$ is given as $$f(x|\theta) = \left \{ \begin{aligned} e^{-(x-\theta)}, \ \ \theta < x < \infty \\ 0, \text{ otherwise.} \end{aligned} \right.$...
5
votes
1answer
95 views

$X_i \sim N(\mu_i ,1)$ A discussion on unbiased concept

I am really confused with following discus. The authors ask a question and answer it. I have problems to understanding this discussion. For example: Where come from "upward bias" ? why largest $X_i$...
2
votes
1answer
103 views

How to check if an estimator is unbiased?

Given random sample $X_1, X_2, ..., X_n$ with the distribution function $$f(x|\theta) = \left \{ \begin{aligned} e^{-(x-\theta)}, \ \ \theta < x < \infty \\ 0, \text{ otherwise.} \end{aligned} \...
1
vote
1answer
19 views

Monte Carlo Gradient Estimator [closed]

How do we derive this Monte Carlo Estimator? \begin{equation} \nabla_{\phi}\mathbb{E}_{q_{\phi}(z)}[f(z)] = \mathbb{E}_{q_{\phi}(z)}[f(z) \nabla_{q_{\phi}(z)}\ln q_{\phi}(z)] \end{equation}
20
votes
4answers
2k views

Flaws in Frequentist Inference

I have problem to understanding the following example. (1) After the next day that the glitch discovered what can tell about the observation? $X_i\nsim N(\mu,1)$ or just $X_i\sim N(\mu_2,1)$. Some ...
0
votes
3answers
36 views

Meaning of “sample size” when sample unit is ambiguous

Suppose I am collecting data on how much money is processed by the 16 banks in an economy. I want to quantify how "concentrated" the flow of money is—that is, the extent to which the larger ...
1
vote
1answer
68 views

How can I find an unbiased estimator for $\frac{1-\theta}{\theta}$ to obtain this quantity's UMVUE?

Let $X_1,\cdots,X_n$ be a random sample from $f(x|\theta)=\theta(1-\theta)^x,x=0,1,\cdots; 0 < \theta <1$ is unknown. Find the UMVUE of $\frac{1-\theta}{\theta}$. My work: I know that I should ...
0
votes
0answers
52 views

Using a single sample sequence for estimates of several integrals whose integrands have disjoint support

Let $(E,\mathcal E,\lambda)$ be a measure space $f:E\to[0,\infty)$ be $\mathcal E$-measurable with $\lambda f<\infty$ $q:E\to[0,\infty)$ be $\mathcal E$-measurable with $\lambda q=1$ and $$\{q=0\}\...
4
votes
1answer
103 views

How to prove whether or not the OLS estimator $\hat{\beta_1}$ will be biased to $\beta_1$?

*I scanned through several posts on a similar topic, but only found intuitive explanations (no proof-based explanations). Let's say I have two models, the first of which represents the true data, $y ...
0
votes
0answers
116 views

Estimate $\lambda\frac{|f-\lambda f|^2}p$ without looping twice

Let $(E,\mathcal E,\lambda)$ be a measure space, $p$ be a probability density on $(E,\mathcal E,\lambda)$ and $f\in\mathcal L^2(\lambda)$. Say I want to estimate $$\int_{\{\:p\:>\:0\:\}}\frac{|f-\...
2
votes
1answer
69 views

Are GAM suitable for inference?

Are the estimators unbiased efficient and consistent? Or is GAM better for classification and prediction than non additive models? Interaction terms aren’t allowed in GAM.
1
vote
1answer
60 views

Is it necessary to simulate unbiased coin in using frequentist approach for determining if coin is unbiased?

I’m trying to determine the best way to detect if a coin is unbiased, given some desired alpha. I understand basic probability/statistical inferencing, but there’s some information out there that ...
0
votes
0answers
53 views

Rao blackwell theorem but the unbiased estimator is a function of the sufficient statistic

The Rao-Blackwell Theorem states the following: Let $T(\mathbf X)$ be a sufficient statistic for the statistical model $(S, \{f_{\theta}: \theta \in \Theta\})$ and $\hat \theta(\mathbf X)$ be and ...
0
votes
1answer
34 views

How to make a Median absolute deviation of $N(0, \sigma^2)$ an unbiased estimator of $\sigma$, asymptotically?

I am looking for a derivation of the fact that $\frac{1}{\Phi^{-1}(3/4)}$ is the multiplier needed for the Median Absolute Deviation (MAD) to be an unbiased estimator of $\sigma$ when $x_i\sim N(0, \...
1
vote
0answers
42 views

Can we find an asymptotically consistent Metropolis-Hastings estimator based on this proposal scheme?

I'm running the Metropolis-Hastings algorithm for a target distribution $\hat\mu$ (see below for the formal setup including the definition of $\hat\mu$) on a product space $I\times E'$. I'm using the ...
0
votes
1answer
56 views

Confusion on Mean Squared Error for Regression

In common statistical textbooks' linear regression topic, Mean Squared Error is often defined as $$MSE = \dfrac{(y-\hat{y})^T(y-\hat{y})}{n-p} = \dfrac{RSS}{n-p}$$ where the $y$ and $\hat{y}$ is a ...
0
votes
0answers
26 views

Unbiased estimators of bivariate gaussian means

What are the best unbiased estimators of bivariate gaussian means given covariance matrix? Is there any such estimator that makes explicit use of the covariance matrix, and which is superior by any ...
1
vote
0answers
70 views

Is this an unbiased estimator for $\theta$?

Let $\hat \theta_1(X)$ and $ \hat \theta_2(X)$ be two unbiased estimators of $\theta$. Prove that for any $ a \in \Bbb R$, $$\hat \theta_3(X) := a \hat \theta_1(X) +(1-a) \hat \theta_2(X)$$ is an ...
1
vote
0answers
142 views

Generalization of Bessel's correction to higher order models? [duplicate]

Multiplying sample variance (i.e. variance from sample mean) by $\frac{n}{n-1}$ to obtain an unbiased estimate of the population variance (i.e. variance from population mean) is called Bessel's ...
0
votes
0answers
17 views

Show estimator is unbiased [duplicate]

Consider the estimator of the variance given by the formula: $(S')^2 = \frac{1}{n} \sum_{i=1}^{n}(Y_i − µ)^2$ Is this a biased or unbiased estimator? I'm not sure if it is possible to prove ...
16
votes
4answers
1k views

In regression, why not use regularization by default?

I remember reading somewhere in another post about the different viewpoints between people from statistics and from machine learning or neural networks, where one user was mentioning this idea as an ...
0
votes
0answers
35 views

Why do some people say that an asymptotically unbiased estimator “satisfies a strong law of large numbers”?

If $x\in\mathbb R$, an estimator for $x$ is an integrable random variable $X$. We say that $X$ is unbiased if $\operatorname{Bias}(x,X):=x-\operatorname E[X]=0$. Now, in the context of Markov chain ...
3
votes
1answer
204 views

Simulation to estimate the standard deviation of a normal distribution

I am trying to estimate the mean and the standard deviation by simulation from a normal distribution. I have the following code in R: ...

1
2 3 4 5
11