Questions tagged [minimum-variance]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
0 answers
65 views

Finding UMVUE for exponential sample [duplicate]

Let $X_1,...,X_n$ be a random sample of i.i.d. exponential distribution with probability density function $$f(x|\theta)=\frac{1}{\theta}exp(-\frac{x}{\theta}), \ x\geq0$$ Let $S_n=\sum_{i=1}^nX_i$ and ...
user avatar
0 votes
0 answers
29 views

UMVUE of the following parameter

Suppose I have $\{X_i : 1\le i \le m\}$ which are i.i.d random variables having Poisson distribution with parameter $\lambda$ and let $N_i = \min\{k : X_k > p \text{ and } k \ge i\}$ where $p<\...
user avatar
1 vote
1 answer
59 views

Experimental Design: Choose Data Points to Minimize Quadratic Term Variance in Multiple Regression

$\newcommand{\eps}{\varepsilon}\newcommand{\szdp}[1]{\!\left(#1\right)} \newcommand{\szdb}[1]{\!\left[#1\right]}$ Problem Statement: Suppose that you wish to fit a model $$Y=\beta_0+\beta_1x+\beta_2x^...
user avatar
0 votes
0 answers
447 views

Prove that the variance of the Generalized Least squares estimator is less than the variance of the OLS estimator

Suppose, we consider the following regression model, $$Y = X\beta + \varepsilon$$ where $\varepsilon$ ~ $N(0, \sigma^2V)$ and V is a known $n\times n$ non-singular, positive definite square matrix. ...
user avatar
  • 3
0 votes
0 answers
624 views

Proof Sample Variance is Minimum Variance Unbiased Estimator for Unknown Mean

I am trying to prove that the unbiased sample variance is a minimum variance estimator. In this problem I have a Normal distribution with unknown mean (and the variance is the parameter to estimate so ...
user avatar
3 votes
1 answer
55 views

Error in Derivation for Control Variate Variance?

I'm trying to derive the variance for a control variate estimator, but I seem to be missing a term that allows me to end up with the covariance in the final answer. Let $f(x)$ be my function and let $...
user avatar
1 vote
0 answers
47 views

How to find an minimum variance unbiased estimator for an integer parameter?

Consider multiple observations $x[n]$ for an integer parameter $A$ under White Gaussian Noise $w[n]$: $x[n]=A+w[n]; \quad$ $n=0,1,...,N−1$ with $w[n] \sim N(0,σ^2)$. Is it possible to have an minimum ...
user avatar
1 vote
1 answer
321 views

Rao-Blackwell for Minimum-Variance Unbiased Estimator

Let $X$ be an observation from a distribution with probability mass function:$f(x;\theta) = \left(\frac{\theta}{2}\right)^{|x|}(1-\theta)^{1-|x|}I_{\{-1,0,1\}}(x), \, \theta \in (0,1).$ Use Rao-...
user avatar
  • 133
0 votes
1 answer
627 views

The UMVUE of ratio of parameters for two uniform distributions,

Let $X_1,\ldots,X_m$ be i.i.d. having the uniform distribution $U(0, \theta_x)$ and $Y_1,\ldots, Y_n$ be i.i.d. having the uniform distribution $U(0, \theta_y)$. Suppose that $X_i$’s and $Y_j$’s are ...
user avatar
1 vote
0 answers
82 views

Understanding Rao-Blackwell [duplicate]

From Casella and Berger: Let $W$ be an unbiased estimator of $\tau(\theta)$ and let $T$ be a sufficient statistics for $\theta$. Define $\phi(T) = E[W|T]$. Then $E_{\theta}[ \phi(T)] = \tau(\theta)$ ...
user avatar
  • 1,250
1 vote
1 answer
565 views

Better to Minimize Absolute Error or Sum of Squared Error?

I have an Excel model which predicts the number of customers for a given month. The prediction depends on a churn rate. I have the absolute error (actual vs predicted), along with squared error and ...
user avatar
0 votes
1 answer
2k views

Showing that the minimum-variance estimator is the OLS estimator

Recap of required theory Consider the following regression: $$y_i = \alpha + \beta x_i + u_i \tag{1}$$ where $y_i$ are iid and $x_i$ are deterministic (i.e. fixed). We know that the OLS estimator $\...
user avatar
  • 183
1 vote
1 answer
468 views

How to measure how "good" or accurate a probability distribution is? Entropy, variance or what?

How can one measure the accuracy of the probability distribution of, say, a physical magnitude? I know one good candidate is the entropy, which measures the amount of information one has about the ...
user avatar
  • 23
1 vote
1 answer
876 views

How does one minimize the standard deviation to find optimal parameters? [closed]

When doing a generalized least squares fit for a line, one computes the residuals as (y - (m*x + b))**2, where (x,y) are the ...
user avatar
4 votes
2 answers
627 views

How is this minimum variance worked out for this importance sampling estimator?

I was stuck with the function 17.13 in the open source book deep learning on page 590. For short, the question is that, For the importance sampling estimator: $$\hat s_q = \frac{1}{n}\sum_{i=1, x^{i}...
user avatar
  • 5,181
0 votes
0 answers
119 views

Maximum likelihood estimator and minimum variance challenge

A random sample of size $n_1$ is to be drawn from a normal population with mean $\mu_1$ and variance $\sigma^2_1$. A second random sample of size $n_2$ is to be drawn from a normal population with ...
user avatar
3 votes
1 answer
73 views

Good parameter estimates vs good computed moment estimates

Suppose I have a distribution from a known parametric family f(x; θ). I have a sample from that distribution. From the sample, I estimate values for the parameters. Suppose I have estimators that I ...
user avatar
  • 2,587
3 votes
1 answer
827 views

Are MVUEs and MLEs always functions of a minimal sufficient statistic?

Is it the case that both minimum variance unbiased estimators (MVUEs) and maximum likelihood estimators (MLEs) are always functions of a minimal sufficient statistic? If so, how do we know? If not, ...
user avatar
2 votes
1 answer
216 views

How does one can guarantee that any unbiased estimator is MVUE due to it containing a minimal sufficient statistic?

Sufficiency is okay. But I don't really get it why the fact guarantees it has minimal variance? Can anyone explain to me somewhat intuitively?
user avatar
4 votes
1 answer
817 views

Minimum-variance unbiased linear estimator

Suppose that it is known that the mean of RV $X_i$ is $\mu_i\theta$, (i = 1, 2,..., n), where $\mu_i$ are known constants, whereas $\theta$ is unknown. Let $\Sigma$ be the variance matrix of the ...
user avatar
  • 131
1 vote
0 answers
785 views

Unbiased estimator based on minimal sufficient statistic has smaller variance than one based on sufficient statistic

Suppose that $T_1$ is sufficient and $T_2$ is minimal sufficient, U is an unbiased estimator of $\theta$, and define $U_1=\mathbb{E}(U|T_1)$ and $U_2=\mathbb{E}(U|T_2)$ a)Show that $U_2=\mathbb{E}(...
user avatar
8 votes
0 answers
1k views

Methods of Proving that a UMVUE does not exist?

Are there efficient methods of showing when a UMVUE does not exist? I can think of the trivial case when no unbiased estimators exist at all. But that's not really interesting. I feel like this ...
user avatar
3 votes
0 answers
204 views

A Proof of Tukey's Inequality

Suppose that $W_1,W_2,...,W_n$ are uncorrelated unbiased estimators of a parameter $\theta$. Consider $W=\sum_{i=1}^na_iW_i$ such that $E(W)=\theta$ and $Var(W_i)=\sigma^2_i$, where the $a_i$'s are ...
user avatar
0 votes
0 answers
2k views

Uniform minimum variance unbiased estimator [duplicate]

Let $X_1, X_2, ..., X_n$ be an iid random sample from a Poisson$(\lambda)$ distribution: a) find the UMVUE of $\theta$ = $\lambda^k$ for $k > 0$ a known integer b) find the UMVUE of $\tau = e^{-\...
user avatar
  • 107
9 votes
1 answer
1k views

Optimal importance sampling with ratio estimator

We want to approximate the following expectation: $$\mathbb{E}[h(x)] = \int h(x)\pi(x) dx$$ Where $h(x)$ is an arbitrary function and $\pi(x)$ is a distribution, also for simplicity, let's assume ...
user avatar
  • 1,188