# Estimators, sufficiency, consistency, and bias

A random variable is said to have the Pareto distribution with parameters $\alpha$ and $\beta$, $P(\alpha, \beta)$, if its cumulative distribution function is given by

$$F(x)= 1 - (\frac{\beta}{x})^{\alpha},$$ if $x \geq \beta$, and $0$ otherwise.

a) If $X_1, ... ,X_n$ are iid $P(\alpha, \beta)$ find the probability density function of min$(X_1, ... , X_n)$.

Let's rewrite the random variables as $Y_1, ... ,Y_n$, where $Y_1 < Y_2 < ... < Y_n$.

Then $F_{Y_1}(y) = P(Y_1 \leq y, ... , Y_n \leq y)$
$$= 1 - P(Y_1 \geq y, ... , Y_n \geq y)$$ $$= 1 - P(Y_1 \geq y) ... P( Y_n \geq y)$$ $$= 1 - [P(Y_1 \geq y)]^n$$

So it suffices to find $P(Y_1 \geq y)$. But $P(Y_1 \geq y) = 1 - P(Y_1 \leq y) = 1 - (1 - (\frac{\beta}{y})^{\alpha}) = (\frac{\beta}{y})^{\alpha}$

So, $f_{Y_1}(y) = (\alpha n)\beta^{\alpha n}y^{-\alpha n -1}$

b) If $X_1, ... ,X_n$ are iid $P(\alpha = 3, \beta)$, show that $\beta = min (X_1, ... ,X_n)$ is:

i. a biased estimator for $\beta$. Compute the bias of $\hat{\beta}$

$E(Y_1) = \int^{\infty}_{\beta} 3n \beta^{3n} y^{-3n} dy$ $$=lim_{u \rightarrow \infty} [\int^u_{\beta} 3n \beta^{3n} y^{-3n} dy]$$ $$=lim_{u \rightarrow \infty} [\frac{3n}{-3n+1} \beta^{3n} y^{-3n+1} ]^u_{\beta}$$ $$= \frac{-3n}{-3n+1 \beta}$$

Now we compute the bias

$E(Y_1 - \beta)$ $$= E(Y_1) - E(\beta)$$ $$= [\frac{-3n}{-3n+1 \beta}] - [\int^{\infty}_{\beta} 3n\beta^{3n+1}y^{-3n-1} dy]$$ $$= [\frac{-3n}{-3n+1 \beta}] - [\frac{3n \beta^{3n+1} y^{-3n}}{-3n}]^{\infty}_{\beta}= [\frac{-3n}{-3n+1 \beta}] +\beta$$

ii. a consistent estimator for $\beta$

This one seems confusing to me. According to the textbook, an estimator is consistent if it approaches the estimated value in probability. However, we have $\lim_{n \rightarrow \infty} P[|Y_1 - \beta| < \epsilon]$...which doesn't seem right to me, because there is no $n$ in the expression $|Y_1 - \beta|$. So I'm guessing if I'm on the wrong track...

iii. a sufficient statistic for $\beta$.

It is sufficient by Neyman's theorem, because we can write the pdf as $f_{Y_1}(y) = [3n] [\beta^{3n}y^{-3n-1}]$.

Are my answers correct? If not, can you give me a hint? Also, can you help me with part ii?

• There is a mistake in the derivation of the distribution of $Y_1$. It should somewhat be related to the distribution of the $X_i$. Can you find the error? (Please add the self-study tag) – Michael M Jan 2 '14 at 16:09
• @MichaelMayer I'm not sure if I understood what you meant. Do you mean that I should be using the variable $x$ instead of the variable $y$? Does it make any difference? After all $Y_1, ... , Y_n$ is the same as $X_1, ..., X_n$, except that the order is different...rigth? – Artus Jan 2 '14 at 16:44
• Yes, it makes a difference. Miraculously, the density of $Y_i$ looks correct at the end, though ;-). Next, the $\beta$ should appear in the numerator of the expectation of $Y_1$ to make any statistical sense, not in the denominator. – Michael M Jan 2 '14 at 17:08
• @MichaelMayer Thanks. So (just to be sure if I understood what you meant) if I did the exact same thing with the variable $x$ instead of $y$, my answer would be correct, right? So I had to say $F_{Y_1}(x)$ instead of $F_{Y_1}(y)$? – Artus Jan 2 '14 at 23:06
• You e.g. derived $F_{Y_1}(y) = 1 - P(Y_1\ge y)^n$, which should read $F_{Y_1}(y) = 1 - P(X_1\ge y)^n$. The $Y_i$ are not iid. – Michael M Jan 3 '14 at 8:49

There is a mistake in computing the expected value of the minimum order statistic. Specifically,

$$E(Y_1) = \int^{\infty}_{\beta} 3n \beta^{3n} y^{-3n} dy = [\frac{3n}{-3n+1} \beta^{3n} y^{-3n+1} ]^{\infty}_{\beta}$$

$$= 0 - \frac{3n}{-3n+1} \beta^{3n} \beta^{-3n+1} = \frac{-3n}{-3n+1} \beta$$

which can also be written

$$E(Y_1) = \frac 1{1-\frac 13 n^{-1}}\beta$$

I will provide an alternative way to prove consistency, and leave for you to deal with the usual way. A set of sufficient conditions for consistency is

$$\lim_{n\rightarrow \infty} E(Y_1) = \beta,\qquad \lim_{n\rightarrow \infty} \operatorname{Var}(Y_1) = 0$$

(they are only sufficient because the variance of an estimator may not exist). We have

$$E(Y^2_1) = \int^{\infty}_{\beta} 3n \beta^{3n} y^{-3n+1} dy = [\frac{3n}{-3n+2} \beta^{3n} y^{-3n+2} ]^{\infty}_{\beta} =- \frac{3n}{-3n+2} \beta^{3n}\beta^{-3n+2} = \frac{-3n}{-3n+2}\beta^2$$

and so

$$\operatorname{Var}(Y_1) = E(Y^2_1) - [E(Y_1)]^2 = \frac{-3n}{-3n+2}\beta^2 - \left(\frac{-3n}{-3n+1}\right)^2 \beta^2$$

$$= \left [\frac 1{1-\frac 23 n^{-1}}- \left(\frac 1{1-\frac 13 n^{-1}}\right)^2\right]\beta^2 = \left [\frac {(1/9)n^{-2}} {\left(1-\frac 23 n^{-1}\right)\cdot \left(1-\frac 13 n^{-1}\right)^2}\right]\beta^2$$

Then it is evident that the sufficient conditions for consistency are satisfied. Intuitively, as the sample size goes to infinity, it "becomes certain" that the smallest realized value will be equal to the theoretical minimum, and so the minimum order statistic stops being a random variable and it becomes a constant (hence the zero variance).

• Thanks for the alternative way. So, for the standard method, we need to show that $\lim_{n \rightarrow \infty} P(|Y_1 - \beta | < \epsilon) = 1$. $P(|Y_1 - \beta| < \epsilon) = P(\beta - \epsilon < Y_1 < \beta - \epsilon) = \int^{\beta + \epsilon}_{\beta-\epsilon} 3n \beta^{3n} y^{-3n-1} dy = [ \frac{3n \beta^{3n} y^{-3n}}{-3n} ]^{\beta + \epsilon}_{\beta - \epsilon} = [ \frac{3n \beta^{3n} y^{-3n}}{-3n} ]^{\beta + \epsilon}_{\beta}$ (since $y \geq \beta$) $=\frac{-\beta^{3n}}{(\beta + \epsilon)^{3n}} + \frac{\beta^{3n}}{\beta^{3n}} = 1$ And $\lim_{n \rightarrow \infty} 1= 1$. Is that right? – Artus Jan 2 '14 at 23:02
• The limit is taken at the last expression, which is not equal to unity, and it is due to the limit operation that we arrive at unity. – Alecos Papadopoulos Jan 2 '14 at 23:28
• Thanks a lot. I first confused the boundaries for the limit and solved for $\lim_{n \rightarrow \infty}[\int^{\infty}_{\beta} 3n \beta^{3n} y^{-3n-1} dy] = \lim_{n \rightarrow \infty}[\lim_{u \rightarrow \infty} \int^u_{\beta} 3n \beta^{3n} y^{-3n-1} dy]$...which is why I computed two limits (even though I changed the boundaries later on but forgot the limit there)...anyway, thanks again. :) – Artus Jan 3 '14 at 11:22