Computing the expected value for a sample

Question

Assume that we have a dataset $D$ having $N$ instances: $X_1,\cdots,X_N$. We select $n$ items $(Y_1,\cdots,Y_n)$ using sampling without replacement.

We use $\bar{Y}$ as an estimate for $\bar{X}$, and now we want to analyze whether it is biased or not. Hence, we have to compute $E[\bar{Y}]$ and compare it with $\bar{X}$, i.e. bias = $E[\bar{Y}] - \bar{X}$.

When computing $E(\bar{Y})$, I do not know what is $p(\bar{Y})$?

$$E(\bar{Y}) = \sum_{??}^{??} \bar{Y} \cdot p(\bar{Y})$$

what are the limits of $\sum$ and the meaning of $p(\bar{Y})$ ??? Besides, how can I get $\bar{X}$ into this equation to find the expectation according to $\bar{X}$?

Answer 1

I am sure others may (actually 'will' is probably a more accurate word) give better answers, but while you wait for their response, I hope my interpretation of what you are asking may provide some small assistance.

As the expectation of the y is the theoretical mean of all the choices y1 to yn... We find this by multiplying each value y by its probability and summing them (as per your formula you have quoted).

In this case the limits for the summation are 1 to n (i.e. we sum probability times y for every possible selection of y)

For example if your dataset contained 100 values of x and 50 (i.e. probability, P=0.5) of these values were -1, 30 of these values (i.e. P=0.3) were -2 and 20 of these values (i.e. P=0.2) were -5, expected value for the first value of y we choose is -1*(0.5) + -2*(0.3) + -5*(0.2)

But we do need to then sum this similarly across all n, this is just a hypothetical example for y1

If we do not know probabilities, I would suggest this route, that the probability is derived directly from the frequency of each class divided by the representation in the population. If the sample is small and sampling is not done with replacement, then we must not forget the frequency of representation (for the previously selected case) and the distribution size both decrease by one after each sample.

I hope that this helps with part A of your question

what are the limits of ∑ and the meaning of p(Y¯)

The last bit about

Besides, how can I get X¯ into this equation to find the expectation according to X¯

troubles me slightly as if we don't know much about X it is hard to estimate probabilities of Y (except e.g. by using sampling frequencies), but hopefully someone can shed further light on this and I hope this at least helps a little with use of limits for sigma and p(Y¯).

Thank you