Simple Random sampling with replacement - Variance If just a single unit is drawn at random from the whole population and $y_1$ is the value of $y$ for this sampled unit, how can I prove that the $\mathbb{V}(y_1)= (1 − 1/N)\sigma^2$ (where $N$ is the population size and $\sigma^2$ the population variance?
To prove that the expected value of $y_1$ was $\mu$ (the population mean), I just used the definition of the expected value, as the probability of taking any y from the sample is $1/N$, the result came pretty fast. I don't see any way to do the same for the variance. 
Any tips or help toward the right idea to prove the results of the variance would be very much appreciated. Thank you in advance!
 A: I am going to chime in with an answer to this question, because it is one where there is sometimes some confusion, owing mostly to the way in which classical sampling theory operates.  Within classical sampling theory (as opposed to the super-population method), all sampling results are derived conditional on the empirical distribution of the finite population (which is equivalent to conditioning on the ordered population vector).  This conditioning is usually implicit, but in this answer I will make it explicit.  The other aspect of classical sampling theory that can be confusing is that the notation differs from the super-population method, and in particular, it is usual to define the "variance" parameter $\sigma^2$ as the population variance, without use of Bessel's correction.  This is what whuber is referring to in some of his comments on your question.

Explicit framing of your problem: The first important issue to note here is that the result you are seeking to prove only holds when the "variance" you are using is the variance of the population, with adjustment for Bessel's correction.  To avoid ambiguity in the notation, and consistent with sampling notation used in O'Neill (2014), I am going to denote this statistic as:
$$S_N^2 = \frac{1}{N-1} \sum_{i=1}^N (Y_i - \bar{Y}_N)^2.$$
(Note: This is the statistic you have denoted by $\sigma^2$.  There is some ambiguity in this latter notation, since it is used to refer to different things in different types of sampling theory.  To avoid ambiguity, my preference is to use $\sigma^2$ to refer to the variance parameter for an infinite population, and use notation like the above for a finite population.)  As stated above, the variance result you are seeking is a classical sampling result that is implicitly a conditional variance, conditional on knowing the population values (but not their order).  To frame this problem explicitly, we will let $\tilde{\boldsymbol{Y}} = (Y_{(1)},...,Y_{(N)})$ be the ordered population vector, which gives you all the population values, without telling you their order.  (Note that this vector gives the same information as the empirical distribution function $F_N$ for the population.)  The explicit form of the result you are asserting is:
$$\mathbb{V}(Y_i|\tilde{\boldsymbol{Y}}) = \frac{1}{N} \sum_{i=1}^N (Y_i - \bar{Y}_N)^2 = \Big( 1-\frac{1}{N} \Big) S_N^2.$$
This is an explicit form of the result you are trying to prove.  In the above notation I have used explicit conditioning notation, which would usually be treated as implicit within classical sampling theory.  This makes it clearer exactly what statistic you are looking at as the "variance", and it also makes the conditioning in the variance calculation clearer.  (Incidentally, you can see from the above equation why many classical theorists take the view that the population variance should not include Bessel's correction; the variance statistic without this correction is the conditional variance of a single sample value, conditional on the population values.)

Proving the result: To prove this result, let $K \sim \text{U} \{ 1,...,N \}$ denote a randomly selected index for a sampled unit, and let $I_k \equiv \mathbb{I}(K=k)$ denote the indicator variables associated with that random index.  Using these indicators, we can write the value of a randomly sampled unit as:
$$Y_K = \sum_{k=1}^N I_k Y_k.$$
It is simple to establish that $\mathbb{V}(I_k, I_{k'}) = \tfrac{1}{N} (1-\tfrac{1}{N})$ and $\mathbb{C}(I_k, I_{k'}) = -\tfrac{1}{N^2}$ for $k \neq k'$, so we have:
$$\begin{equation} \begin{aligned}
\mathbb{V}(Y_i|\tilde{\boldsymbol{Y}}) 
= \mathbb{V}(Y_K|\tilde{\boldsymbol{Y}}) 
&= \mathbb{V} \Bigg( \sum_{k=1}^N I_k Y_k \Bigg| \tilde{\boldsymbol{Y}} \Bigg) \\[6pt]
&= \sum_{k=1}^N \mathbb{V} (I_k Y_k| \tilde{\boldsymbol{Y}} ) + 
\sum_{k \neq k'} \mathbb{C} ( I_k Y_k I_{k'} Y_{k'} | \tilde{\boldsymbol{Y}} ) \\[6pt]
&= \sum_{k=1}^N Y_k^2 \cdot \mathbb{V} (I_k) + 
\sum_{k \neq k'} Y_k Y_{k'} \cdot \mathbb{C} ( I_k  I_{k'}) \\[6pt]
&= \frac{1}{N} \Big( 1-\frac{1}{N} \Big) \sum_{k=1}^N Y_k^2 - \frac{1}{N^2} 
\sum_{k \neq k'} Y_k Y_{k'} \\[6pt]
&= \frac{1}{N} \Bigg[ \Big( 1-\frac{1}{N} \Big) \sum_{k=1}^N Y_k^2 - \frac{1}{N} 
\sum_{k \neq k'} Y_k Y_{k'} \Bigg] \\[6pt]
&= \frac{1}{N} \Bigg[ \sum_{k=1}^N Y_k^2 - \frac{1}{N} 
\sum_{k =1}^N \sum_{k' =1}^N Y_k Y_{k'} \Bigg] \\[6pt]
&= \frac{1}{N} \Bigg[ \sum_{k=1}^N \Big( Y_k^2 - Y_k \frac{1}{N} 
\sum_{k' =1}^N Y_{k'} \Big) \Bigg] \\[6pt]
&= \frac{1}{N} \Bigg[ \sum_{k=1}^N Y_k^2 - N \bar{Y}_N^2 \Bigg] \\[6pt]
&= \frac{1}{N}\sum_{k=1}^N (Y_k - \bar{Y}_N)^2 \\[6pt]
&= \Big( 1 - \frac{1}{N} \Big) S_N^2. \\[6pt]
\end{aligned} \end{equation}$$
(Note that this is the same proof method as the other answer by user158565, but I will add some additional steps to fill in some blanks.)
A: When the sample size = 1, with or without replacement does not matter.
All the summation is from 1 to $N$.
For finite population, the variance is defined as:
$$\sigma^2 = \frac 1 {N-1} \sum(Y_i-\bar Y)^2$$
where $N$ is population size.
Let $Z$ be the value you get from sample with sample size 1.Then
$$Z = \sum Z_iY_i$$ where $Z_i$ is the random variable, = 1 if $Y_i$ is sampled, and =0 if not selected.
When sample size = 1, we have,
$$Var(Z_i) = \frac 1N \left(\frac{N-1}N\right)$$
$$Cov(Z_i,Z_j) = E(Z_iZ_j)-E(Z_i)E(Z_j) = 0 - \frac 1N \frac 1N = -\frac 1{N^2}$$
We have
$$Var(Z) = \sum Var(Z_i) Y_i^2 + \sum_{i\ne j}Cov(Z_i,Z_j) Y_iY_j$$
$$= \frac {N-1}{N^2}\sum Y_i^2 -\frac1{N^2}\sum_{i\ne j} Y_iY_j$$
$$=\left(1-\frac 1N\right)\left( \frac 1 {N-1} \sum (Y_i-\bar Y)^2\right)$$
$$ = \left(1-\frac 1N\right) \sigma^2$$
