I am having a hard time to understand how one derives the jackknife bias for the variance and mean.
1) Why do we need an inflation factor of $(n-1)$ when calculating the jackknife bias of the mean?
2) How does one derive the jackknife bias for the variance?
Let $\sigma^{2}$ be the variance of your random variable, and $\hat{\sigma}_{n} = \frac{1}{n}\sum_{i=1}^{n}(X_{i}-\bar{X})$ your estimate of the variance. Then it is well known fact that the bias of this estimator is $-\sigma^{2}/n$.
How does the Jackknife works?. For the sample $\left\{ X_{1},..., X_{n}\right\}$ it calculates the variances, $\hat{\sigma}_{n}^{(-i)}$, where $(-i)$ denotes that the ith measurement has been omitted. This estimator has thus a bias of $-\sigma^{2}/(n-1)$. Then calculates its estimate of the bias as, $$ \mathbf{bias}_{jack} = (n-1)\left(\frac{1}{n}\sum_{i}\hat{\sigma}_{n}^{(-i)}-\hat{\sigma}_{n}\right) $$
If we calculate its expected value, $$ E[\mathbf{bias}_{jack}] = (n-1)\left(\frac{1}{n}\sum_{i}E[\hat{\sigma}_{n}^{(-i)}]-E[\hat{\sigma}_{n}]\right) = \frac{\sigma^{2}}{n} + O\left(\frac{1}{n^{2}}\right) $$
that is, it gives us an estimate of the actual bias up to order $O\left(\frac{1}{n^{2}}\right)$. If your estimate is unbiased, so it is also the estimate of the Jackknife.
The previous answer was almost right, except it is wrong by a minus sign. \begin{eqnarray} \theta_{(\cdot )} &=& \frac{1}{n} \sum_i \theta_{ (i)} = \frac{1 }{ n(n-1)} \sum_i \sum_{j, j \ne i} ( x_j - \bar{x}_{(i)} )^2 \\ &=& \frac{1}{ n(n-1)^3 } \sum_{ij} [ (n-1)^2 (x_j -\bar{x} )^2 + 2 (n-1) (x_j - \bar{x}) ( x_i - \bar{x} ) + ( x_i - \bar{x} )^2 ]\\ &-& \frac{n}{(n-1)^3 }\sum_i ( x_i -\bar{x} )^2 \\ &=& \frac{n(n-1)}{ (n-1)^2 } \hat{\sigma}^2 \end{eqnarray} with $\bar{x} $ being the sample mean.
Note that \begin{eqnarray} \sum_{i,j} (x_j - \bar{x}) ( x_i - \bar{x} ) &=& \frac{1}{n^2} \sum_{i,j,k,l} (x_i - x_k) (x_j - x_l ) \\ &=& \frac{1}{n^2} \sum_{j,l} (x_j - x_l) \sum_{i,k} (x_i - x_k ) = 0 . \end{eqnarray} One would get the previous wrong answer if mistakenly used $ \langle \sum_{ij} (x_i - \bar{x} )( x_j -\bar{x} ) \rangle = n \langle \sum_{i} (x_i - \bar{x} )^2 \rangle = n^2 \hat{\sigma}^2 $.
Therefore we have \begin{equation} \langle (n-1) ( \hat{\theta}_{(\cdot)} - \hat{\theta} ) \rangle = - \frac{ 1 }{ n-1 } \hat{\sigma}^2 = - \frac{ \sigma^2 }{ n } . \end{equation} This is the same as the expected bias from the sample variance estimator for $\hat{ \sigma}^2 $.
