Are there general formula I could use to simplify/shorten my derivations of the expected value of the biased sample variance? In the derivation for the expected value of the (biased) sample variance, there were some intermediate equalities shown below and I wanted to check them myself:
$$
\frac{1}{k}\sum_{j=1}^k\mathbb{E}\left\{\left[x(j) - \frac{1}{k}\sum_{i=1}^{k}x(i)\right]^2\right\} = \frac{1}{k^3}\sum_{j=1}^k\mathbb{E}\left\{\left[(k-1)x(j) - \sum_{\substack{i=1 \\ i\neq j}}x(i)\right]^2\right\} = \frac{1}{k^2}[(k-1)^2 + k - 1]\sigma_0^2,
$$
where $\sigma_0^2$ is the true variance of the i.i.d. random variable $x(j) \sim \mathcal{N}(0, \sigma_0^2)$ at every discrete time step $j$.
My attempts are shown below and I am almost convinced with them. However, it took me quite long and I feel like I am missing out on a few general/known identities that could be used to make derivations simpler.
For example, I am wondering if there are formulas for squaring sums of such forms, or if taking the expectation operator earlier can reduce some steps with known identities.
It would be great if I can get my derivations verified, and awesome if you can point out some known formula to shorten some steps.
Attempt for first equality:
For the first equality, I tried to rearrange the expression inside the expectation first as below
\begin{align}
\left[x(j) - \frac{1}{k}\sum_{i=1}^{k}x(i)\right]^2 &= [x(j)]^2 +\frac{1}{k^2}\left[\sum_{i=1}^k x(i)\right]^2 - \frac{2}{k}x(j)\cdot\sum_{i=1}^kx(i) \\
&=\frac{1}{k^2}\cdot\left\{k^2[x(j)]^2 + \left[\sum_{i=1}^k x(i)\right]^2- 2k\cdot x(j)\sum_{i=1}^kx(i) \right\},
\end{align}
So now I can factor out $\frac{1}{k^2}$ outside of the expectation and just work with the expression inside $\{\cdot\}$.
\begin{align}
k^2[x(j)]^2 + \left[\sum_{i=1}^k x(i)\right]^2- 2k\cdot x(j)\sum_{i=1}^kx(i) = k^2[x(j)]^2 +\left[\sum_{i=1}^k x(i)\right]^2 - 2k\cdot[x(j)]^2 -2k\cdot x(j)\sum_{\substack{i=1 \\ i\neq j}}^kx(i) \\
 = k^2[x(j)]^2 - 2k\cdot[x(j)]^2 + \left[[x(j)]^2 + \sum_{\substack{i=1 \\ i\neq j}}^k[x(i)]^2 + 2\sum_{\substack{i = 1 \\ i\neq{j}}}^kx(i)\cdot x(j)\right] -2k\cdot x(j)\cdot \sum_{\substack{i=1 \\ i\neq j}}^kx(i) \\
 = (k^2 - 2k + 1)\cdot[x(j)]^2 + \left[\sum_{\substack{i=1 \\ i\neq j}}^k[x(i)]^2 + 2\sum_{i\neq{j}}^kx(i)\cdot x(j) -2k\cdot x(j)\cdot \sum_{\substack{i=1 \\ i\neq j}}^kx(i)\right] \\
 = [(k-1)\cdot x(j)]^2 + \left[\sum_{\substack{i=1 \\ i\neq j}}^k[x(i)]^2 + 2\sum_{i\neq{j}}^kx(i)\cdot x(j) -2k\cdot x(j)\cdot \sum_{\substack{i=1 \\ i\neq j}}^kx(i)\right]
\end{align}
It would be great if I can make the square braket term equal to $-2(k-1)x(j)\sum_{\substack{i=1 \\ i\neq j}}^kx(i) + [\sum_{\substack{i=1 \\ i\neq j}}^kx(i)]^2$ to complete the square of the RHS and some terms like $[\sum_{\substack{i=1 \\ i\neq j}}^kx(i)]^2$and $-2kx(j)\sum_{\substack{i=1 \\ i\neq j}}^kx(i)$ are already there.
I am a bit uncertain about this but $2x(j)\sum_{\substack{i=1 \\ i\neq j}}^kx(i)$ equals to $2\sum_{i\neq{j}}^kx(i)\cdot x(j)$, therefore:
$$
k^2[x(j)]^2 + \left[\sum_{i=1}^k x(i)\right]^2- 2k\cdot x(j)\sum_{i=1}^kx(i) = \left[(k-1)\cdot x(j) - \sum_{\substack{i=1 \\ i\neq j}}^kx(i)\right]^2,
$$
arriving at the first equality.
Attempt for second equality:
$$
\frac{1}{k^3}\sum_{j=1}^k\mathbb{E}\left\{\left[(k-1)x(j) - \sum_{\substack{i=1 \\ i\neq j}}x(i)\right]^2\right\}  \\
= \frac{1}{k^3}\sum_{j=1}^k\mathbb{E}\left\{[(k-1)\cdot x(j)]^2-2(k-1)x(j)\sum_{\substack{i=1 \\ i\neq j}}^kx(i) + \left[\sum_{\substack{i=1 \\ i\neq j}}^kx(i)\right]^2\right\}\\
=\frac{1}{k^3}\sum_{j=1}^k \left[(k-1)^2 \sigma_0^2 -2(k-1)\require{cancel}\cancelto{0}{\mathbb{E}\left\{x(j) \sum_{\substack{i=1 \\ i\neq j}}^kx(i)\right\}} + \mathbb{E}\left\{ \left[\sum_{\substack{i=1 \\ i\neq j}}^kx(i)\right]^2\right\} \right] \\
=\frac{1}{k^3}\sum_{j=1}^k \left[(k-1)^2 \sigma_0^2 + \mathbb{E}\left\{ \sum_{\substack{i=1 \\ i\neq j}}^k [x(i)]^2 + 2\sum_{\substack{i=1 \\ i\neq j}}^k x(i)\cdot x(j) \right\} \right]\\
=\frac{1}{k^3}\sum_{j=1}^k \left[(k-1)^2 \sigma_0^2 + (k-1) \sigma_0^2 \right]\\
=\frac{1}{k^2} \left[(k-1)^2 \sigma_0^2 + (k-1) \sigma_0^2 \right],
$$
arriving at the second equality.
 A: Your "biased variance" is equivalent to a scaled sample variance
given by: $\frac{(k-1)}{k}S^{2}=\frac{1}{k}\sum_{i=1}^{k}\left(X_{i}-\bar{X}\right)^{2}$
So we can derive the expected value of this as follows:
\begin{eqnarray*}
E\left[\frac{1}{k}\sum_{i=1}^{k}\left(X_{i}-\bar{X}\right)^{2}\right] & = & \frac{1}{k}E\left[\sum_{i=1}^{k}\left(X_{i}^{2}-2X_{i}\bar{X}+\bar{X}^{2}\right)\right]\\
 & = & \frac{1}{k}E\left[\sum_{i=1}^{k}X_{i}^{2}-2\bar{X}\sum_{i=1}^{k}X_{i}+\sum_{i=1}^{k}\bar{X}^{2}\right]\\
 & = & \frac{1}{k}E\left[\sum_{i=1}^{k}X_{i}^{2}-2\bar{X}\sum_{i=1}^{k}X_{i}+\sum_{i=1}^{k}\bar{X}^{2}\right]\\
 & = & \frac{1}{k}E\left[\sum_{i=1}^{k}X_{i}^{2}-2\bar{X}\frac{k}{k}\sum_{i=1}^{k}X_{i}+k\bar{X}^{2}\right]\\
 & = & \frac{1}{k}E\left[\sum_{i=1}^{k}X_{i}^{2}-2k\bar{X}^{2}+k\bar{X}^{2}\right]\\
 & = & \frac{1}{k}E\left[\sum_{i=1}^{k}X_{i}^{2}-k\bar{X}^{2}\right]\\
 & = & \frac{1}{k}\left[\sum_{i=1}^{k}E\left(X_{i}^{2}\right)-kE\left(\bar{X}^{2}\right)\right]\\
 & = & \frac{1}{k}\left[kE\left(X_{i}^{2}\right)-kE\left(\bar{X}^{2}\right)\right]\\
 & = & E\left(X_{i}^{2}\right)-E\left(\bar{X}^{2}\right)\\
 & = & \sigma_{0}^{2}+\mu^{2}-\left(\frac{\sigma_{0}^{2}}{k}+\mu^{2}\right)\\
 & = & \sigma_{0}^{2}+\mu^{2}-\frac{\sigma_{0}^{2}}{k}-\mu^{2}\\
 & = & \sigma_{0}^{2}-\frac{\sigma_{0}^{2}}{k}\\
 & = & \sigma_{0}^{2}\left(1-\frac{1}{k}\right)
\end{eqnarray*}
Now, notice the second line of your first equation block is given
by:
\begin{eqnarray*}
\frac{1}{k^{2}}\left[\left(k-1\right)^{2}+k-1\right]\sigma_{0}^{2} & = & \frac{1}{k^{2}}\left[k^{2}-2k+1+k-1\right]\sigma_{0}^{2}\\
 & = & \frac{1}{k^{2}}\left[k^{2}-k\right]\sigma_{0}^{2}\\
 & = & \sigma_{0}^{2}\left(1-\frac{1}{k}\right)\\
\end{eqnarray*}
Since this matches the result above, we have shown they are equivalent,
and the expected value is given by $\sigma_{0}^{2}\left(1-\frac{1}{k}\right)$.
$\blacksquare$
Additional Comments based on Comment
You were right, that I had my signs incorrect and $E(\bar{X}^2)=\left(\frac{\sigma_{0}^{2}}{k}+\mu^{2}\right)$.
The term $\left(\frac{\sigma_{0}^{2}}{k}+\mu^{2}\right)$ comes from the identity that for any random variable $Y$, the variance can be written as $Var(Y)=E(Y^2)-E(Y)^2$.  In the case above, however, we have the random variable $\bar{X}$, so we can write $Var(\bar{X})=E(\bar{X}^2)-E(\bar{X})^2$.  It's is easily shown that $Var(\bar{X}) = \frac{\sigma^2_0}{k}$.  To see we have:
\begin{eqnarray*}
Var(\bar{X}) & = & Var\left[\frac{1}{k}\sum_{i=1}^{k}X_{i}\right]\\
 & = & \frac{1}{k^{2}}Var\left[\sum_{i=1}^{k}X_{i}\right]\\
 & = & \frac{1}{k^{2}}\sum_{i=1}^{k}Var\left(X_{i}\right)\\
 & = & \frac{1}{k^{2}}kVar\left(X_{i}\right)\begin{array}{ccc}
 &  & (\text{by independence of each $X_i$})\end{array}\\
 & = & \frac{1}{k}\sigma_{o}^{2}
\end{eqnarray*}
Response to comment about squaring a constant in a variance
You indicated that you weren't " understanding how a constant ($k$) multiplied to the argument of variance comes out squared."  Here's a way to understand that and prove it to yourself.  This uses the variance identity I described in my previous comment section above:
\begin{eqnarray*}
Var(kX) & = & E[\left(kX\right)^{2}]-E[kX]^{2}\\
 & = & E[\left(k^{2}X^{2}\right)]-E[kX]^{2}\\
 & = & k^{2}E[\left(X^{2}\right)]-[kE(X)]^{2}\\
 & = & k^{2}E[X^{2}]-k^{2}E(X)^{2}\\
 & = & k^{2}\left\{ E[X^{2}]-E(X)^{2}\right\} \\
 & = & k^{2}\left\{ \sigma_{0}^{2}\right\} \\
 & = & k^{2}\sigma_{0}^{2}
\end{eqnarray*}
