Why is there a E in the name EM algorithm? I understand where the E step happens in the algorithm (as explicated in the math section below). In my mind, the key ingenuity of the algorithm is the use of the Jensen's inequality to create a lower bound to the log likelihood. In that sense, taking the Expectation is simply done to reformulate the log likelihood to fit into Jensen's inequality (i.e. $E(f(x)) < f(E(x))$ for concave function.)
Is there a reason why the E-step is so-called? Is there any significance to the thing that we're taking expectation of (i.e. $p(x_i, z_i| \theta)$? I feel like I'm missing some intuition behind why the Expectation is so central, rather than simply being incidental to the use of Jensen's inequality.
EDIT: A tutorial says:

The name 'E-step' comes from the fact that one does not usually need
  to form the probability distribution over completions explicitly, but
  rather need only compute 'expected' sufficient statistics over these
  completions.

What does it mean "one does not usually need to form the probability distribution over completions explicitly"? What would that probability distribution look like?

Appendix: E-step in the EM algorithm
$$\begin{align}
ll &= \sum_i{\log p(x_i; \theta)} && \text{definition of log likelihood} \\
&= \sum_i \log \sum_{z_i}{p(x_i, z_i; \theta)} && \text{augment with latent variables $z$} \\
&= \sum_i \log \sum_{z_i} Q_i(z_i) \frac{p(x_i, z_i; \theta)}{Q_i(z_i)} && \text{$Q_i$ is a distribution for $z_i$} \\
&= \sum_i \log E_{z_i}[\frac{p(x_i, z_i; \theta)}{Q_i(z_i)}] && \text{taking expectations - hence the E in EM} \\
&\geq \sum E_{z_i}[\log \frac{p(x_i, z_i; \theta)}{Q_i(z_i)}] && \text{Using Jensen's rule for $\log$ which is concave} \\
&\geq \sum_i \sum_{z_i} Q_i(z_i) \log \frac{p(x_i, z_i; \theta)}{Q_i(z_i)} && \text{Q function to maximize}
\end{align}
$$
 A: Expectations are central to the EM algorithm. To start with, the likelihood associated with the data $(x_1,\ldots,x_n)$ is represented as an expectation
\begin{align*}
p(x_1,\ldots,x_n;\theta) &= \int_\mathfrak{{Z}^n} p(x_1,\ldots,x_n,\mathfrak{z}_1,\ldots,\mathfrak{z}_n;\theta)\,\text{d}\mathbf{\mathfrak{z}}\\
&=\int_\mathfrak{{Z}^n} p(x_1,\ldots,x_n|\mathfrak{z}_1,\ldots,\mathfrak{z}_n,\theta)p(\mathfrak{z}_1,\ldots,\mathfrak{z}_n;\theta)\,\text{d}\mathbf{\mathfrak{z}}\\
&=\mathbb{E}_\theta\left[ p(x_1,\ldots,x_n|\mathfrak{z}_1,\ldots,\mathfrak{z}_n,\theta)\right]
\end{align*}
where the expectation is in terms of the marginal distribution of the latent vector $(\mathfrak{z}_1,\ldots,\mathfrak{z}_n)$, which depends on $\theta$.
The intuition behind EM is also based on an expectation. Since $\log p(x_1,\ldots,x_n;\theta)$ cannot be directly optimised, while $\log p(x_1,\ldots,x_n,\mathfrak{z}_1,\ldots,\mathfrak{z}_n;\theta)$ can but depends on the unobserved $\mathfrak{z}_i$'s, the idea is to maximise instead the expected complete log-likelihood $$\mathbb{E}_\vartheta\left[ \log p(x_1,\ldots,x_n,\mathfrak{z}_1,\ldots,\mathfrak{z}_n;\theta) \big| x_1,\ldots,x_n \right]$$ except that this expectation also depends on a value of $\vartheta$, chosen as $\theta_0$, say, hence the function to maximise (in $\theta$) in the M step:
$$Q(\theta_0,\theta)=\mathbb{E}_{\theta_0}\left[ \log p(x_1,\ldots,x_n,\mathfrak{z}_1,\ldots,\mathfrak{z}_n;\theta) \big| x_1,\ldots,x_n \right]$$
Jensen's inequality only comes as a justification for the increase in the observed likelihood at each M step.
A: Just some extension regarding the edited part.

The name 'E-step' comes from the fact that one does not usually need
to form the probability distribution over completions explicitly, but
rather need only compute 'expected' sufficient statistics over these
completions.

Since the value of $z$ is not observed, we estimate a distribution $q_x(z)$ for each data point $x$ as completions of the unobserved data. The Q function is the sum of expected log likelihoods over $q_x(z)$
$$Q(\theta)=\sum_x E_{q_x}[\log p(x,z|\theta)]$$
The mentioned probability distribution over completions should refer to $p(x,z|\theta)$.
For some distributions (especially the exponential family, since the likelihood is in its log form), we only have to know the expected sufficient statistics (instead of the expected likelihood) in order to compute and maximize $Q(\theta)$.

There's a very good introduction in Chapter 19.2 of Probabilistic Graphical Models.
