Definition of Partial Autocovariance Function The most interpretable definition of the Partial Autocovariance of a $(L^2)$ time series $\{X_t\}$ that I have seen is the following:
$$
\phi_{hh} = \mathrm{Cov} (X_{t+h}, X_t | X_{t+h - 1}, \cdots,X_{t+1})
$$
This expands to the expression:
$$
\mathbb{E}[\left(X_{t+h} - \mathbb{E}[X_{t+h}|X_{t+h - 1} , \cdots , X_{t+1}]\right)\cdot\left(X_{t} - \mathbb{E}[X_{t}|X_{t+h - 1} , \cdots , X_{t+1}]\right)|X_{t+h - 1}, \cdots,X_{t+1}]
$$
Now, in the textbook of Brockwell and Davis, the term(s) of the form:
$$
\mathbb{E}[X_{t+h}|X_{t+h - 1} , \cdots , X_{t+1}]
$$
are replaced with:
$$
\mathrm{Proj}_{h}(X_{t+h})
$$
which signifies the linear projection of the random variable $X_{t+h}$ onto the span of the $X_{t+h-1} , \cdots X_{t+1}$ - that is to say, the estimate given by regressing the shifted variate on the past (or future) observations. My question is - why is this formulation equivalent? The conditional expectation could be some horribly nonlinear function of the variables $X_{t+h-1} , \cdots X_{t-1}$, and is formally the projection onto the subspace of random variables that are measurable with respect to $X_{t+h -1}, \cdots , X_{t+1}$ (that is to say, any variate $Y$ such that $Y = f(X_{t+h-1}, \cdots , X_{t+1})$ i.e, the space of all deterministic functions of $(X_{t+h-1}, \cdots , X_{t+1})$. I understand that in real life this conditional expectation is impossible to calculate unless we say, assume it is linear, but is adding this to definition correct? Does it only apply for models we assume are linear (ARMA)?
 A: 
...The most interpretable definition of the Partial Autocovariance of
a $(L^2)$ time series $\{X_t\}$ that I have seen is the following...

"Interpretability" is somewhat subjective (not necessarily disagreeing that the conditional variance expression is empirically more interpretable).
However, in the $L^2$ covariance-stationary setting, clearly the natural definition is the one defined by a population linear regression, not the conditional covariance. $L^2$ is a Hilbert space. Autocovariances and partial autocovariances are defined naturally via Hilbert space projections.
The sample ACF and PACF are defined analogously, by corresponding sample regressions.
Indeed, without additional assumptions besides covariance-stationarity, your conditional covariance expression is not even deterministic. It's a random variable; in particular, it's not a parameter that can be estimated. (In the special Gaussian case, it would reduce to the standard PACF, defined via linear regressions.)

Does it only apply for models we assume are linear (ARMA)?

No, it's well-defined for any covariance-stationary time series. (It's the conditional covariance expression that is problematic as a definition of PACF.)
