5
$\begingroup$

Francis Diebold has a blog post "Causality and T-Consistency vs. Correlation and P-Consistency" where he presents the notion of P-consistency, or presistency:

Consider a standard linear regression setting with $K$ regressors and sample size $N$. We will say that an estimator $\hat\beta$ is consistent for a treatment effect ("T-consistent") if $$ \text{plim} \ \hat\beta^k = \frac{\partial E(y|x)}{\partial x_k}, $$ $\forall k=1,\dots,K$; that is, if $$ \left( \hat\beta_k−\frac{\partial E(y|x)}{\partial x_k} \right) \xrightarrow{p} 0, $$ $\forall k=1,\dots,K$. Hence in large samples $\hat\beta_k$ provides a good estimate of the effect on $y$ of a one-unit "treatment" performed on $x_k$. T-consistency is the standard econometric notion of consistency. Unfortunately, however, OLS is of course T-consistent only under highly-stringent assumptions. Assessing and establishing credibility of those assumptions in any given application is what makes significant parts of econometrics so tricky.


Now consider a different notion of consistency. Assuming quadratic loss, the predictive risk of a parameter configuration $\beta$ is $$ R(\beta)=E(y−x′\beta)^2. $$ Let $B$ be a set of $\beta$'s and let $\beta^∗\in B$ minimize $R(\beta)$. We will say that $\hat\beta$ is consistent for a predictive effect ("P-consistent") if $$ \text{plim} \ R(\hat\beta)=R(\beta^∗); $$ that is, if $$ (R(\hat\beta)−R(\beta^∗)) \xrightarrow{p} 0. $$ Hence in large samples $\hat\beta$ provides a good way to predict $y$ for any hypothetical $x$: simply use $x′\hat\beta$. Crucially, OLS is essentially always P-consistent; we require almost no assumptions.

<...>

The bottom line: In sharp contrast to T-consistency, P-consistency comes almost for free, yet it's the invaluable foundation on which all of (non-causal) predictive modeling builds. Would that such wonderful low-hanging fruit were more widely available!

Questions:

  1. What are the conditions under which P-consistency holds?
  2. Simple counterexample(s) where P-consistency does not hold
  3. Does presence of T-consistency imply presence of P-consistency?
$\endgroup$
  • $\begingroup$ Note to self: Hayashi "Econometrics" section 2.9 "Least squares projection" could be relevant. Specifically: That is, under Assumption 2.2 (ergodic stationarity) and Assumption 2.4 guaranteeing the nonsingularity of $\mathbb{E}(XX^\top)$, the OLS estimator is always consistent for the projection coefficient vector, the $\beta^*$ that satisfies the orthogonality condition (2.9.5). $\endgroup$ – Richard Hardy Feb 9 '19 at 17:57
2
$\begingroup$

This post by Francis is conflating causal concepts with statistical (regression) concepts.

First, the definition of a treatment effect is incorrect: a causal effect is defined in terms of a causal model, not in terms of the observed joint probability distribution.

The quantity $\frac{\partial E[y|x]}{\partial x_k}$ is not a treatment effect, it is the marginal change in the expected value of $y$ when we observe a change in $x_{k}$. For any linear conditional expectation function, this quantity is always the population regression coefficient of $x_k$.

A treatment effect estimand would make reference to interventions or counterfactuals,such as $\frac{\partial E[y_{x}]}{\partial x_k}$ in counterfactual notation, or $\frac{\partial E[y|do(x)]}{\partial x_k}$ in $do()$ notation.

Thus, his statement, “the distinction between P-consistency and T-consistency is clearly linked to the distinction between correlation and causality” is total nonsense.

Now to more precisely answer your questions:

1) By definition, OLS is the solution to minimizing $R(\beta)$, so it always holds under the usual standard assumptions that you have i.i.d samples from a stationary (well behaved) process.

2) Your RW example is not really valid, because a random walk is not stationary. I’m saying this because otherwise a counterexample could be just: “Imagine you have n observations, each coming from different arbitrary distributions.” In the RW case, you could estimate a specific time point regression coefficient consistently with OLS if you had infinite samples from that time point. Thus, to make this question meaningful, you would need to properly define the constraints, such as in Wasserman’s post, where he confined the coefficients to a l1 ball, and let the dimension of the ball grow with the sample size. And note this has nothing to do with causality.

3) As explained, in the usual set-up P-consistency always holds regardless of whether $E[y|x]$ is linear, and thus whether the OLS coefficients corresponds to $\frac{\partial E[y|x]}{\partial x_k}$ (which is not the definition a "treatment effect", as Francis incorrectly states).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you for the very relevant an interesting points! These are very helpful for my understanding. At the same time, your answers to the specific questions are not quite sufficient / not concrete enough. In (1), is i.i.d samples from a stationary process the condition? Could you be more specific? Usually, I encounter either i.i.d. or stationary, not both (though i.i.d. implies stationarity). In (2), the example seems to be valid precisely because it is a counterexample which I am asking for. It specifically violates the stationarity assumption. Do you think this does not make sense? $\endgroup$ – Richard Hardy Nov 6 '18 at 6:11
  • $\begingroup$ In (3), could you elaborate a bit more? Could you specify the conditions under which T-consistency implies P-consistency? Thanks again! $\endgroup$ – Richard Hardy Nov 6 '18 at 6:12
  • $\begingroup$ @RichardHardy thanks for pinging, I had forgotten about it! I have a deadline coming up, but after will come back to complement the answer. $\endgroup$ – Carlos Cinelli Jan 16 '19 at 7:31
  • $\begingroup$ @RichardHardy back here! So, regarding (i) my main point is that by definition the population regression coefficients are always identifiable. You do not need i.i.d necessarily, the most well known case is when you have heteroskedastic data, or some forms of autocorrelation, in which the data is not iid, yet consistency still holds. I believe there is a paper spelling out necessary conditions for consistency under a general set-up, I will try to find it. But my main point is that since OLS is the minimizer of $R(\beta)$ in the population, consistency generally holds barring pathological cases. $\endgroup$ – Carlos Cinelli Feb 4 '19 at 23:40
  • $\begingroup$ @RichardHardy Regarding (iii) if the conditional expectation is linear, the population regression coefficients are also the derivatives of the conditional expectation. Thus, in this case consistency of $\beta$ implies consistency of the derivatives of the conditional expectation, the only extra assumption is linearity of $E[Y|X]$. In the non-linear case, Francis question is underspecified, since to start we must define better what the target is (the derivative at one point X = x^*, the derivative averaged over the other X?). $\endgroup$ – Carlos Cinelli Feb 4 '19 at 23:55
1
$\begingroup$
  1. --- ? ---
  2. P-consistency will not hold when regressing two independent random walks on each other. In this setup, the OLS estimator converges to a random variable rather than the true parameter value (which is zero).
    (Then w.r.t. 1., we need some sort of stationarity or moment conditions for P-consistency.)
  3. --- ? ---
| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Besides you I don't think many people on this site have the background or knowledge required to answer such an advanced question ! Is there some form of "canonical" or base question regarding risk consistency ? Here are a set of notes which are very good from Tibshirani, in particular sparcity and linear regression stat.cmu.edu/~ryantibs/statml/lectures $\endgroup$ – Xavier Bourret Sicotte Sep 13 '18 at 6:12
  • $\begingroup$ @XavierBourretSicotte, I will take this as a compliment, thank you. However, I know quite a few people on this site who are clearly more knowledgeable and/or experienced than I am. I am sure more than one of them could answer the question if they tried hard enough. The problem might be that either they are not interested in the question or they do not have the time to try hard enough (or simply they have not seen the question – the view count is quite low). Regarding a canonical question regarding risk consistency, I do not know the answer, sorry. $\endgroup$ – Richard Hardy Sep 13 '18 at 7:22
1
$\begingroup$

What are the conditions under which P-consistency holds?

What is defined as "predictive risk" is just the mean square error of a linear prediction. "P-consistency" just means consistent estimation of the best linear predictor $x' \beta^*$, in time series language.

The OLS estimate $\hat{\beta}$ consistently estimates $\beta^*$, under very general assumptions. This is because $\hat{\beta}$ is just a sample version of $\beta^*$, and you just need the sample moments that enter into $\hat{\beta}$ to converge to population moments entering $\beta^*$. In other words, one needs LLN to hold (same for the consistency of any method of moments estimator).

The conditions needed are just weak stationarity (so that $\beta^* = \frac{Cov(x,y)}{Var(x)}$ is defined) and, e.g. mixing-type of conditions like $\alpha$-mixing with no restriction on the mixing rate and existence of enough moments (usually 4 would do it).

Therefore, "OLS always identifies the best linear prediction", in more econometric vernacular.

Simple counterexample(s) where P-consistency does not hold

There may be examples of weakly stationary processes for which mixing conditions do not hold and LLN does not hold. In such cases, the probability limit of OLS $\hat{\beta}$ would not exist and "P-consistency" does not hold.

For your spurious regression example, $\beta^*$ is not defined, as the processes are not stationary. In talking about "P-consistency", one already implicitly assumes stationarity so $\beta^*$ is defined.

Does presence of T-consistency imply presence of P-consistency?

In the context of linear models, "T-consistency" means $\hat{\beta}$ estimates the "true" $\beta$ where regressors are exogenous $E[\epsilon x] = 0$. But exogeneity just means that true $\beta$ is equal to $\beta^*$.

So, since "T-consistency" and exogeneity are empirically the same (the latter is a sufficient condition but this conflation is standard), yes would be a fair answer.

Estimating the conditional mean ("T-consistency") is a stronger requirement than estimating the linear projection (P-consistency).

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ No disagreement with previous answer by @CarlosCinelli. $\endgroup$ – Michael Jun 13 at 8:57
  • $\begingroup$ Thank you and +1. Could you spell out any concrete counterexample? I am not comfortable with mixing, unfortunately. $\endgroup$ – Richard Hardy Jun 13 at 9:17
  • $\begingroup$ @RichardHardy Will try to see if it would be possible to add one later. (Such an example does not change the empiricist's take away that OLS is "always" consistent, under stationary.) $\endgroup$ – Michael Jun 13 at 9:20
  • $\begingroup$ @RichardHardy Since you quoted an ergodicity condition in the comment, for now let me also say that mixing is something like ergodicity in that it describes a kind of "asymptotic independence", but ergodicity is a notion restricted to strictly stationary series. So you could replace mixing with the ergodicity condition you quote, which gives the ergodic LLN. $\endgroup$ – Michael Jun 13 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.