14
$\begingroup$

I have read some articles about partial autocorrelation of time series and I have to admit, that I do not really comprehend the difference to a normal autocorrelation. It is often stated that the partial autocorrelation between $y_t$ and $y_t-k$ is the correclation between $y_t$ and $y_t-k$ with the influence of the variables between $y_t$ and $y_t-k$ removed? I do not understand this. If we calculate the correlation between $y_t$ and $y_t-k$ then anyways the variables in between are not consindered at all if you use the correlation coefficient for doing that. The correlation coefficient considers two variables only as far as I know.

This really confuses me. I hope you can help me on that. I'd appreciate every comment and would be thankful for your help.

Update: Can anyone try to explain how one could calculate autocorrelation and partial autocorrelation for a time series. I understood how to do this with a sample but not with a time series (because you need three variables according to the example here https://en.wikipedia.org/wiki/Partial_correlation). Do you know any example where this is done?

$\endgroup$
2
  • $\begingroup$ Check this question/answer: stats.stackexchange.com/questions/472426/… $\endgroup$
    – Ale
    Aug 17, 2020 at 14:32
  • $\begingroup$ Thanks Ale for the comment and the link. Unfortunately it does not help. I still have a fundamental comprehension problem with partial autocorrelation. Further I do not understand the specific answer given in that post. $\endgroup$
    – PeterBe
    Aug 17, 2020 at 16:30

2 Answers 2

19
+50
$\begingroup$

For a while forget about time stamps. Consider three variables: $X, Y, Z$.

Let's say $Z$ has a direct influence on the variable $X$. You can think of $Z$ as some economic parameter in US which is influencing some other economic parameter $X$ of China.

Now it may be that a parameter $Y$ (some parameter in England) is also directly influenced by $Z$. But there is an independent relationship between $X$ and $Y$ as well. By independence here I mean that this relationship is independent from $Z$.

So you see when $Z$ changes, $X$ changes because of the direct relationship between $X$ and $Z$, and also because $Z$ changes $Y$ which in turn changes $X$. So $X$ changes because of two reasons.

Now read this with $Z=y_{t-h}, \ \ Y=y_{t-h+\tau}$ and $X=y_t$ (where $h>\tau$).

Autocorrelation between $X$ and $Z$ will take into account all changes in $X$ whether coming from $Z$ directly or through $Y$.

Partial autocorrelation removes the indirect impact of $Z$ on $X$ coming through $Y$.

How it is done? That is explained in the other answer given to your question.

$\endgroup$
24
  • $\begingroup$ Thanks Dayne for your answer. Now I understand the basic idea behind partial autocorrelation due to you great explanation. However, I still do not know how I can calculate the PACF? For normal autocorrelation I would just use the Pearson correlation coefficient. Is there a similar formular or (partial)correlation coefficient for the PACF? As stated before, I do not like the other answer given above and it is not useful for me (too many unexplained variables and statements) $\endgroup$
    – PeterBe
    Oct 12, 2020 at 7:57
  • $\begingroup$ Altough my question has still not been fully answered, I awarded the bounty to you because of your great explanation and at least I understand the concept behind it now. $\endgroup$
    – PeterBe
    Oct 12, 2020 at 7:58
  • $\begingroup$ Great that you liked the answer. About the calculation part: the math of it is obviously complicated and requires a lot of qualifiers, particularly related to stationarity. Let me try to give geometric interpretation in next comment. $\endgroup$
    – Dayne
    Oct 12, 2020 at 11:42
  • $\begingroup$ Consider three vectors X, Y, Z in an N dimensional space. Each observation of X form one dimension. The length of each vector represent standard deviation. The dot product is the covariance and cosine of the angle between two vectors is the correlation. Now if you remember, dot product projects one vector onto another. So dot product can be used to break a vector in two parts. One part is parallel to the vector with with dot product is taken and another part is orthogonal/perpendicular to it. So if you subtract from X, projection of X on Y, it becomes perpendicular to Y. $\endgroup$
    – Dayne
    Oct 12, 2020 at 11:53
  • 2
    $\begingroup$ @PeterBe: i think my language was a bit confusing. I meant that there is a relationship between X and Y which is independent of the relationship between Y and Z and X and Z. Hope it makes sense now. $\endgroup$
    – Dayne
    Dec 11, 2020 at 11:57
8
$\begingroup$

The difference between (sample) ACF and PACF is easy to see from the linear regression perspective.

To get the sample ACF $\hat{\gamma}_h$ at lag $h$, you fit the linear regression model $$ y_t = \alpha + \beta y_{t-h} + u_t $$ and the resulting $\hat{\beta}$ is $\hat{\gamma}_h$. Because of (weak) stationarity, the estimate $\hat{\beta}$ is the sample correlation between $y_t$ and $y_{t-h}$. (There are some trivial differences between how sample moments are computed between time series and linear regression contexts, but they are negligible when sample size is large.)

To get the sample PACF $\hat{\rho}_h$ at lag $h$, you fit the linear regression model $$ y_t = \alpha + \, ? y_{t-1} + \cdots + \, ? y_{t-h + 1} + \beta y_{t-h} + u_t $$ and the resulting $\hat{\beta}$ is $\hat{\rho}_h$. So $\hat{\rho}_h$ is the "correlation between $y_t$ and $y_{t-h}$ after controlling for the intermediate elements."

The same discussion applies verbatim to the difference between population ACF and PACF. Just replace sample regressions by population regressions. For a stationary AR(p) process, you'll find the PACF to be zero for lags $h > p$. This is not surprising. The process is specified by a linear regression. $$ y_t = \phi_0 + \phi_1 y_{t-1} + \cdots \phi_p y_{t-p} + \epsilon_t $$
If you add an regressor (say $y_{t-p-1}$) on the right-hand side that is uncorrelated with the error term $\epsilon_t$, the resulting coefficient (the PACF at lag $p+1$ in this case) would be zero.

$\endgroup$
3
  • 2
    $\begingroup$ Thanks Michael for your answer. Unfortunately there are many thing that I do not understand: 1) What do you mean by 'the resulting Beta is y^_h in the ACF formular? 2) What does '?' mean in the PACF formular 3) PACF formular: how and why do we do the "controlling for the intermediate elements"? 4) Why shall one add an regressor on the right-hand side of the last equation. 5) What is the difference between population and sample regression? I'd appreciate further comments as I am still quite confused and do not undestand the difference between autocorrelation and partial autorcorrelation $\endgroup$
    – PeterBe
    Aug 18, 2020 at 8:29
  • $\begingroup$ Would you mind answering my follow up questions because I honestly did not understand your answer. I'd really appreciate it and it would help me a lot. $\endgroup$
    – PeterBe
    Aug 19, 2020 at 8:04
  • 3
    $\begingroup$ Thanks for the answer Michael. I think the difference is not explained in a good way in your answer from a conceptual point of view for someone who has no experience with these terms. Because of this it is not useful for me. Still I appreciate your effort. $\endgroup$
    – PeterBe
    Sep 28, 2020 at 9:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.