0
$\begingroup$

I am reconciling two time series, $X_{1...n}$ and $Y_{1...n}$. They are supposed to have close to 100% correlation, but in reality often not. When that happens, I want to quickly find the points that make the biggest impact to the correlation. Does there exist a formula for me to find the points with the biggest impact?

edit: specifically I want to find a fixed list of points to remove, that will give me closest to 100% correlation.

$\endgroup$
6
  • 2
    $\begingroup$ It's conceptually easy to find a single point of greatest impact: find a way to measure the change in correlation and then systematically remove the times one by one and compute that change. Pick the time where it is greatest. But finding multiple points can mean two different things: (1) do you want to find the point of greatest impact, remove it from the data, and repeat; or (2) do you want to find the subset (of a given number of points) which, when removed, changes the correlation the most? The two answers are not necessarily the same. What is your objective? $\endgroup$
    – whuber
    Sep 24, 2021 at 17:34
  • $\begingroup$ @whuber (2) better describes my objective. find a subset (probably a handful), after removed, changes the correlation the most $\endgroup$
    – Matt Frank
    Sep 24, 2021 at 17:41
  • 1
    $\begingroup$ How will you determine the size of the subset and how do you wish to measure differences in correlation? (It would help to understand better why you want to do this, for then we could propose reasonable answers to these subquestions.) $\endgroup$
    – whuber
    Sep 24, 2021 at 19:40
  • $\begingroup$ If you have three non-colinear points, then each of them them forces the correlation away from $\pm 1$ so in many cases move the correlation by the same amount $\endgroup$
    – Henry
    Sep 24, 2021 at 20:52
  • $\begingroup$ @whuber it's just a practical way to see the "erroneous points" for two supposedly very correlated time series. So I can just set an arbitrary fixed number of points to inspect. $\endgroup$
    – Matt Frank
    Sep 25, 2021 at 0:52

1 Answer 1

3
$\begingroup$

Cook' s distance calculates the effect on the regression slope of removing a single point from the data. Each point has a value, bigger values have more effect.

A correlation coefficient is the same as the regression coefficient of the standardised variables regressed against each other. Thinking of it this way leads to many other possible residuals and measures of influence of single data points.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.