6
$\begingroup$

I have two large datasets from nonlinear control systems. I am trying to correlate the error with the control action taken.

If the error is plotted as fx and the control action as fy we end up with this: error and control action plot

We can see that fx from t = 0 to t = 1244 is decreasing. It is hard to see in the figure, but from t = 1245 to t = 9500 the error is slightly increasing towards 0 but never actually achieving it. On the other hand, fy is always decreasing.
There are changes in every single observation from fx and fy but they are very subtle.

The correlation between fx and fy is very different from Pearson and Spearman's coefficients. Since, for the most part, fx increases and fy decreases, the latter coefficient produced a more expected value than Pearson's. Pearson: 0.998, Spearman: -0.699.
Just out of curiosity, Kendall tau is -0.788.

I decided then to do a scatter plot with the correlations. Points being (x, y) from fx and fy respectively: Correlations scatter plot. Pearson on the left, Spearman on the right

The pearson correlation is so close to being perfectly linear that it is actually causing me to question the logic used above to explain the expected negative value and go for it.

I've read this question many times before asking this one up. I decided to do so because I'm not so sure why would Pearson's coefficient produce such a value when there is so much data indicating a negative trend.

I would rather ask more than just a single question for this:
Q1: Am I doing/assuming something wrong?
Q2: Should I stick with my logic and use Spearman's or consider Pearson's instead?

Edit: I've made a histogram of both error (fx) and control action (fy) observations. The majority of occurrences is during the stage where the error is negative and increasing and the control action is positive and decreasing. The data is really skewed, so there is a really significant concentration of points in the left tail of the Pearson plot. Is that perhaps the reason why I should use Spearman?

Histogram:Histogram of distribution. Error (fx) on the left, control action (fy) on the right

Pearson's left tail where lies the highest concentration of points:Pearson left tail section concetration

Improve this question
$\endgroup$
9
  • 2
    $\begingroup$ Are your measurements linearly spaced or log spaced? This will have big implications for pearsons as it is based on magnitude of values and only a small portion of your graph contributes the biggest changes in magnitude. $\endgroup$
    – ReneBt
    Apr 8, 2018 at 7:35
  • 1
    $\begingroup$ @Alexis I will try to upload the data and link it here. $\endgroup$
    – Alex Newman
    Apr 8, 2018 at 22:02
  • 1
    $\begingroup$ @AlexNewman Are you using the simplified formula for Spearman's $r_{S} = 1 - \frac{6\sum^{n}_{i=1}{d^{2}}}{n\left(n^{2}-1\right)}$? Because that formula does not work when you have tied ranks, and your data appear to possibly have mostly tied ranks (those upper tails). In such a case you should calculate Spearman's $r_{S}$ by (a) ranking both variables (independently), and (b) calculating Pearson's $r$ on the ranks. $\endgroup$
    – Alexis
    Apr 8, 2018 at 23:11
  • 1
    $\begingroup$ @Alexis. No. I'm actually taking the Pearson's using the ranks. $\endgroup$
    – Alex Newman
    Apr 8, 2018 at 23:55
  • 1
    $\begingroup$ @Alexis the error is x and control action y data. They're just in different files. $\endgroup$
    – Alex Newman
    Apr 9, 2018 at 0:25

1 Answer 1

Reset to default
3
$\begingroup$

Your last graph—$y$ vs $x$—provides the insight:

  1. Pearson's correlation coefficient is an inappropriate measure of association here, because your data are not linearly associated, which violates a fundamental assumption of Pearson's measure.

  2. Spearman's correlation coefficient is also inappropriate, because your data are not monotonically associated: in your last graph values of $y$ tend to both increase and decrease as a function of $x$, and if you swap $x$ and $y$ axes on you graph (as below), you will see that $x$ first decreases, then increases as a function of $y$ precisely where you said the highest concentration of points are massed. In monotonic function one variable either only fails to increase, or fails to decrease as a function of the other, but you have both—increase and decrease—happening, which violates a fundamental assumption of Spearman's rank correlation coefficient.

  1. Visually, your variables are obviously very strongly associated (if "associated" means "knowing something about one variable tells you something about the other variable"). I think the solution to your problem is not to use a correlation coefficient to characterize this relationship, but:

  2. a nonparametric regression (e.g., a generalized additive model),

  3. a parametric curve-fitting regression algorithm (e.g., fractional polynomial regression), or

  4. using nonlinear least squares regression to fit a parametric approximation to the previous two methods, or even a theory-driven model.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Thank you very much for the explanation. I will look further in the leads you've given me. Both variables are indeed related, the control action drives the process which produces the error. The point of this investigation is to check whether the error is affected by the control action or by the external forces in the control process. $\endgroup$
    – Alex Newman
    Apr 9, 2018 at 3:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.