6
$\begingroup$

I'm looking at data defined on a given feature with respect to two measures. Whilst both measures are defined on the same domain, both measures are defined on different ranges, so, with a view to displaying both curves on the same plot, they have been normalized (to sum to one across the support like a probability density function).

Two measures defined on same feature.

From the visualization, there can be observed an "alternating peaks" pattern between both measures. This is interesting because theory hypothesizes that the measure represented by the blue curve has an inhibitory or silencing effect on the measure represented by the red curve.

Scatter Plot of Curve Values

In some respects these alternating peaks can be considered to be anti-correlated with one another; however, the anti-correlation in this scatter does not look particularly strong. Similarly, a regression model doesn't look particularly appropriate in this case either as the scatter plot demonstrates. The relationship doesn't really seem to be apparent.

Is there a better way to capture and quantify the alternating peaks pattern present between the two measures in the data?

Edit:

As mentioned in the comments, the following table is sample data constructed to have similar properties to the original data over a shorter domain.

             Coordinate Measure1 (Blue Curve) Measure2 (Red Curve)
 1           1            0.01190476           0.01369863
 2           2            0.01190476           0.01369863
 3           3            0.01190476           0.01369863
 4           4            0.02380952           0.02739726
 5           5            0.15476190           0.01369863
 6           6            0.15476190           0.02739726
 7           7            0.11904762           0.05479452
 8           8            0.00000000           0.08219178
 9           9            0.00000000           0.10958904
 10         10            0.00000000           0.10958904
 11         11            0.00000000           0.10958904
 12         12            0.00000000           0.10958904
 13         13            0.00000000           0.10958904
 14         14            0.03571429           0.06849315
 15         15            0.15476190           0.04109589
 16         16            0.14285714           0.02739726
 17         17            0.15476190           0.02739726
 18         18            0.02380952           0.02739726
 19         19            0.00000000           0.01369863
 20         20            0.00000000           0.00000000
|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ The chance of observing this scenario by chance is zero. $\endgroup$ – Neil G May 12 '15 at 10:21
  • 1
    $\begingroup$ It will be useful if you post some sample data here. $\endgroup$ – rnso May 12 '15 at 12:04
  • 1
    $\begingroup$ @rnso I've made some edits to my original post. The original data is defined across 76 coordinates for each measure, so I've constructed sample data across 20 coordinates such that it has more or less the same properties. $\endgroup$ – user9171 May 12 '15 at 14:42
  • 3
    $\begingroup$ Your statement "Obviously, the curves are neither linear nor monotonic, so it seems that both Pearson and Spearman correlations are inappropriate in this case" is puzzling, because neither measure of correlation has anything at all to do with the sequences of the data, which is what the apparent linearity or monotonicity of the curves refers to. Resolving this misconception may answer your question. $\endgroup$ – whuber♦ May 12 '15 at 14:48
  • 1
    $\begingroup$ @whuber I'm probably missing the point of your comment, but it seems to me that the visual representation of the correlation is given by the scatter above. In other words, the extent to which the first point of Measure1 is correlated with the first point of Measure2, the second of Measure1 correlated with the second of Measure2, and so on. The scatter suggests weak anti-correlation. This is confirmed by the returned values of -0.36 and -0.32 for Pearson and Spearman methods respectively (sample data). This doesn't characterise the apparent strong anti-correlation present in the density plot. $\endgroup$ – user9171 May 12 '15 at 19:12
6
$\begingroup$

One curve almost looks like the derivative of the other and sometimes such pairs of curves are plotted against each other with curved connections. For instance, for plotting velocity versus acceleration to see cycles better. Here is red versus blue for your toy data:

enter image description here

Arrows and annotations are sometimes added. I don't know what these kinds of plots are properly called. I've heard "phase-plane" diagrams but that term includes a lot of other kinds of plots, too.

The data points are connected in this case. With more and noisier data, you'd probably want some kind of interpolated curve that just goes near each point.

Update: In case it doesn't go without saying, I'm not sure what you mean by "anti-correlation". I'm thinking you want to show a relationship between two curves that is not functional in the usual sense. For the chart I've shown, you can think of it as parametric plot in that each variable (blue and red) is a function of a parameter ("Coordinate" in your table).

For comparison, here's another application of this kind of diagram from a NYT graphic on gas prices.

enter image description here

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Alberto Cairo calls these types of charts connected scatterplots. $\endgroup$ – Andy W May 13 '15 at 20:28
  • $\begingroup$ Thanks @AndyW. Now that you mention it, I see that's what it's called on Jon Schwabish's Graphic Continuum poster. I guess there could be different names for the curved specialization. $\endgroup$ – xan May 13 '15 at 23:23
  • $\begingroup$ @xan Thanks for your answer. I really like the effect of illustrating the cycles. However, it's more visualization-based inference, which, while undoubtedly informative, doesn't seem to allow for quantification. Does it? $\endgroup$ – user9171 May 14 '15 at 11:01
  • $\begingroup$ Right @user9171. I was assuming a different meaning of "show" :) $\endgroup$ – xan May 14 '15 at 13:06
  • $\begingroup$ @xan That's my fault for not being clearer. Although, I really like the plot! Thanks! :) $\endgroup$ – user9171 May 14 '15 at 13:13
0
$\begingroup$

The plot of one measure against the other (@xan's answer) is a good idea, except that I don't think it makes sense to join the points in this way. It only makes sense if the order of observations is really important. My understanding is that the fact that they're anti-correlated doesn't have anything to do with their ordering.

So you should plot them one against the other, to get a cloud of points. Measure the correlation using a metric like Pearson's correlation coefficient, and you will presumably get a negative value like -0.5 or so.

Then, you can show that this is statistically significant by a randomization test:

you have the values:

blue0, blue1, ... , bluen and red0, red1, ... redn

Saying that the anti-correlation you observe is significant and is unlikely to happen by chance basically means that if instead of matching bluek to redk for all k, you match them randomly, then it is very unlikely that the resulting dataset will display this level of correlation.

So you can prove this by generating many random permutations of the red data, and calculating the correlation of the permuted red values with the original blue.

Sort the correlation values obtained, and see how extreme the true correlation is. Is it in the top 1%? 0.1%? This gives you an estimate of how unlikely it is to happen by chance.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ Thanks for your answer. Actually, initially, I had intended to implement a scheme like the one you suggested, but, unfortunately, the anti-correlation isn't really manifest in the results. Basically, the correlation amounts to approximately -0.3, so the approach doesn't really work. To see why this is, have a look at the second figure provided in my original question. $\endgroup$ – user9171 May 14 '15 at 11:06
  • $\begingroup$ @user9171 right, I had missed that plot. I would say although -0.3 is not a very high value in absolute (-0.99 would be nicer), statistically it's probably very meaningful (if you have lots of data points). But -0.3 may be enough to argue that one measure has a silencing effect over the other. So maybe you should extract the more meaningful data points (not "those that prove your preferred point", but "those that have an important interpretation in the domain"), and see if those are more correlated. $\endgroup$ – user3780968 May 15 '15 at 16:42
  • $\begingroup$ For example, when both measurements are 0, does it make sense to say they are correlated/anti-correlated? maybe it's only meaningful if one is non-zero. Also maybe the ordering is meaningful, and they tend to vary in opposite directions, so you can see if the derivatives of the measurements are anti-correlated... $\endgroup$ – user3780968 May 15 '15 at 16:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.