3
$\begingroup$

I have a set of points $(x_i,y_i)$ where each x & y value is circular & can take on a value from -pi to pi. (The topology of the data is a torus, but I am not sure how relevant that is to the question). An example dataset is shown below:

dataset

My goal is to see if there is a significant relationship between x & y. My issue is that the data is sparse, any relationship would be nonlinear (my hypothesis is that it would show some kind of peak), and the data is non evenly sampled over x.

My first approach has been to bin the x data into segments, calculate the mean y value in each segment, and then run a test of uniformity on whether the average y-values are uniform or not, as can be seen below:

1st approach

However, my issue is that due to uneven sampling, some of those average y-values have a very low variance, while others have a high variance, thus leading to noise. The uniformity tests I am using (KS & Kuiper) will be deceived by that & may give artificially small P-values.

Any ideas on how to do this better?

Thanks Roman

$\endgroup$
6
  • $\begingroup$ Your title ("Uniformity test on data with unequal variance") does not really match what you are trying to do ("My goal is to see if there is a significant relationship between x & y", with circular x and y data). I suggest you edit it. Interesting question, btw. There will likely be some trigonometry involved. $\endgroup$ Jul 14 '16 at 7:12
  • $\begingroup$ @StephanKolassa: do you have any suggestions for a better title? thnx! $\endgroup$ Jul 14 '16 at 7:48
  • 1
    $\begingroup$ I just edited the title. If you find a better one, please edit it further. $\endgroup$ Jul 14 '16 at 7:56
  • $\begingroup$ can you assume that the population means of $x$ and $y$ are zero? Or could they be any value in $[0,2\pi]$? $\endgroup$
    – DeltaIV
    Jul 15 '16 at 8:33
  • $\begingroup$ Well since its circular data, the notion of a population mean being 0 loses values since zero is just like any # around a circle. But no i dont think the means would be zero $\endgroup$ Jul 15 '16 at 14:15
1
$\begingroup$

Ok, this is not a complete answer, but formulas were too long for comments, and, since I think your question is very interesting, I'm posting my thoughts anyway, in the hope that someone will take it from here and build a better answer.

It's clear that computing the classic correlation coefficient just won't work, because $-0.99\pi$ and $0.99\pi$ are very different values if seen as linear data, when they actually correspond to very similar angles. I propose to compute the versors $\mathbf{v}_{x_i}=(\sin(x_i),\cos(x_i))$ and $\mathbf{v}_{y_i}=(\sin(y_i),\cos(y_i))$ and their sample means $$\overline{\mathbf{v}_x}=\frac{1}{N}\sum\limits_{i=1}^N(\sin(x_i),\cos(x_i)), \quad\overline{\mathbf{v}_y}=\frac{1}{N}\sum\limits_{i=1}^N(\sin(y_i),\cos(y_i))$$

Then compute the average of the dot product of the centered versors, divided by the product of their norms:

$$\overline{d}=\frac{1}{N}\sum\limits_{i=1}^N\frac{\left(\mathbf{v}_{x_i}-\overline{\mathbf{v}_x}\right)\cdot\left(\mathbf{v}_{y_i}-\overline{\mathbf{v}_y}\right)}{\lVert\left(\mathbf{v}_{x_i}-\overline{\mathbf{v}_x}\right)\rVert \lVert \left(\mathbf{v}_{y_i}-\overline{\mathbf{v}_y}\right) \rVert }$$

A centered versor is actually not a real versor (its norm is not necessarily 1). For this reason, we divide the dot product among two centered versors by the product of the norms of the centered versors. This normalized dot product is guaranteed to be in $[-1,1]$, which implies that also $\overline{d}\in[-1,1]$. If two centered versors are parallel, then the corresponding normalized dot product will be $\pm1$. If they are orthogonal, it will be 0. Thus $\overline{d}$ seems to me an interesting index of linear correlation among $x$ and $y$. I don't know how to build a test from here, but you could try with simulation: generate random $x$ and $y$ with no correlation, compute $\overline{d}$ and see which percentage of cases falls in $[-a,a]$ with $|a|<1$.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.