2
$\begingroup$

What is the proper way to report the goodness-of-fit for circular data estimates? Is there an equivalent to $R^2$ that can be used. The toolbox I have can calculate the correlation between two sets of circular variables, but I don't believe that $R^2$ is simply the square of that value. If I have a target variable, x, and $\hat{x}$ is it appropriate to calculate it in a way similar to linear variables. $R^2$ = $1-mse(x,\hat{x})/var(x)$.

EDIT: Just to provide a little bit more context, I have series of angles, $\theta$ that I'm trying to predict, and a set of inputs, $Y$ that are used to make that prediction. I have built a (neural network) model to make predictions, $\hat\theta = f(Y)$, and am trying to evaluate how good the model is in making those predictions.

$\endgroup$
  • $\begingroup$ can you clarify what you mean by circular. Is it in polar coordinates? $\endgroup$ – EngrStudent Feb 22 '16 at 13:19
  • $\begingroup$ By circular I mean that the variables are like angles or time-of-day. The key feature is that the difference between these variables is not x-y (e.g. difference 350 degrees and 10 degrees, is not 350-10=340, but rather 20 degrees). $\endgroup$ – gwk Feb 24 '16 at 2:34
1
$\begingroup$

As in non-directional statistics, seeking to summarize goodness of fit by scalar statistics or some significance test can be helpful, but more can be done. Competent statistical practitioners do not stop at $R^2$ for a regression-like model; they always want at the very least to look at the residuals too.

In general, suppose some model gives fitted or predicted directions to compare with observed directions. On the circle, the difference between two directions is reasonably the magnitude of the shorter rotation from one to the other, given a sign according to direction of rotation when appropriate. Concretely, and using degrees as units of measurement, the difference between $350^\circ$ and $10^\circ$ is just $20^\circ$, rotating clockwise, and that between $10^\circ$ and $350^\circ$ is similarly $-20^\circ$, where the minus sign marks anti- or counterclockwise rotation.

The distribution of these differences, themselves angles or directions between $-180$ and $180^\circ$ with a clear reference level at $0^\circ$, or the equivalent in any more congenial units (e.g. hours for time-of-day problems), is a simple reflection of the adequacy of any model. If various different models are being considered, comparing the pattern of these differences (which are just residuals) is usually straightforward.

Given directions collectively $\theta, \phi$ to compare (in circular statistics, trigonometric conventions frequently trump statistical conventions, so that Greek letters are used freely for variables):

Batschelet (1981, p.242) uses the notation |$\theta, \phi|$ for the absolute value of this difference and points out that it is also arccos(cos($\theta - \phi$)).

The cosine of the difference varies between $1$ and $-1$ as the difference varies from $0$ to $180^\circ$. It therefore measures similarity of angles and its mean thus defines one kind of circular correlation. On the last detail, cf. Batschelet (1981, p.182).

Another scale on which to measure difference is thus $1 -$ cos$(\theta - \phi)$. Batschelet (1981, p.243) uses the notation $d(\theta,\phi)$. Yet another is chord length $2$ sin$[(\theta - \phi) / 2]$. The latter two scales both yield results which are $0$ when $\theta = \phi$ and $2$ when $\theta$ and $\phi$ are $180^\circ$ apart, but results coincide only at those two endpoints.

See also Mardia and Jupp (2000, e.g. p.18).

Batschelet, E. 1981. Circular statistics in biology. London: Academic Press.

Mardia, K.V. and P.E. Jupp. 2000. Directional statistics. Chichester: John Wiley.

$\endgroup$
  • $\begingroup$ I am using the difference measure as you suggest with the examples with angles, but is it appropriate to scale that by the variance of the observed directions (which in my mind is akin to what R^2 does). $\endgroup$ – gwk Feb 24 '16 at 2:37
  • $\begingroup$ I really don't see any reason to do that and several reasons why not. (What is variance of angle, any way, and what are its units?) The point is that all directions (observed, fitted, your dataset, anybody else's) can be expressed on the same scale. One of the merits, supposedly, of $R^2$ is that you can compare models for quite different responses, but that's not an issue here. $\endgroup$ – Nick Cox Feb 24 '16 at 9:49
  • $\begingroup$ Difference between angles is a rotation but we'd often expect most of the values to be fairly near zero. In those circumstances, standard measures (mean, SD, median, IQR) may reasonably be used, I would suggest. The classic argument for directions that their origin is arbitrary, which implies the necessity for special methods, does not apply to differences. $\endgroup$ – Nick Cox Feb 24 '16 at 10:15
1
$\begingroup$

There is no direct analogue of $R^2$ in the circular case, because the connection between $R^2$ as the proportion explained variance and the correlation coefficient $R$ does not exist in the circular case as in the linear case.

Tests of goodness-of-fit for circular data generally involves transforming data to circular uniform data according to the model, then testing the circular uniformity of the data.

More formally, say we have data $\boldsymbol\theta = \theta_1, \dots, \theta_n$, and our model produces distribution function $F(\theta)$. Then, $2 \pi F(\theta_1), \dots, 2 \pi F(\theta_n)$ should have a circular uniform distribution.

To test circular uniformity, also called isotropy, various methods are available. The most common ones are Watson's $U^2$, Kuiper, Rayleigh and Rao spacing tests.

On the website for Circular Statistics in R, there is an R data file that implements this method of goodness of fit for several basic situations. The book itself provides a bit more information on this method of goodness of fit, in paragraph 6.2.3, p. 103.

$\endgroup$
  • $\begingroup$ I'll have to do some further reading to understand this, but are you suggesting a transformation using some assumed cumulative distribution function (like how normal random variables can be transformed into uniformed distribution using the normal cdf), and then testing if the transformed variables are in fact uniform? If that is the case, I'm not clear what the distribution would be. In my problem, I'm not making any assumption of the data distribution (just like least-squares regression doesn't require that the data be normally distributed). $\endgroup$ – gwk Feb 22 '16 at 23:51
  • $\begingroup$ The distribution is given by your model. You state you "have built a model to make predictions, $\hat\theta = f(Y)$", which would imply there is also an associated $F(Y)$, the cdf, which I mention in my answer. Maybe you are in a non-parametric context, but this is hard to judge from your question. What is $f(Y)$ in your case? $\endgroup$ – Kees Mulder Feb 23 '16 at 10:17
  • $\begingroup$ Yes, it is a non-parametric model, effectively a neural network that takes in Y to make predictions of $\theta$. $\endgroup$ – gwk Feb 24 '16 at 2:30
  • $\begingroup$ In such case, it seems you are less interested in the fit of a distributional model, such as the von Mises distribution, and more interested in predictional accuracy. In this case, it might be useful to define distance in one of the ways outlined by Nick Cox, and do some form of mean error and maybe crossvalidation. $\endgroup$ – Kees Mulder Feb 24 '16 at 17:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.