Calculating goodness-of-fit for circular data?

Question

What is the proper way to report the goodness-of-fit for circular data estimates? Is there an equivalent to $R^2$ that can be used. The toolbox I have can calculate the correlation between two sets of circular variables, but I don't believe that $R^2$ is simply the square of that value. If I have a target variable, x, and $\hat{x}$ is it appropriate to calculate it in a way similar to linear variables. $R^2$ = $1-mse(x,\hat{x})/var(x)$.

EDIT: Just to provide a little bit more context, I have series of angles, $\theta$ that I'm trying to predict, and a set of inputs, $Y$ that are used to make that prediction. I have built a (neural network) model to make predictions, $\hat\theta = f(Y)$, and am trying to evaluate how good the model is in making those predictions.

Answer 1

As in non-directional statistics, seeking to summarize goodness of fit by scalar statistics or some significance test can be helpful, but more can be done. Competent statistical practitioners do not stop at $R^2$ for a regression-like model; they always want at the very least to look at the residuals too.

In general, suppose some model gives fitted or predicted directions to compare with observed directions. On the circle, the difference between two directions is reasonably the magnitude of the shorter rotation from one to the other, given a sign according to direction of rotation when appropriate. Concretely, and using degrees as units of measurement, the difference between $350^\circ$ and $10^\circ$ is just $20^\circ$, rotating clockwise, and that between $10^\circ$ and $350^\circ$ is similarly $-20^\circ$, where the minus sign marks anti- or counterclockwise rotation.

The distribution of these differences, themselves angles or directions between $-180$ and $180^\circ$ with a clear reference level at $0^\circ$, or the equivalent in any more congenial units (e.g. hours for time-of-day problems), is a simple reflection of the adequacy of any model. If various different models are being considered, comparing the pattern of these differences (which are just residuals) is usually straightforward.

Given directions collectively $\theta, \phi$ to compare (in circular statistics, trigonometric conventions frequently trump statistical conventions, so that Greek letters are used freely for variables):

Batschelet (1981, p.242) uses the notation |$\theta, \phi|$ for the absolute value of this difference and points out that it is also arccos(cos($\theta - \phi$)).

The cosine of the difference varies between $1$ and $-1$ as the difference varies from $0$ to $180^\circ$. It therefore measures similarity of angles and its mean thus defines one kind of circular correlation. On the last detail, cf. Batschelet (1981, p.182).

Another scale on which to measure difference is thus $1 -$ cos$(\theta - \phi)$. Batschelet (1981, p.243) uses the notation $d(\theta,\phi)$. Yet another is chord length $2$ sin$[(\theta - \phi) / 2]$. The latter two scales both yield results which are $0$ when $\theta = \phi$ and $2$ when $\theta$ and $\phi$ are $180^\circ$ apart, but results coincide only at those two endpoints.

See also Mardia and Jupp (2000, e.g. p.18).

Batschelet, E. 1981. Circular statistics in biology. London: Academic Press.

Mardia, K.V. and P.E. Jupp. 2000. Directional statistics. Chichester: John Wiley.

Answer 2

There is no direct analogue of $R^2$ in the circular case, because the connection between $R^2$ as the proportion explained variance and the correlation coefficient $R$ does not exist in the circular case as in the linear case.

Tests of goodness-of-fit for circular data generally involves transforming data to circular uniform data according to the model, then testing the circular uniformity of the data.

More formally, say we have data $\boldsymbol\theta = \theta_1, \dots, \theta_n$, and our model produces distribution function $F(\theta)$. Then, $2 \pi F(\theta_1), \dots, 2 \pi F(\theta_n)$ should have a circular uniform distribution.

To test circular uniformity, also called isotropy, various methods are available. The most common ones are Watson's $U^2$, Kuiper, Rayleigh and Rao spacing tests.

On the website for Circular Statistics in R, there is an R data file that implements this method of goodness of fit for several basic situations. The book itself provides a bit more information on this method of goodness of fit, in paragraph 6.2.3, p. 103.