Loss function (and encoding?) for angles

Question

I'm training a network to predict the angle of arrival of a signal. Labels are single values in the [-180, 180) interval.

I'm seeing a discontinuity in predictions around ±180 degrees, which makes sense as losses around that gap are incorrectly calculated by root mean square error.

I'm looking for a loss function that works in a modular way. A difference between 175 and -175 degrees should be calculated as 10 (instead of 350), if such thing exists.

It's my understanding that such a function introduces a discontinuity and thus may not be a valid approach. I'm looking for some guidance about how to deal with these kind of circular variables like angles, hour of the day, day of the week...

This has been addressed in the question "Encoding Angle Data for Neural Network", and I feel preserving linearity in angle variables is important (my input is also several angles), and I'm not getting good results with the sin/cos encoding approach proposed in that question. The problem is also discussed here: What is a correct loss for a model predicting angles from images?.

Here is what I'm doing currently, which works quite well with angles (-180, 180).

def metric_stddev_diff(y_true, y_pred):
    return tf.keras.backend.std(y_true - y_pred)

def model_create():

    model = tf.keras.Sequential([
        tf.keras.layers.Dense(128, activation='sigmoid', dtype='float64'),
        tf.keras.layers.Dense(64, activation='linear', dtype='float64'),
        tf.keras.layers.Dense(1, activation='linear', dtype='float64'),
    ])

    model.compile(optimizer='adam',  # 'rmsprop'  'adam',
                  loss='mean_absolute_error',  # 'mean_absolute_error'  'mean_squared_error'  'sparse_categorical_crossentropy'
                  metrics=['mean_absolute_error', metric_stddev_diff])

    return model

Answer 1

Almost any loss function that is symmetric and differentiable at $0$ is locally quadratic. Thus, you don't have to be too fussy when searching for a good loss function when you need symmetry and differentiability.

Notice that with nearby angles $\phi$ and $\theta,$ the Taylor series expansion of the cosine gives

$$\mathcal{L}(\phi,\theta)=2(1 - \cos(\phi-\theta)) = (\phi-\theta)^2 + O((\phi-\theta)^4)$$

is locally quadratic at $\phi-\theta=0$ (and all integral multiples of $2\pi$) through third order. Moreover, this function of $\phi$ and $\theta$ isn't badly behaved: it's defined for all angles, is differentiable everywhere, and--most importantly--respects the modular nature of angle comparison. Thus $\mathcal{L}$ is a natural and simple angular version of a quadratic loss. This would be a good place to start your analysis.

If you need more flexibility, consider defining your loss as a function of $\sqrt{2(1-\cos(\phi-\theta))}:$ clearly this is a circular analog of the absolute difference.