I'm training a network to predict the angle of arrival of a signal. Labels are single values in the [-180, 180) interval.

I'm seeing a discontinuity in predictions around ±180 degrees, which makes sense as losses around that gap are incorrectly calculated by root mean square error.

I'm looking for a loss function that works in a modular way. A difference between 175 and -175 degrees should be calculated as 10 (instead of 350), if such thing exists.

It's my understanding that such a function introduces a discontinuity and thus may not be a valid approach. I'm looking for some guidance about how to deal with these kind of circular variables like angles, hour of the day, day of the week...

This has been addressed in the question "Encoding Angle Data for Neural Network", and I feel preserving linearity in angle variables is important (my input is also several angles), and I'm not getting good results with the sin/cos encoding approach proposed in that question. The problem is also discussed here: What is a correct loss for a model predicting angles from images?.

Here is what I'm doing currently, which works quite well with angles (-180, 180).

def metric_stddev_diff(y_true, y_pred):
    return tf.keras.backend.std(y_true - y_pred)

def model_create():

    model = tf.keras.Sequential([
        tf.keras.layers.Dense(128, activation='sigmoid', dtype='float64'),
        tf.keras.layers.Dense(64, activation='linear', dtype='float64'),
        tf.keras.layers.Dense(1, activation='linear', dtype='float64'),

    model.compile(optimizer='adam',  # 'rmsprop'  'adam',
                  loss='mean_absolute_error',  # 'mean_absolute_error'  'mean_squared_error'  'sparse_categorical_crossentropy'
                  metrics=['mean_absolute_error', metric_stddev_diff])

    return model
  • $\begingroup$ I kept using "mean absolute difference" which was working nicely in the angles case. I suspect that's wrong... I don't really know how to interpret this metric in the $\sin \theta, \cos \theta$ case and I guess the loss function is not valid. $\endgroup$
    – jjmontes
    Sep 6, 2019 at 17:55
  • $\begingroup$ I think that working with points in the unit circle and using mse will yield better results. Instead of feeding the model with angles, use the points (cos(a), sin(a)). $\endgroup$
    – rbarreiro
    Jul 8, 2020 at 9:35
  • $\begingroup$ Welcome to Cross Validated! What makes you think this? Do you have any references or simulations? How would you address the fact that this now is a multivariate regression with a response variable in the plane instead of the line? $\endgroup$
    – Dave
    Jul 8, 2020 at 10:21
  • $\begingroup$ Could you please share your new custom loss function? One suggested above doesn't converge for me either. I'm training a CNN for learning 3 orientation angles. $\endgroup$
    – latida
    Feb 16, 2021 at 13:34
  • $\begingroup$ The one above is not any of the recommended solutions. Try the complex approach (sin/cos) for the input encoding, and for the loss function try using either MSE of the angles, or one of the loss functions suggested by @whuber below. My approach was simply an ad-hoc weighted sum of MAE of angle and magnitude which I cannot recommend as I cannot really explain formally why it worked. $\endgroup$
    – jjmontes
    Feb 16, 2021 at 15:49

1 Answer 1


Almost any loss function that is symmetric and differentiable at $0$ is locally quadratic. Thus, you don't have to be too fussy when searching for a good loss function when you need symmetry and differentiability.

Notice that with nearby angles $\phi$ and $\theta,$ the Taylor series expansion of the cosine gives

$$\mathcal{L}(\phi,\theta)=2(1 - \cos(\phi-\theta)) = (\phi-\theta)^2 + O((\phi-\theta)^4)$$

is locally quadratic at $\phi-\theta=0$ (and all integral multiples of $2\pi$) through third order. Moreover, this function of $\phi$ and $\theta$ isn't badly behaved: it's defined for all angles, is differentiable everywhere, and--most importantly--respects the modular nature of angle comparison. Thus $\mathcal{L}$ is a natural and simple angular version of a quadratic loss. This would be a good place to start your analysis.

If you need more flexibility, consider defining your loss as a function of $\sqrt{2(1-\cos(\phi-\theta))}:$ clearly this is a circular analog of the absolute difference.

  • 5
    $\begingroup$ +1 A possible additional point of interest regarding this loss function: Minimizing $\mathcal{L}$ (or equivalently $-\cos(\phi-\theta)$) is equivalent to maximizing the likelihood of a von Mises distribution, analogous to how minimizing the squared error is equivalent to maximizing the likelihood of a Gaussian distribution. $\endgroup$
    – user20160
    Sep 7, 2019 at 18:22
  • $\begingroup$ I had realized that $(\phi-\theta)^2$ was the first term of the Taylor series but couldn't make such a connection to the cosine! However, I tried $1 - \cos(\phi-\theta)$ and it doesn't converge... maybe this approximation seems to only work well for small differences (as expected), whereas the output space spans the whole interval $[-\pi, \pi]$ :-? I am now using the complex representation with a custom loss that combines the absolute errors in angle and magnitude, and seems to behave much better near the limits and converge faster, while the absolute error is a bit larger for noisy datasets. $\endgroup$
    – jjmontes
    Sep 8, 2019 at 14:42
  • $\begingroup$ The cosine, being the real part of a global analytic function $\exp,$ converges everywhere. $\endgroup$
    – whuber
    Jun 10, 2021 at 20:05
  • $\begingroup$ @whuber ...with sufficient terms. $\endgroup$
    – TLW
    Feb 20, 2022 at 7:37
  • 1
    $\begingroup$ @TLW It's hard to understand that qualification, because "sufficient" implies a finite number, in which case we're talking about a polynomial and there's no question of convergence. You probably have accuracy of approximation in mind -- and certainly how well the first few terms of any series approximate its limit will always be of interest. $\endgroup$
    – whuber
    Feb 20, 2022 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.