Particular classifiers can give consistently wrong leave-one-out predictions (typically with small sample sizes).
The reason is often that with LOO, the tested class is always underrepresented in the training set wrt the whole data set.
If this is the case, statified resampling validation (e.g. stratified k-fold cross validation) would be better. In your case, stratified 20-fold (= leave-3-out) or 10-fold would fit nicely with the data.
And no, reversing labels (without clear proof how they came to be confused - which means you correct a then-known error) is IMHO never a good idea.
What you can do, though: randomly mix the labels. This gives you a data set where you know that performance can only be guessing. If on that data you also find << 0.5 AUC for the LOO, that also hints at the "consistently wrong because systematically underrepresented" explanation for the LOO results.