Does it make sense to say that the odds of some risk for a person in group A is $0.4$ times lower that that of someone in group B? Or would it be better to phrase it in "higher than" language?
Mathematically there really is no right answer. Whether you want to phrase this contrast as "0.4 times lower than" vs "2.5 times higher than" depends on the argument you are trying to build.
The argument is easier to build if you choose a more intuitive reference group and let that be the denominator. This may be chosen as:
I assume that your logit includes a dichotomous (or otherwise polytomous) independent variable that has been dummy-coded to represent group A versus group B (etc.) and for which you have an odds ratio value (OR=0.4). The interpretation of the odds ratio for any independent variable (especially continuous and even simple dichotomous ones) is, unfortunately, not very practical. Truly, a one-unit change in your independent variable (i.e., switching from group A to B) will reflect the change in the odds for your outcome in this case (0.4), but don't confuse odds with probability. Please do not mistake the odds ratio for meaning the "times more or less likely" your risk event will occur for group A versus group B.
A more practical interpretation might be the percentage change in the odds proposed by Roncek & Swatt (2006) in the Social Science Quarterly, 87(3), which is typically applied to continuous independent variables but can be calculated for categorical predictors, as well.
Even better, if you want to say group A or B is more or less likely to have a risk outcome, it is advised that you calculate predicted probabilities. A simple Google search of "predicted probabilities" will yield some straightforward and helpful readings and even Excel worksheets that will transform your logit coefficients into probabilities for you. That way, you can compare interesting cases and find the change in the probability of your risk event for switching between group A and group B.
Ultimately, interpreting your results as "higher than" or "lower than" is really a matter of personal preference or practicality depending upon the way you've worded your hypotheses and/or what you're trying to demonstrate in your overall argument.