Sycorax addresses the use of auxiliary heads / shortcut losses for addressing gradient flow.
But there are many other types of auxiliary losses still in common use today. For example, it is common to add an auxiliary GAN loss or perceptual loss for conditional generative modeling tasks (3D shape reconstruction, depth estimation, etc). In reinforcement learning, people will often add auxiliary rewards for reaching some intermediate states of the environment, or tack some unrelated vision task loss onto an agent's CNN to encourage learning of useful features. Some sort of entropy or diversity loss is used in many models from many different areas (but you could think of that as more regularization than loss).
Usually, we try to optimize the model for something not easily formulated as a differentiable mathematical function -- for example, whether an image looks "natural". The loss function is an initial best attempt at specifying that objective. I like to think of auxiliary losses as additional adjustments to fine-tune and nudge the model towards doing what we want it to do.
So is the purpose of having an auxiliary loss function be to improve
performance that cannot be done by the loss function alone?
Yes, exactly -- because the loss function is an imperfect specification of the objective.