# How to construct a cross-entropy loss for general regression targets?

It's common short-hand in neural networks literature to refer to categorical cross-entropy loss as "cross-entropy," even though there are a number of loss functions which could properly be described that way.

So, in general, how does one move from an assumed probability distribution for the target variable to defining a cross-entropy loss for your network? What does the function require as inputs? (For example, the categorical cross-entropy function for one-hot targets requires a one-hot binary vector and a probability vector as inputs.)

A good answer will discuss the general principles involved, as well as worked examples for

• categorical cross-entropy loss for one-hot targets
• Gaussian-distributed target distribution and how how this reduces to usual MSE loss
• A less common example such as a gamma distributed target, or a heavy-tailed target
• Explain the relationship between minimizing cross entropy and maximizing log-likelihood.

Inspired by Tensorflow Cross Entropy for Regression? with thanks to CowboyTrader.