For me a model (the way we calculate an output given an input) and a loss function (the way we estimate the accuracy of a model) always were two different (independent, "orthogonal") things. In other words, I though that any model can be trained and evaluated with any loss function.
However, it looks to me that there is some "binding" between models and loss functions. For example here I see the following statement:
You can also create your own loss function. Some examples of existing losses are: LinearRegressionOutput, which computes the l2-loss between it’s input symbol and the labels provided to it; SoftmaxOutput, which computes the categorical cross-entropy.
So, it looks like softmax function is somehow "bound" with the "cross-entropy" and linear regression is bound to the l2-loss. But what does this binding mean? In what way are they bound? I thought that softmax is just a way to normalize output (so that all the values are forced to sum up to one, so that they can be interpreted as probabilities). Is it incorrect to measure accuracy of softmax using squared deviations? Or, alternatively, can't we measure accuracy of linear regression by something other that l2-loss?