I am not very familiar with all methods in Machine Learning. However, I know for example when I apply linear regression, the y is always assumed to be one-dimensional.

My target is multi-dimensional. One approach would be to train a model for each dimension. But I wonder if there are other approaches that considers all dimensions together (so not to consider them independently, like if I train a different linear regression model for each dimension)?


1 Answer 1


We usually "reduce" multidimensional target to a single number. For example using cross entropy.

Here is an example of calculation in python. (from this link)

# example of calculating cross entropy
from math import log2

# calculate cross entropy
def cross_entropy(p, q):
    return -sum([p[i]*log2(q[i]) for i in range(len(p))])

# define data
p = [0.10, 0.40, 0.50]
q = [0.80, 0.15, 0.05]

# calculate cross entropy H(P, Q)
ce_pq = cross_entropy(p, q)
print('H(P, Q): %.3f bits' % ce_pq)

Think about this is for one data point, the output of the model $\hat y$ is vector $p$ and the target is vector $q$. The above calculation will get the loss in one dimension.

  • $\begingroup$ Thank you very much. This seems to be an interesting method. But I guess in my problem I need to consider all dimensions and my prediction should be also consisted of similar amount of dimensions. $\endgroup$
    – Kadaj13
    Commented Mar 12, 2020 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.