I have N neural networks trained on different subsets of features of a dataset and with slightly different methods. My problem is a multiclass output, with the output layer comprising of the softmax output for each class.
The outputs for each input on every network look like:
array([ 0.05925509, 0.10252554, 0.72204741, 0.05106815, 0.00929179,
0.05581202])
With each element denoting the prediction 'pseudo-probability' of the network choosing that class
And I ofcourse have the actual labels of the form:
array([ 0, 0, 1, 0, 0, 0])
I want to use regression to find an optimal linear combination of these outputs for which to weight each network to in the ensemble model.
I want to find $\alpha_i$ where $Y = \sum{\alpha_i}X_i$ where $X_i$ is the prediction in the above format for the $ith$ network.
Would I need to use some sort of multi-dimensional regression here? Is it even valid to treat this as a classical regression problem since each 'feature' in my regression input is actually six dimensional?