Given a set of $n$ instances. For each instance I have a feature vector consisting of $m$ (numerical) features ($x_1$, $x_2$,...,$x_m$), n>>m. Moreover, for each instance I have a numerical score $y$ (observable). I would like to:
- find out which subset of features, or linear combination thereof, explains the scores the best.
- create a nice visualization for this.
I have been pointed to Principal Component Analysis (PCA). The problem with PCA is that it only takes the feature vectors into account; PCA does not relate the features to the numerical score $y$.
Practical application: Given a large number of problem instances (e.g. traveling salesman problems) and some algorithm to solve the problem. Each time we solve the instance we can measure the total time (=score) it took to solve the instance. Moreover, for each instance we can obtain a number of features, e.g. size of te instance, graph diameter, etc. Which of these features explain the computation time best?