I am working on a research project and I am trying to find one variable (execution time) as a function of a number of other variables/ metrics which were logged during the execution of a job. There are hundreds of these metrics and I want to find out which of these metrics/variables can actually be used for predicting execution times of jobs. So it is dimensionality reduction before I can use some sort of regression or machine learning. The problem is the number of rows (observations) is much smaller (45) as compared to the number of columns and therefore, rank deficiency. I have tried PCA, did not give any significant results. What other techniques can I use to find out a smaller subset of metrics that actually matter in execution time?

  • $\begingroup$ This is usually addressed by profiling the execution. $\endgroup$ – Aksakal Dec 20 '14 at 3:29
  • 3
    $\begingroup$ Have you considered penalized regression techniques such as Ridge, LASSO or LARS? $\endgroup$ – Zachary Blumenfeld Dec 20 '14 at 4:54
  • $\begingroup$ I have not actually, I will read up on that. $\endgroup$ – Wajahat Dec 21 '14 at 3:26