How to approach regression problem with ~30 datapoints and thousands of features? I'm trying to build a model that predicts the 6 numerical target values from a molecule. I encode a molecule with "descriptors", which give me about 2000 numerical features that describe the molecule in various ways. I have no chemical insight in this problem, so I have no idea which features may be relevant for predicting the 6 target values.
My dataset is very small: I only have the responses (the 6 target values) for 30 molecules, as it is very costly to test a new molecule. My goal is to create a model that can predict these 6 target values for a molecule, so I can choose which molecule to try next to see if it has the performance I wish.
I've read a lot about QSPR and QSAR (quantitative structure property/activity relationship, in essence predicting something about a molecule), but I'm overwhelmed by the fact that everyone uses a different machine learning model for their problem. Also, in my case I have much fewer datapoints than the typical QSPR models.
I'm equally overwhelmed by the variety of feature selection and dimensionality reduction algorithms: PCA, univariate feature selection, feature elimination... Can I know that some algorithms have more chances of being useful for my problem beforehand, or do I just have to test them all and see for myself?
To sum up, my questions are:

*

*What machine learning model would be indicated for this?

*What feature selection models would be useful, if any?

*Should I build an independent model for each of my 6 target responses, or try a multiple regression model? The responses are correlated.

My guesses would be perhaps SVM and random forest, as they have some feature selection inherent to the model and robustness to overfitting. But I'm inexperienced in the field of ML and any input would be greatly appreciated!
 A: 
What machine learning model would be indicated for this?

You need more data.  30 data points is not a lot to work with, regardless of the approach.  At best, I would say collapse the features onto the few principle components and then do a linear regression.  That's about as best you could do from what you're telling us.

What feature selection models would be useful, if any?

None would be useful.  Feature selection is known to be deficient for many reasons (many of which you can find here by searching other posts).  At best, you can collapse the data into the first few principle components.
A: This situation is common for example in Neuro imaging or genetics. The common recommendation there is to use linear regularized models such as ridge regression, elastic net, linear SVM or linear gaussian process. Often it is NOT recommended to tune parameters or try many models, because with such a small sample size, The model selection bias due to vibration effects is more dangerous than having sub optimal lambda. Also all these linear models perform very similar anyway.
The core problem is not that you don't have enough data to fit your models, but that you don't have enough data to estimate if they are any good or which approach is better.
PCA is closely related to ridge regression and L2 penalization, so running PCA regression or those regularized models also won't produce much of a difference. And again, you don't have enough data to detect model improvement that is not large.
A: I think you could use PCA or SVD to get the first 5-10 principal components, and you can look at the explained variance to see how pca works. Then, you could do a regression to find the connection or try random forest models(you have enough data compared to the feature now, but do not tune the hyper-parameters too many times, they could still lead to overfitting.). Or you could using lasso to pick 5-10 number of features and do the regression.
