Pattern mining on a small data set I have a small data set 30 features/predictors and 30 observations. My target variable is Oil production and my predictors are well & reservoir properties (depth, trajectory, temperature, pressure etc) I want to identify the manner in which these features impact the target variable via identifying patterns that may exist in my dataset. The relationships may be non-linear and many features are correlated to each other. 
Can you please suggest me some great ways to approach this problem? 
Any help will be greatly appreciated! Thanks.
 A: Here are some strategies that may help you:


*

*Try glmnet, a user-friendly R package that implements elastic-net-penalized regression. In tuning the hyperparameters, you might be best off using leave-one-out cross-validation because your small sample size is so small. I suggest glmnet because it can handle $p\gg n$ cases where you have more predictors than observations. You could try this procedure using as many orders of interaction as your RAM can handle.

*Variable reduction strategies like Principle Components Analysis (choosing the components that explain, say 85-95% of the variance) may help, as well. Then again, every single time I have tried variable reduction strategies in $p\gg n$ cases, it hasn't reduced the parameter space that much, and once I found out about regularized regression I've depended even less on this. When using variable reduction, maybe focus on subsets of the variables that show high multicollinearity. If you find yourself in the happy situation whether you've reduced your predictor set to something quite small. Still, you may have separation or quasi-separation in your data, so maybe still go with glmnet for regularization, or a Bayesian regularization method as implemented in R package bayesglm.

*Use subject matter experts to create a set of alternative models containing more reasonable subsets of the predictors, and which actually use scientific theory to arrive at the functional relationships between the smaller subset of predictors and the response. Use Bayesian model averaging to combine these models together into a single predictive model. If you don't end up using all 30 features, who cares so long as you have improved prediction substantially beyond a coin flip? Sometimes, the machine learning approach isn't the right one. When you have a very small data set, and you're trying to reduce the dimensionality as much as possible, well, that's pretty much the anti-machine-learning-big-data-type use case.

*Collect more data.
A: Get more data.
This is probably not the answer you're looking for but there's nothing else to tell you. Frankly, you currently don't even have enough data for a test set. [Are there really no resources online that have similar datasets you can (freely) use??]
Sure, someone may come along and suggest you try some fancy bootstrapping technique but, frankly, I really think you're best served by scouring the internet for some more data you can use (even if you don't end up using this data in your final analysis, additional data could still indirectly help you a lot by suggesting a functional form for your analysis).
Good luck!
A: http://www.math-gnostics.com/index.php?a=introduction
Forget the old Gauss and his models on small datasets. This one is possible the best solution. It will provide you with a continuous empirical distribution model (passing all Goodness-of-Fit tests ex. Komolgorov-Smirnov) even on small sets of data.
I haven't talked to Kovanic (he must be old man now) for years but I have worked with his older SW for years and it is more like miracle than statistical method.
