Machine Learning questions here.

Suppose I have a dataset of labeled 1,000 records, and 200 features - and I use R to test out a neural net or random forest's abilities to predict a certain feature.

What, if anything, should change if I instead have 1,000 features, and 200 records? Is feature selection my only option?


Is feature selection my only option?

NO. With 200 observations and 1000 variables, the variables must necessarily be heavily correlated in the sample, (even if the population variables might be unrelated.) But feature selection (based on the sample, that is) is not a good solution. A better solution is often regularization, see for instance Intuitive understanding of regularization or Problem specific regularization, or search this site.


From what I understand, the wide vs narrow data is just used to classify the presentation of data: https://en.wikipedia.org/wiki/Wide_and_narrow_data

A neural net is mostly a advanced form of fitting a linear fit such as

y =a*x+b.

To fit the above equation you would need atleast 2 data points. Imagine what would happen, if you just had 1 data point.

Your case of having 1000 features and 200 data points is similar to having 1 data point to fit a linear equation.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.