0
$\begingroup$

I have a Machine Learning problem at hand but I'm not sure how to approach it. I have a dataset which has around 5000 observations and around 250 features(most of them are numeric and around 3-4 are categorical, like A,B,C or red,blue,orange). This synthetic dataset was generated using some model and I don't have any information about that. My goal is to predict the target variable which is real valued.

Just to start off my prediction task, I handled the missing values in the dataset by replacing them with the median of the attribute. Then I used a Linear Regression model with all the numeric features (removing the categorical variables for the time being). However, i feel this is definitely not the right approach firstly because I'm using too many features compared to the number of observations I have. Also, I should be making use of the categorical variables.

Could someone please tell me what should be the right way to approach this problem? I was thinking of doing feature selection first but I'm not sure how to do that. What steps should we take if we want to select a particular feature (like various statistical tests or anything else) ? Any kind of input will be extremely appreciated.

Improve this question
$\endgroup$
1
  • $\begingroup$ I am confused. How are you using too many features compared to number of observations? You mean using 250 features is too many? Any reason why you feel so? $\endgroup$
    – caveman
    Mar 12, 2016 at 0:46

1 Answer 1

Reset to default
2
$\begingroup$

If you're new to machine learning and familiar with R I might suggest reading up on random forest and gbm. I am very much still learning but have found these to both be relatively simple to apply and found they produce good models even with a great many x variable. Variable selection will help with managing computer resources and is often important part of an optimisation strategy, but from the size of your data set I would consider these two to be an excellent start.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Tests that spring to mind for variable selection; correlation tests such as Pearson's or Spearman's for excluding some of the highly correlated variables, excluding variables with little variance (try inter quartile), clustering or even PCA (if you don't have to explain easily what your x variables mean). $\endgroup$
    – Jernau
    Mar 11, 2016 at 20:44
  • $\begingroup$ you can edit your own answers, and include content there. Welcome to CV. $\endgroup$
    – EngrStudent
    Mar 11, 2016 at 20:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.