I have a Machine Learning problem at hand but I'm not sure how to approach it. I have a dataset which has around 5000 observations and around 250 features(most of them are numeric and around 3-4 are categorical, like A,B,C or red,blue,orange). This synthetic dataset was generated using some model and I don't have any information about that. My goal is to predict the target variable which is real valued.
Just to start off my prediction task, I handled the missing values in the dataset by replacing them with the median of the attribute. Then I used a Linear Regression model with all the numeric features (removing the categorical variables for the time being). However, i feel this is definitely not the right approach firstly because I'm using too many features compared to the number of observations I have. Also, I should be making use of the categorical variables.
Could someone please tell me what should be the right way to approach this problem? I was thinking of doing feature selection first but I'm not sure how to do that. What steps should we take if we want to select a particular feature (like various statistical tests or anything else) ? Any kind of input will be extremely appreciated.