How do we decide on how to fill missing values in data? I have a data set with NA values in many predictor variables.
How do we impute the best values ?
I have 302 variables in total. Out of them 236 belong to some abstract category, 37 to other, 9 to other category.
PS: I am solving the following regression problem from Kaggle
https://www.kaggle.com/c/liberty-mutual-fire-peril
 A: As mentioned before, you can impute the missing values using means, medians KNNs or even more sophisticated models. We sometimes choose to omit the whole row if the NAs are not that many.
I would go a step back and I would get the % of missing values for each column and try to treat each column in a different way or group “similar” columns together. What I usually do afterwards is for categorical or numerical values with a lot or NAs is that I create a new category “No info” with the missing values. If that variable was numerical, then you will have to make it categorical by cutting it at different cut off points based on quantiles or “reasonable” points depending on what this variable is about. Again you lose some info but it’s sometimes better than losing the whole row or biasing the models with imputations.  
A: @hvedrung has already suggested few good methods for missing value imputation,
1)Replace missing   values with mean,mode,median.
2)If data is categorical or text one can replace missing values by most frequent observation.
3)EM algorithm is also used for these purpose.
4)In R language, 
   4.1)package DMwR has  "knnImpute" method.
   4.2)base packages has "with" method, mice package has "complete" methode. 
through which missing value imputation can be done. 
A: I would not try to fill in values for predictive variables, I would simply eliminate them from your analysis.  My main concern is if these unknown values follow some sort of pattern, then you are going to introduce bias into your model.  With 550MB of data, this should not cause a problem of reducing your dataset.
However, the danger of using a method to replace these values is that you can create a model that picks up on your replacement technique, not the data.  Your model will produce artificial predictions that will fit your "fixed" data, but perform poorly against new data.
Also note that this contest was over last September.
A: Dont forget to check for things like serial correlation and correlation between x variables (you might be able to just remove the most deficient x variable if it is supplying redundant information)
One of the better substitution methods I have found is to create a random dataset with a similar distribution to the variable with the missing values, and then sample from that dataset to fill in the missing values. 
If you know that the previous measurement is related to the next measurement, you could perform an interpolation over the values you have and estimate the missing values that way.
A: *

*Simplest way but often not very efficient - delete data with missing values in any column.

*Substitute with average value. Average for entire data set or set grouped by features you know is important.

*Useful trick - add feature which is true if your value is missing and false otherwise.
