Missing data is always a delicate area in analyses. Unfortunately there is no 'gold standard' and many times researchers simply exclude samples with missing values (which you are trying to avoid and also has its own implications). There are many methods that are acceptable but the most important part is that you document and report whatever method you implement. The first thing you should ask is ‘Why are my data missing?’ There are generally considered 3 different scenarios.
Missing Completely at Random (MCAR) – Ideal situation, nothing can explain what certain values are missing.
Missing at Random (MAR) – Data is missing conditionally at random, that the data ‘may’ depend on an observed measurement.
Missing not at Random (NMAR) – Data is missing conditionally, that the data depends on an unobserved measurement. Worst scenario, must be very careful about your missing value imputation.
If your data fit either MCAR or MAR you can jump in to the next stage. If NMAR, then you should be very cautious and explore specific methods for this scenario.
Next, naturally it depends on your data. There are several simple methods that may work very well if you are missing only a few data points. These include mean, median, and minimum imputation. Here is already a place to apply your knowledge of your dataset. Let's look at an example:
Let's say I have run a mass spectrometry experiment I measured many variables (100’s to 1000’s) but not every variable was seen in each sample. Now, I could explore this dataset and see that the variables which are missing are only seen at low levels or very high levels. In this scenario, it may be best to impute the ‘minimum’ (either of measured value or machine detection limit) as it is probable that the variable levels are below detection levels. An oversimplification of a complex experiment but should demonstrate the idea.
Of course, there are far more advanced techniques you can also apply. I am not sure about your programming background but I primarily use R for this type of analysis. There are many methods available in R. These can include using k-nearest neighbors (KNN) and bagged trees which are included the
caret R package, which I may add is wonderfully documented here and you may find useful for your model training (including parallelism to speed things up). There is also the
missMDA package for PCA, MFA, MCA, FAMD imputation. There are also packages that are strictly for the purpose of missing value imputation such as
mice and Amelia. Amelia is nice that it allows parallelism but I have read that it may cross if you have highly collinear variables. The
mice package is nice for exploring your missing data. There is an example of implementing it here, however I have also heard that
mice may take a long time (feel free to experiment).
Regarding your very high values, if you are certain that they are impossible you can still apply the above techniques where you have specified a cutoff for 'too high therefore missing value'. Again you must decide whether a simple or advanced method is most appropriate. Even with all the computers and mathematical advancements, there still is 'no free lunch' but this information should help get you started.