What is the best way of going about dealing with few instances in support vector regression, e.g. only approximately 40? Also - is there an optimal way of dealing with outliers in this case of few instances?
First, I'd advise you to separate the outlier detection problem from the estimation one. Both however will have to take account of the fact that you are dealing with very few observations in a very large space.
For the outlier detection problem, I'de use some high dimensional outlier detection algorithm on the design space. Specifically, ROBPCA with $k$ set to $\approx n/5$ (in your case $k=8$).
ROBPCA will give you a rank $k$ approximation of the covariance matrix and the location vector of the non outlying part of your data. Using these I would advise you to compute a rank $k$ approximation to the vector of squared Mahalanobis distances $d_i^2(k)$ to attach to each observation a weight
another possibility is to attach to each observation a weight
This is done to prevent observations located far away from the bulk of the data to exert and out-sized pull and essentially drive the final estimates. In a nutshell, it insure that your model adequately describes the pattern of the majority of your observations.
see also page 14 of this note.
Given these weights, I'd use some form of bayesian shrinkage and a logit to get the final model. This is again necessary because of the extreme sparsity of the space you are considering. Have a look at the Firth method. It's implemented in the package logistf.