how do you determine which kind of classifier is best for your data,
The high level answer here is that you want to select a model (classification or regression) that suits
- your application task
- your data, and
- the available sample size.
Wrt. 1, @FrankHarrell already pointed out that you need to decide classification as in the output of the model being a class label vs. predicting the probability of the case to belong to a given class. There are further considerations here: e.g. are your classes mutually exclusive or not and how well-defined you expect your classes to be (rough guideline: can you describe characteristics of your class by "positive" words => usually well-defined), or is that description negative in the sense that you can mainly say what it is not => usually ill-defined)
As someone working with spectroscopic imaging data as well (my background is spectroscopy and chemometric data analysis), I'd recommend that you think even more about "proper" regression here. Such a regression model would e.g. predict for each pixel the "concentration" (or a fraction) of shrubs, grasses, trees and mosses. This would allow for pixels to contain mixtures such as half grasses half shrubs - whereas classification models cannot deal well with this situtation.
Which brings us also to point 2: spectroscopy (I guess UV/VIS/NIR reflectance?) is widely used because the spectra (if expressed in the proper unit, e.g. absorbance or log reflectance) are in good approximation linearly related to concentration of chemical species, and that in turn means that they are also linearly related to defined mixtures. Such as the phenotypcial composition of grasses vs. shrubs.
Which points to (bi)linear models being appropriate*. Also, assuming you have full spectra rather than only a few wavelengths, the underlying physics and chemistry of the spectroscopic technique allows to say e.g. that regularization via latent variables/factors (PCA/PLS/ridge) fits better with your particular type of data than trying to select a few wavelengths (LASSO).
- sample size. This is in a sense easy: the smaller your sample size, the more you need to take care that you don't overfit.
- the smaller the sample size, the more important it is that you bring in external knowledge via selecting an appropriate model ansatz (see 1.).
- you need to monitor/measure overfitting, which is possible e.g. via repeated cross validation or out-of-bootstrap valiation, but not with a single run of cross validation nor leave-one-out cross validation nor a single split test set.
- you then have two approaches available to reduce overfitting: either your regularize your model more, i.e. force it to be less complex (e.g. dimensionality reduction by PCA or PLS, or adding a ridge) or you can aggregate overfit models (e.g. random forest).
* Note that the statistician's "linear" in linear model refers to the parameters, whereas the chemist's "linear" refers to the concentration. Chemometrics uses the term bilinear to express that a model is linear in both senses.
and if and what kind of resampling method to use?
The choice of "validation scheme" is IMHO less critical/fairly easy:
- you will pretty much always be OK with repeated k-fold cross validation or out-of-bootstrap.
- I tend to reserve "single" test sets for proper valiation studies. more here
- I avoid resampling or splitting "flavors" which do not allow to measure model stability/overfitting. E.g. I use leave one out only if there are so few independent samples that I cannot possibly leave out 2 at the same time - this boils down to doing LOO if there are < ≈5 independent cases.
The crucial point instead is to ensure independence. To speak in your example, the data I get would typically contain thousands or millions of spectra, which are however, only from, say, 10 different (separate) peatland regions. This basically means we're speaking about sample size 10 for application scenarios that claim to predict for any other peatland area. Thus, it wouldn't matter whether you use repeated k-fold cross validation vs. out-of-bootstrap. But in both cases it would matter crucially that you split by peatland region.
(Even if your application scenario stays within one peatland region, neighbour pixels are likely not independent - establishing independence is more difficult then, but just as important)
For the sake of completeness, independence also means independent of any modeling choices. I.e., once you use resampling results to choose between modeling options, they cannot be used as performance estimate any more since they are now part of the model training process.
Let's say I were to pick RandomForest, which I think is one classification algorithm; would I still need to cross-validate my data with a resampling technique?
So, let's check the random forest against the points above.
Random forests have been (successfully) used for spectroscopic data even though they don't fit that well with spectroscopic data since they look at single wavelengths while we expect the information to be spread out over many, in particular neighbouring, wavelength channels for physical and chemical reasons.
(I'd recommend looking up random forests fairly in detail if you use them: they are aggregated models and thus act different from many other models in specific senses.)
The usual random forest implementations implicitly assume that every row of the data is an independent case. Under this assumption, the internal out-of-bag performance estimates will be fine. But this assumption is clearly violated here (as in almost all real-world data I've met so far).
The overfitting and bad calibration (predicted class membership probability being severely off) @FrankHarrel points out are symptoms of this.
You now have two possibilities:
use a random forest implementation that provides independent splits also for your data. (Which would be best, but likely boils down to you writing your own implementation of RF)
You may be able to live with the fact that the RF is overfitted, and thus stays behind what would be possible with approach 1. This "only" requires that you show the RF you obtain is anyways good enough for your application purposes.
For this, you need an independent test of the (overfit) RF you obtain - e.g. by wrapping the RF training in your own repeated k-fold cross validation which provides independent splits for your data.
