I am constructing different configurations of a Random Forest in order to investigate the influence of well-design variables and location, on the first-year production volumes of shale oil wells, within a given area in the US. In the different model configurations, I control for location in different ways, to show how the influence of well-design variables may be biased when the spatial resolution of the models is inadequate. Here, location acts as a proxy for geological properties/reservoir quality.
I have a dataset of ~4500 wells, with 6 variables. The response is the first-year production volume, and the predictors are three different well-design variables in addition to longitude and latitude.
I have been researching and putting some thought into the subject of data partitioning when working with spatial data. For instance, in this chapter of "Geocomputation with R" by Lovelace et al. (https://geocompr.robinlovelace.net/spatial-cv.html), they highlight the importance of spatial cross-validation: "Randomly splitting spatial data can lead to training points that are neighbors in space with test points. Due to spatial autocorrelation, test and training datasets would not be independent in this scenario, with the consequence that CV fails to detect possible overfitting. Spatial CV alleviates this problem and is the central theme of this chapter."
The point is that due to spatial autocorrelation (near things are more related than distant things), you will end up with some observations in the training set that are very similar to observations in the test set if the proximity of observations is not accounted for when splitting the data. This may cause "information leakage" between the sets.
My question is, does this information leakage necessarily pose a problem? I figure that this and the similarity of observations is something that may just as well be representative of the problem at hand, and therefore make the performance assessment more representative of a real-life application of the model. I understand that a spatially disjoint test set yields a more representative performance assessment of your model if it should be used for predicting on a completely new and distant area. But if you want to assess a model's predictive performance with respect to a mix of near and distant locations, wouldn't a random split be more reasonable?
Hoping for some input here, thanks!
Edit: After reaching out to the authors of the abovementioned book on Twitter, I was advised to check out the following lecture by Hanna Meyer: https://www.youtube.com/watch?v=mkHlmYEzsVQ. She makes a distinction between "data reproduction" and "data prediction" (at approximately 16:40 in the video). This is something that crossed my mind while initially writing this post; that I am not really applying these models for prediction, but rather using predictive models as a tool for investigating factors that influence well productivity. After watching the video, I have become more confident that this application is more like "data reproduction", where a random partitioning seems OK, rather than "data prediction".