We are trying to train a random forest model on land-use and meteorological variables to predict daily concentrations of air pollution at a 1km resolution. Our input data consists of 1km raster stacks of these predictors, as well as a network of air quality point data which serve as a response matrix. We are extracting raster cells from our predictor stack that intersect with our response matrix throughout the course of an entire year before training the model. In other words, each day and at each location where we have air pollution data, we compare mean daily concentrations as a function of that day's land-use and meteorology data (which also change in time and space). In total, there is over a year's worth of daily observations to train on.
This approach yields an $R^2$ value of 0.87 after performing a 70/30 holdout and comparing the test data with the predictions.
Despite the decent model accuracy, the predicted rasters (air pollution heatmaps at 1km each day) are not aligned with our expectations of our study area. We have two main concerns with this model:
We are seeing the cleanest air pollution in the heart of the city (center of each tile), while the rural areas (outskirts of the tile) of our study area are predicted as being unreasonably high. This could be due to our unbalanced response matrix, as there are few locations in rural areas for the model to train on. Are there good ways for accounting for unbalanced response data when using random forest for regression? In addition to this imbalance, we are also concerned about possible edge effects that may be influencing the model in these areas.
Another main concern pertains to the degree of spatial heterogeneity of our predicted rasters. Throughout the literature, the air pollution response variable of interest has not been observed to have a high-degree of spatial heterogeneity. However, our model is suggesting otherwise. Even more confusing is that the most important variables (in terms of IncNodePurity) are the predictor variables that have been buffered to a coarser spatial resolution (usually we're seeing the 10 km rasters having higher variable importance than the 1 km rasters). And yet, the predicted surfaces have a surprising amount of variation at a 1 km resolution. We are worried that by letting the model see an entire year's worth of input data, that it's confounding observations across all four seasons when trying to make a prediction about a single day. As a result, we might be seeing a year's worth of spatial heterogeneity predicted each day.
Does it make sense to train a random forest model on a year's worth of observations if the ultimate goal is to predict daily exposures? Or is this introducing noise into our model? Is there a better way to account for the temporal component in addition to the spatial component?
