Location variables as factor or numeric?

Question

I have a data set where my observations are geographically referenced by longitude & latitude, as well as by township & range. If you're unfamiliar with township & range:

The land is divided into survey townships of roughly 36 square miles. This is done by establishment of township and range lines. Township lines run parallel to the baseline, while range lines are true meridians. (https://en.wikipedia.org/wiki/Public_Land_Survey_System)

I want to run two different configurations of a Random Forest model. One where I account for spatial autocorrelation by including longitude & latitude as predictors, and one where I use township & range for the same. The point is to develop two models of different "spatial resolution", where the one with longitude & latitude is the high-precision case and the one with range & township is the lower-precision case (this would in a way resemble a county-level fixed effect).

At the moment, I have both these variables encoded as numeric variables (I am using R). For longitude & latitude, the precision can be infinitely increased by increasing the number of decimals, so this seems reasonable. However, range & township may only be integers (there's no such thing as township = 5.5). I have tried to encode them as integers, but this doesn't seem to make any difference, as R seems to treat them in the same manner as normal numeric variables. If I encode them as factors, the variable importance gets way more messy since each category acts as a dummy variable, and I am not sure whether it's "more correct" anyway.

My question is: Would there be any problems related to keeping the township & range variables as numeric in this case? If so, does anyone have any suggestions on how to store these variables in a different way?

PS, I have read this post: GLMM: Elevation as numerical or factor in model?, but since random forests are capable of modelling non-linear relationships, the accepted answer does not seem to be exactly what I'm after.

Answer 1

I think it will be reasonable to consider encoding township and ranges in a way other than simply treating them as integers. While it is safe to assume that RF are able to encapsulate non-linear relations, it will still be rather difficult to interpreter results from a integer encoding. In addition, spurious effects might be very prevalent. For example, if a relatively low-counts categorical level has an assigned numerical value that is close to the assigned numerical variable of high-counts categorical level, it will be probably pooled together with it even if they encode relatively different information; and that would be just due to the encoding, another (random) integer encoding might give something else.

Similarly, as you have correctly deduced, because township and range have high cardinality (i.e. they have many distinct variables) as well as they have an inherit hierarchy, using them directly as factors quickly becomes unwieldy. The main problems are: 1. We might overfit out data when concerned about the influence of very unusual/rare factors and 2. with thousands of factor levels is difficult to estimate the overall importance of a factor as a whole. Bonus problem: (3.) We might not respect the hierarchy among two level (e.g. in the UK, where the post-codes (e.g. EC1Y 8LX) have an outward code (EC1Y) and inward code (8LX) part, it makes little sense to analyse inward post codes in absence of outward post codes).

I would suggest looking into target encoding (Micci-Barreca (2001) A preprocessing scheme for high-cardinality categorical attributes in classification and prediction problems) (or other target aware encodings like the James-Stein Encoder or the M-Estimator encoding). The basic idea is that a discrete factor variable is replace by a (regularised) average of the response variable. Here is a quick example: if we model a response variable y and we have three options of a variable Sex: Female/Male/Other, each of them represent 53%, 46% and 1% of our sample respectively. We will then create a new "numeric version" of the variable Sex where Female will be replaced with the mean of y for Female correspondents, $\mu_{y;Female}$, Male will be replaced with the mean of y for Male correspondents $\mu_{y;Male}$ (or something a bit closer to the overall average of y, $\mu_y$ due to regularisation) and Other will be replaced with the mean of y for Other correspondents, $\mu_{y;Other}$ (or likely something even closer to $\mu_y$, due to even stronger regularisation as the proportion is smaller).

That and a bunch of other goodies can be found in the vtreat package that has a number of pretreatment step for data. They have a number of vignettes that touch upon a number of different variable pre-processing issues.

Side-note: There are other encoding schema like Binary Encoding (i.e. turn everything into integers and then make $p$ distinct variable where they hold the $p$ digits 0/1 required to encode the integer) and Feature Hashing (Weinberger et al. 2009 Feature Hashing for Large Scale Multitask Learning). I do not touch upon them here as variable importance and influence is even harder to interpreter in these use cases. They are mostly dimensionality reduction pre-processing steps.