What distance function to use for matching observations on a mix of ordinal, nominal and numeric variables?

Question

I'm trying to match potential respondents to a random sample from the target population, and not sure whether any of the usual methods available out-of-the-box in e.g. R are right for the job.

Variables to be included in the distance--or dissimilarity--measure include geographic location (continuous), income class (ordinal) and gender (nominal), and about five others.

I would like a distance measure that works on this type of mixed data, and that has no parameters to be set manually (doesn't require the researcher to make "informed" decisions about the model once the measure is chosen). Ideally it would already be implemented in an existing R or Stata package.

Some distances I've considered so far:

• Propensity scores have the disadvantage of reducing the whole variable space down to a single dimension and can lead to imbalance (King, Gary, and Richard Nielsen. “Why Propensity Scores Should Not Be Used for Matching,” 2019).
• I know YouGov uses a custom weighted Euclidean-type metric for a similar purpose, which involves making judgment calls about each variable's weight (Ansolabehere, Stephen, Brian Schaffner, and Sam Luks. “Guide to the 2016 Cooperative Congressional Election Survey,” 2017).
• Mahalanobis distance looks like a good candidate, but in its standard form is only appropriate for numeric data--there exist extensions to it but they look like they would be difficult to implement (e.g. Leon, A.R. de, and K.C. Carrière. “A Generalized Mahalanobis Distance for Mixed Data,” 2005).

Any suggestions would be much appreciated!

Answer 1

The simple answer is to try whichever yields the best balance between your matched group and the target (i.e., random sample) group. Try propensity score matching, try Euclidean distance matching, try Mahalanobis distance matching. The theoretical properties of each approach do not describe how they will function in your data set, only in how they function in data sets on average. If an unlikely method works in your data set, that's the method you should use.

(Note that the King & Nielsen paper hasn't been peer-reviewed, and their argument is that PS matching can reduce balance, which you can empirically assess, and that it fails to recover a block randomized trial, but you only care about reproducing a randomized trial, so their critiques are not really relevant.)

There are a variety of techniques you can try. Among matching techniques, you can try genetic matching, which is a variant of Mahalanobis distance matching where the importance of the variables is determined by weights that are chosen using a genetic algorithm whose cost function is imbalance, so it will tend to find a balanced sample. It is implemented in the Matching and MatchIt packages in R. You can try cardinality matching, which allows you to specify a set of balance constraints, and it will find the largest matched set that satisfies those constraints. It is implemented in the designmatch and MatchIt packages in R and has options specifically for nominal variables.

Another option is to use weighting, which involves estimating sampling weights that yield balance in the weighted samples. You can use standard propensity score weighting using logistic regression propensity scores (note King & Nielsen mention their critique doesn't apply to propensity score weighting), you can use weights that are computed from propensity scores estimated with machine learning (such as generalized boosted modeling with balance optimization, implemented in the twang package in R), or you can use weights estimated directly using optimization that bypass propensity scores, such as entropy balancing (implemented in the ebal package in R and the ebalance package in Stata) or stable balancing weights (implemented in the optweight package in R). All these weighting methods and others are implemented in the WeightIt package in R.