Say I have a way of predicting, for each link on a network, the likelihood of road accidents. I also have a (mercifully quite sparse) point data set of recorded accidents.
Bearing in mind that the occurrence of accidents is rare, I would like to test the efficacy of my prediction, in a way that is comprehensible to planners. The binary classifier framework would seem to be a good one for this, allowing me to give a sensitivity and specificity e.g. "the method correctly predicts 70% of accident locations, and correctly predicts 70% of non-accident locations".
If I divide the network into segments and bin accidents and predictor variable into segments, the analysis is heavily dependent on segment size (modifiable area unit problem). In any case, in general it is impossible to divide networks into segments of equal size. It is possible instead to estimate accident density based instead on the inverse of network length in each cell of a Voronoi diagram (each cell being a histogram cell containing an equal number $n$ of points where $n=1$) but that is also subject to bias.
An alternative is to estimate (nonparametrically) the kernal density of accidents, then sample the network randomly to produce a variety of (predictor,accident density) pairs for regression. Again this requires suitable choice of bandwidth for the kernal density estimation.
A final alternative is to randomly generate a bunch of points where accidents did not happen. The question is how many to generate? Say I wish to predict accidents within 30m, then I should take each accident to occupy a 30m length of road and generate enough random non-accidents (each over 30m away from any known accident) to account for the rest of the network length. This gives a bunch of pairs (predictor, accident [0,1]) suited to binary classification.
Can anyone comment on the validity of each of these proposed methods? I have seen (2) alluded to in the literature but not (3) which is of my own devising.