# Problem with least squares cross validation in statsmodels

I'm trying to calculate a KDE bandwidth using the least-squares cross validation (LSCV) method. I'm using the statsmodels KDEMultivariate function in Python. But I'm running into an issue where the result is very dependent on the data supplied and sometimes returns a bandwidth that does not make sense. By moving one of the input points slightly it returns a reasonable answer. Is this an error in statsmodels or to be expected with this technique? If the error is inherent in the technique can it be handled with jittering, normalization or some other data cleanup? If the error is in statsmodels is there alternate statistical library that can calculate a bandwidth using the LSCV method?

import statsmodels.api as sm
#this is the original data
df_breaks = pd.DataFrame([[  538139.47293084,  3506243.03653242],
[  538150.09004195,  3506237.73434416],
[  538080.50079698,  3506151.83736346],
[  538124.70281065,  3506254.52157404],
[  538136.83381139,  3506245.77673011],
[  538155.19176266,  3506217.94499408],
[  538157.14459963,  3506216.01139807],
[  538120.47723684,  3506250.05577059],
[  538094.69172001,  3506211.3862365 ],
[  538140.71494773,  3506238.02450588],
[  538150.71773573,  3506242.01951111]])

kde = sm.nonparametric.KDEMultivariate(df_breaks, "cc", "cv_ls")
print "kde.bw", kde.bw


kde.bw [ 1.87127711e+01 1.00000000e-10]

import statsmodels.api as sm
#this is the original data with the y value of the last moved slightly
df_works = pd.DataFrame([[  538139.47293084,  3506243.03653242],
[  538150.09004195,  3506237.73434416],
[  538080.50079698,  3506151.83736346],
[  538124.70281065,  3506254.52157404],
[  538136.83381139,  3506245.77673011],
[  538155.19176266,  3506217.94499408],
[  538157.14459963,  3506216.01139807],
[  538120.47723684,  3506250.05577059],
[  538094.69172001,  3506211.3862365 ],
[  538140.71494773,  3506238.02450588],
[  538150.71773573,  3506242.5]])

kde = sm.nonparametric.KDEMultivariate(df_works, "cc", "cv_ls")
print "kde.bw", kde.bw


kde.bw [ 9.61735219 5.29485534]

• Does it get better if you subtract the mean from each column? I have no guess on the problem, but the scaling of the data doesn't look good and the sample is small for nonparametric methods. – Josef Aug 8 '16 at 16:43
• This is a subset of the total points, but there are not many more, 30 instead of 10. Subtracting the mean does not change the result. – Colin Talbert Aug 8 '16 at 16:51
• Is an LSCV bandwidth estimator inappropriate for this few number of points? I was under the impression that is was the standard method for bandwidth estimation KDE home range estimation (in wildlife biology and ecology that is). – Colin Talbert Aug 8 '16 at 16:54
• The bandwidth selection uses nonlinear optimization, Nelder-Mead fmin from scipy. but doesn't allow users to change any options. I don't know if that makes it fragile. The implementation follows Racine and co-authors and should be similar to the np package in R. (I don't know the statistics well enough to tell whether problems are "inherent" to LSCV.) – Josef Aug 8 '16 at 17:24
• "The least squares cross-validation function LSCV(h) can have more than one local minimum" stat.washington.edu/courses/stat527/s13/readings/… page 591. bw="cv_ml" seems to work more robustly – Josef Aug 8 '16 at 17:58