I have a KMeans model that is trained on features that are percentage-transformed descriptions of events. Each observation contains between 1 and 180 events. To help with meaningful comparisons, I restrict the model training set to only those with sufficient events (say 10) so that I'm not comparing, say, an observation that includes 2 events and has a score of 50% against an observation that includes 20 events and has a score of 50%.

My client has asked me to score all of the observations, and suggested fitting the trained model to all data. I'm reluctant to do so because the restricted subset differs systematically from the training set by having fewer than 10 observations, and though I don't have a test for it the systematic difference would mean the two subsets aren't iid, with the violation on "identically".

Is it best to cast the restricted group into one cluster, or is it possible to estimate whether I can safely include at least some level of the restricted group (say, those observations with 8 or 9 events) when I fit the trained model?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.