Skip to main content
Tweeted twitter.com/#!/StackStats/status/103989148213456897
edited tags
Link
Post Migrated Here from math.stackexchange.com (revisions)
Source Link

Bias from increased information in FLAME clustering

I hope this is an appropriate forum for this question...if not, any pointers on a place to ask would be great. If my questions is not clear, please just let me know and I'll try to add information/explanation where I can.

Say I have a set of data points that I run through a FLAME clustering algorithm, and I get my set of cluster supporting objects, and the fuzzy memberships among them.

Next, let's say I get an additional set of data, for argument's sake, let's say it doubles the amount of data. If I want to add that to my clustering, without having to reprocess the old data (i.e. just using the cluster supporting objects, and adding these new observations to the cluster), will I be introducing some kind of bias? To get the most "accurate" picture, would it be better to reprocess ALL the data, re-establish CSO's, re-compute distances, etc.?

To add to the complexity, let's say I'm going to be periodically adding similar quantities of data points. Does the answer to the above change?

And I guess a final question - is there a clustering algorithm (or family of algorithms) that is not subject to extreme bias in the beginning, such that added data points can be added relatively pain-free? I'm sort of thinking that FLAME clustering, with its nice fuzziness properties is a reasonable way of going about this, but any more suggestions would be helpful.

Thanks!