Timeline for Robust mean estimation with O(1) update efficiency
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 10, 2016 at 14:37 | vote | accept | Bitwise | ||
| Jun 10, 2016 at 14:37 | |||||
| Apr 22, 2014 at 19:57 | answer | added | whuber♦ | timeline score: 4 | |
| Apr 22, 2014 at 16:15 | comment | added | Innuo | To continue with the idea in @whuber's first comment, you can maintain a uniformly sampled random subset of size $100$ or $1000$ from all the data seen thus far. This set and the associated "fences" can be updated in O(1) time. | |
| Apr 22, 2014 at 15:13 | answer | added | Deathkill14 | timeline score: 4 | |
| Apr 22, 2014 at 14:06 | history | edited | Bitwise | CC BY-SA 3.0 |
added 37 characters in body
|
| Apr 5, 2014 at 2:04 | history | tweeted | twitter.com/#!/StackStats/status/452265565139513344 | ||
| Apr 4, 2014 at 15:24 | comment | added | Jason S | I would offer a bounty but i don't have enough reputation | |
| Apr 18, 2013 at 17:18 | comment | added | whuber♦ | That is a crucial observation. It implies you need to take more care than usual, because initially you will be obtaining a "robust" estimate of the mean high outliers. By continuing to update that estimate, you could wind up throwing out all the lower values. Thus you will need a data structure in which key parts of the entire distribution of data are recorded and periodically updated. Check out our threads with keywords "online" and "quantile" for ideas. Two such promising ones are at stats.stackexchange.com/questions/3372 and stats.stackexchange.com/q/3377. | |
| Apr 18, 2013 at 17:12 | comment | added | Bitwise | @whuber I cannot guarantee that the initial sample will represent the rest of the data. For example, the order in which I am given the data is not random (imagine a scenario where I am first given higher values and then lower values). | |
| Apr 18, 2013 at 16:50 | comment | added | whuber♦ | Why not just use an initial segment of the data--such as the first 100 or first 1000 or whatever--to erect "fences" for screening outliers? You don't have to update them again, so there's no need to maintain additional data structures. | |
| Apr 18, 2013 at 16:00 | history | asked | Bitwise | CC BY-SA 3.0 |