Timeline for Finding most likely permutation
Current License: CC BY-SA 4.0
26 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 20, 2021 at 8:34 | vote | accept | GioMott | ||
| Jan 14, 2021 at 22:32 | comment | added | JiK | @Aksakal I guess I just don't understand your phrases "doesn't work", "always works", "works on average" etc. or "on most datasets this will be the best correspondence". | |
| Jan 14, 2021 at 22:28 | comment | added | JiK | @Aksakal I don't think sorting would work in those cases either? | |
| Jan 14, 2021 at 22:26 | comment | added | Aksakal | @JiK disagree. If your tail is fat enough there will be cases when double flips happen and your permutation likelihood will misfire | |
| Jan 14, 2021 at 22:24 | comment | added | JiK | @Aksakal Well, then for a given fat-tailed error distribution a method that computes the likelihood of every permutation by brute-force, and returns the one with best likelihood, gives better results than sorting. It cannot be worse than sorting on any dataset, and it's sometimes better, such as in fblundun's example. | |
| Jan 14, 2021 at 22:18 | comment | added | Aksakal | @JiK I mean that on most datasets this will be the best correspondence | |
| Jan 14, 2021 at 22:04 | comment | added | JiK | @Aksakal What do you mean by "works on average"? | |
| Jan 13, 2021 at 18:08 | answer | added | Sextus Empiricus | timeline score: 2 | |
| Jan 13, 2021 at 13:32 | history | edited | GioMott | CC BY-SA 4.0 |
added 223 characters in body
|
| Jan 13, 2021 at 13:31 | comment | added | GioMott | @Aksakal: I implicitly assumed that the "true" weights are all distinct (although due to error the actual measurements may not be all distinct). I edited my question for clarity. Note that by "method" I mean any algorithmic approach (it does not have to be simply sorting the values, when the errors are not normally distributed) | |
| Jan 13, 2021 at 13:03 | comment | added | Aksakal | No method always works. Imagine equal weights | |
| Jan 13, 2021 at 12:54 | history | edited | GioMott | CC BY-SA 4.0 |
added 103 characters in body
|
| Jan 13, 2021 at 12:25 | comment | added | GioMott | @Aksakal: you both make interesting points. Dependence between measurements would be less of a concern in a practical case (at least from my experience), but fat-tailed errors could be an issue and indeed could change the result. Sorting likely gives the correct solution on average (here, a proof would also be very interesting) but I am more interested in a method that always gives the right solution. | |
| Jan 13, 2021 at 12:21 | history | edited | GioMott | CC BY-SA 4.0 |
added 281 characters in body
|
| Jan 13, 2021 at 12:19 | comment | added | GioMott | @whuber: you are right, I was uncertain about the terminology, but in the end I opted for the ISO standard (see: en.wikipedia.org/wiki/Accuracy_and_precision). I am curious about your suggestion about computing the likelihood of every permutation; please see the edit to my question. Certainly finding an efficient algorithm is desirable, but this point is rather moot given your proof that sorting always gives the correct solution. I am now studying your proof, which is a bit beyond my capabilities | |
| Jan 13, 2021 at 8:28 | history | became hot network question | |||
| Jan 13, 2021 at 1:26 | comment | added | Aksakal | @fblundun, can you prove that sorting doesn't work for fat tailed distributions in average. yes, you can always make up a sample where it doesn't, but on average it should work | |
| Jan 13, 2021 at 1:24 | comment | added | Aksakal | here's a conjecture: sorting doesn't work only in presence of dependence between measurements. prove it | |
| Jan 13, 2021 at 0:00 | history | tweeted | twitter.com/StackStats/status/1349144127666061313 | ||
| Jan 12, 2021 at 22:58 | answer | added | whuber♦ | timeline score: 16 | |
| Jan 12, 2021 at 22:56 | comment | added | fblundun |
If the measurement error isn't normally distributed, just sorting might not give the best answer. For example, suppose that the error distribution is fat-tailed so that there are occasionally huge errors. Then if Joe measures 1,52,53,54,55 and you measure 52,53,54,55,100 then Joe's 1 most likely corresponds to your 100.
|
|
| Jan 12, 2021 at 22:29 | answer | added | BruceET | timeline score: 5 | |
| Jan 12, 2021 at 21:31 | review | First posts | |||
| Jan 13, 2021 at 1:28 | |||||
| Jan 12, 2021 at 21:31 | comment | added | Aksakal | sort both sets. done. this is the most likely correspondence. | |
| Jan 12, 2021 at 21:27 | comment | added | whuber♦ | +1 For the record, "trueness" is usually termed "accuracy." As to your question: the obvious solution is to compute the likelihood of every permutation and select a permutation having the largest likelihood. This can take some time, since there are $n!$ permutations to examine. By the phrase "how would one go to find" are you perhaps trying to ask whether there is an efficient algorithm? (I believe there is: make the measurements correspond in rank order. That's a $O(n\log(n))$ algorithm) | |
| Jan 12, 2021 at 21:23 | history | asked | GioMott | CC BY-SA 4.0 |