Timeline for Model performance when ground truth is not available
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 30 at 16:54 | comment | added | Ggjj11 | I want to add: even for clustering you should consider external evaluation metrics like rand index etc. Your clusters could otherwise be useless (some arbitrary pattern) | |
| May 9, 2022 at 1:46 | comment | added | learnlifelong | Hi @frank, I have a question regarding model improvement. Let's say I deployed an LSTM Autoencoder (unsupervised model) to detect anomalies in a time series dataset. After sometime I created a new model, on what basis can I compare the deployed and the new model to decide whether or not to deploy the new model? | |
| May 7, 2022 at 15:39 | vote | accept | learnlifelong | ||
| May 6, 2022 at 5:09 | comment | added | frank | @learnlifelong Pertaining: "The idea of those criteria...": my point is only that many of those criteria, e.g. the Davies–Bouldin index (DBI), can be used as clustering methods themselves by simply scoring all possible partitions. Since this doesn't scale, we settle for much faster algorithms, which might be much worse than DBI. But then we use DBI at least as a tie-breaker at the end. However, while DBI seems to be quite reasonable, it might still be inappropriate. | |
| May 6, 2022 at 4:57 | comment | added | frank | @learnlifelong I haven't read it fully either, but it seems the point is that you have to settle for some "master" criterion and there is no way to know whether it is correct. | |
| May 6, 2022 at 1:15 | comment | added | learnlifelong | Thanks @frank. Could you please elaborate on the stuff said in the 2 paragraphs: "In clustering..." and "The idea of those criteria..."? I could not understand it fully. I found a paper that evaluated various methodologies. arxiv.org/pdf/2104.01422.pdf I haven't read it in detail but it seems they are saying that there is no metric that can help pick the right methodolgies i.e. algorithm and hyperparameters | |
| May 5, 2022 at 7:51 | history | answered | frank | CC BY-SA 4.0 |