Timeline for Which is a better baseline - Probabilistic Assignment vs Deterministic Assignment
Current License: CC BY-SA 3.0
17 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 11, 2020 at 23:35 | vote | accept | The Wanderer | ||
| Oct 9, 2020 at 20:00 | history | bumped | CommunityBot | This question has answers that may be good or bad; the system has marked it active so that they can be reviewed. | |
| Dec 27, 2017 at 5:31 | answer | added | robsmith11 | timeline score: 2 | |
| Dec 27, 2017 at 5:21 | history | edited | The Wanderer | CC BY-SA 3.0 |
Reworded the problem to explain more clearly what I have asked
|
| Dec 27, 2017 at 5:20 | comment | added | Matthew Drury | I think @Kodiologist had a good answer to a similar question, but I can't seem to find it at the moment. | |
| Dec 27, 2017 at 5:18 | history | edited | The Wanderer | CC BY-SA 3.0 |
Reworded the problem to explain more clearly what I have asked
|
| Dec 26, 2017 at 6:26 | review | Close votes | |||
| Dec 26, 2017 at 12:10 | |||||
| Dec 25, 2017 at 23:51 | history | edited | The Wanderer | CC BY-SA 3.0 |
added 350 characters in body
|
| Dec 25, 2017 at 23:46 | comment | added | The Wanderer | @MatthewDrury: I completely understand your point of view. At this stage, let me clarify that I am looking for Baseline methods. Those are really simple stupid methods that are used to compare "fancy" models. My actual "fancy method" uses a HMM with EM to generate the probability of cancerous genes. | |
| Dec 25, 2017 at 23:42 | comment | added | Matthew Drury | In many situations it's an oversimplification to do that. How are you going to act if you classify as "have cancer", are you going to immediately order chemotherapy? Probably not on the basis of a prediction from one model. It would be better to be more precise about your classification, which is almost certainly "needs further tests". This should advise how you want to balance the precision and recall of our model. To do so, you want to find a threshold for your probability predictions that balances precision and recall given your use case. | |
| Dec 25, 2017 at 22:48 | comment | added | The Wanderer | @MatthewDrury: In the end, I have to do a hard assignment. I can say that the probability of cancer is xyz but I have to classify it to have_cancer or no_cancer. | |
| Dec 25, 2017 at 22:46 | comment | added | The Wanderer | @MatthewDrury: Evaluation metric is not the main concern here. I am fine using Precision and Recall as well. My concern is, how do I measure that for the case of the probabilistic assignment? | |
| Dec 25, 2017 at 22:03 | comment | added | Matthew Drury | Is there a reason you need to hard assign each case to cancer or not cancer? Often a better idea is to use a proper scoring rule loss function like log-loss, which measures how well the probability agrees with the class. It is really only in rare cases where accuracy is a reasonable evaluation metric. | |
| Dec 25, 2017 at 20:30 | comment | added | The Wanderer | @EdM: Let me also briefly introduce the concept of baselines. Baselines are meant to be some really simple way of accomplishing the task. One generally compares "fancy methods" with the performance of baseline methods. | |
| Dec 25, 2017 at 20:27 | comment | added | The Wanderer | @EdM: Generally there is a concept of training set and test set. Also one assumes that both training and test set follow the same distribution. P(cancer) = 0.6 was determined by looking at the training set and then for every case you see in the test set, you have to determine whether it is a case of cancer or not. So my "assignment" is for the cases in the test set. | |
| Dec 25, 2017 at 19:58 | comment | added | EdM | You'll have to say a bit more about what you are trying to accomplish with this "assignment." If you already know which cases have cancer, it's not clear that there's any need for "assignment." | |
| Dec 25, 2017 at 19:48 | history | asked | The Wanderer | CC BY-SA 3.0 |