Timeline for What happen if I choose the hyperparameters of a classifier based on lowest generalization error?
Current License: CC BY-SA 4.0
3 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jun 15, 2019 at 13:11 | comment | added | ReneBt | The independence of the data helps bolster the chance that fewer biases will leak across as much as if they were simple subsets of one data collection. However, since it is being used to direct choices that influence the settings available to the model, there will be leakages from the test set to the model as described above. Without a further set for a one look confirmation you can't get a useful estimate of how much influence it has. | |
| Jun 15, 2019 at 12:37 | comment | added | Mithridates the Great | Thanks for your answer. I have to say my test dataset is completely independent from my training dataset. In fact, I could say even the test dataset is given by another research group but it is for the same phenomenon as my training dataset. So, in summary: 1. My test data is independent from my training data, 2. It’s not like training a model on East Asia data and expect to generalize it for North America, it’s exactly like training a model on East Asia data, that is extracted by me, and generalize on East Asia data extracted by another research group. So, now what’s your opinion then? | |
| Jun 15, 2019 at 2:09 | history | answered | ReneBt | CC BY-SA 4.0 |