I have seen this term come up a fair amount in machine learning. My guess would be that gold data is data which has been manually tagged, rather than learned by some process. However, I'm not too sure. So, what is the meaning of this phrase?
$\begingroup$
$\endgroup$
1
-
3$\begingroup$ Please post an example of the usage of the term. Never heard of it myself. $\endgroup$– FirebugCommented Aug 8, 2017 at 15:32
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
1
I think you're probably referring to gold standard data. This refers to data of very high quality, which is more or less as close as you can get to the ground truth. For example, Alzheimer's disease can be diagnosed through behavioral tests, but it's not a perfect diagnosis and can be confused with other types of dementia. A definitive diagnosis of Alzheimer's can be made by performing an autopsy on the brain, resulting in an unambiguous diagnosis about which there is no uncertainty. In this case, the autopsy diagnosis represents the gold standard test.
Gold standard data is great for machine learning tasks, since it is known to be of high quality, and avoids the "garbage in, garbage out" problem. If you want to build a model to predict Alzheimer's disease, you'd much rather have the brain autopsy data, since there will be virtually no mislabeled data. Gold standard data may be hard to come by, however, due to it being difficult or expensive to obtain. I'll also note that while gold standard data represents the highest possible data quality, it still may not be perfect.
answered Aug 8, 2017 at 15:39
-
$\begingroup$ If interested in learning more about how one can apply Machine Learning with non-gold data, consider looking into keywords like: Weak Supervision, Semi-supervised learning, Confident Learning, Crowdsourcing, Dawid-Skene $\endgroup$ Commented Jan 12, 2023 at 7:17
$\begingroup$
$\endgroup$
2
In a machine learning context, "gold" data usually refers to hand-labelled (and therefore) very high quality data.
Readers may wonder "isn't all training and test data hand-labelled and known to be correct?". (Ha ha ha) This is often not the case. ML techniques can be very data-hungry, and it's not uncommon to use all sorts of methods that trade off a little bit of data quality for a lot of data quantity. For example, if you're trying to replace an old production system with a new one, it makes sense to use the old one's predictions as training data to bootstrap the new system. Or you might use user-interaction signals to generate training data. Or perhaps you semi-automatically generate training data using some slow offline process over some enormous corpus with some minor curation of the results.
However, in all of these cases, it would be foolish to assume these predictions are error free. Hence, it's useful to distinguish your (expensive, almost certainly correct) gold data from all your other training/test data.
answered Nov 17, 2022 at 12:46
-
1$\begingroup$ Even hand-labeled data often contains many incorrect annotations (aka label errors) due to humans making occasional mistakes. When all data has been hand-labeled, the "gold" data may the subset of samples whose label has been additionally reviewed by an expert. $\endgroup$ Commented Jan 12, 2023 at 7:15
-
1$\begingroup$ @JonasMueller I have worked with teams where they had some "platinum" data sets, which basically meant it was annotated by product managers. 😂 $\endgroup$ Commented Jan 12, 2023 at 7:31
$\begingroup$
$\endgroup$
Gold data is a clean data set in 'lab' conditions. Lab condition data sets generally are free from mislabeling, duplications in the test set, unidentifiable data, and ambiguous classes. That is, gold data are manually selected which introduces human prior knowledge of the target. Most times such data sets are expensive or inaccessible.
Real-world data sets are generally comproised by all kinds of real-world variabilities. For example, overexposure, underexposure, motion blur, and off-focus images are common in real-world image data sets.