7
$\begingroup$

The confusion matrix is a fundamental term in machine learning. However, that name itself is a bit, well, confusing. I tried to trace its origin but I'm not sure I'm correct, and I'd like to hear what others know.

The term became popular in the ML community thanks to Kohavi and Provost at 1998.

However, already at 1971 Townsend wrote a paper discussing it. The paper discusses and experiment in which the 26 English alphabet letters (stimuli) are presented to a subject that should present reply with the same letter (reaction). The confusion is a 26 by 26 matrix with the probability of each reaction to each stimulus. This explains the name (the matrix of the subject confusion) and matches the use in machine learning today.

I found even earlier references to the term confusion matrix, starting at 1953 using Google books, but the text wasn't available. It seems that there are even earlier reference to the term in psychology since it is used as a known phrase.

Is the origin of confusion matrix indeed in psychology and due to such usage? What else is known about the term?

References

(Kohavi & Provost, 1998) ⇒ Ron Kohavi, and Foster Provost. (1998). "Glossary of Terms." In: Machine Learning 30(2-3).

(Townsend, 1971) ⇒ J. T. Townsend. (1971). "Theoretical analysis of an alphabetic confusion matrix." In: Attention, Perception, & Psychophysics, 9(1).

1955 Journal of applied psychology, Volume 39

American Psychological Association, 1955

Реферативный журнал: Математика, Volume 2

Изд-во Академии наук СССР., 1953 - Mathematics

$\endgroup$
1

4 Answers 4

7
$\begingroup$

The question as I understand is not "where did the grid/matrix come from?" but rather "why is the term 'confusion' used?" I think referring back to Pearson is not likely to help.

My sense, having learned the term in psychology, is that "confusion" refers to the intuition that an item can be correctly or incorrectly labeled/identified/detected and so on (i.e,. "confusion of labels"). Following this logic, I've understood the word "confusion" in this case to loosely refer to the idea: did our classifier (or in the case of psychology, the observer) correctly identify the item, or "was it CONFUSED with another label"? That's my best guess!

$\endgroup$
5
  • 2
    $\begingroup$ Do you have any sources which would support your answer? $\endgroup$
    – Silverfish
    Jun 23, 2016 at 22:41
  • 1
    $\begingroup$ Good question...let me research it a bit. My use of the word "intuition" was meant to imply that I'm relying on 20+ years of experience in academia, but if I can support my hunch with objective sources that would be awesome! $\endgroup$
    – Matthew
    Jun 24, 2016 at 16:53
  • $\begingroup$ OK, here's a quick and dirty possibility (emphasis mine): oxfordreference.com/view/10.1093/acref/9780199534067.001.0001/… $\endgroup$
    – Matthew
    Jun 24, 2016 at 17:01
  • $\begingroup$ "A matrix representing the relative frequencies with which each of a number of stimuli is mistaken for each of the others by a person in a task requiring recognition or identification of stimuli. Analysis of these data allows a researcher to extract factors (2) indicating the underlying dimensions of similarity in the perception of the respondent. For example, in colour-identification tasks, relatively frequent confusion of reds with greens would tend to suggest daltonism." $\endgroup$
    – Matthew
    Jun 24, 2016 at 21:57
  • $\begingroup$ The question is indeed about the term. As you can see in the link in both the question & the other answer, there are references to using the term in psychology from the fifties. The "confusion" in "confusion matrix" is indeed the one of the human classifier. $\endgroup$
    – DaL
    Jun 26, 2016 at 5:56
5
$\begingroup$

I think I understand the history of the term, thanks to an answer here: What is the origin of the term confusion matrix?

The confusion matrix was invented at 1904 by Karl Pearson. He used the term Contingency Table. It appeared at Karl Pearson, F.R.S. (1904). Mathematical contributions to the theory of evolution (PDF). Dulau and Co..

During War World 2, Detection Theory was developed as investigation of the relations between stimulus and responds. The confusion matrix was used there.

Due to detection theory, the term was used in psychology. From there the term reached machine learning.

It seems that though the concept was invented in statistics, a field very related to machine learning, it reached machine learning after a detour in during a period of 100 years.

$\endgroup$
3
$\begingroup$

What is odd to many veterans of data science is that what is now often called a confusion matrix has been called a classification matrix in the past. Both have the same structure: predicted outcomes in the columns and actual outcomes in the rows. I've always found the term classification matrix to be less confusing than confusion matrix, and I don't see any advantages to using the newer term.

$\endgroup$
1
$\begingroup$

It was called a "confusion matrix" by Miller and Nicely in 1955. I don't know if this is the origin of the term "confusion matrix," but it might be.

G.A. Miller and P.E. Nicely, "Analysis of Perceptual Confusions Among Some English Consonants," Journal of the Acoustical Society of America 27:338-352, 1955

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.