I searched on Google, Wikipedia, Google scholar, and more, but I could not find the origin of Autoencoders. Perhaps it's one of those concepts that evolved very gradually, and it's impossible to trace back a clear starting point, but still I would like to find some kind of summary of the main steps of their development.

The chapter about autoencoders in Ian Goodfellow, Yoshua Bengio and Aaron Courville's Deep Learning book says:

The idea of autoencoders has been part of the historical landscape of neuralnetworks for decades (LeCun, 1987; Bourlard and Kamp, 1988; Hinton and Zemel,1994). Traditionally, autoencoders were used for dimensionality reduction or feature learning.

This presentation by Pascal Vincent says:

Denoising using classical autoencoders was actually introduced much earlier (LeCun, 1987; Gallinari et al., 1987), as an alternative to Hopfield networks (Hopfield, 1982).

This seems to imply that "classical autoencoders" existed before that: LeCun and Gallinari used them but did not invent them. I see no trace of "classical autoencoders" earlier than 1987.

Any ideas?


3 Answers 3


According to the history provided in Schmidhuber, "Deep learning in neural networks: an overview," Neural Networks (2015), auto-encoders were proposed as a method for unsupervised pre-training in Ballard, "Modular learning in neural networks," Proceedings AAAI (1987). It's not clear if that's the first time auto-encoders were used, however; it's just the first time that they were used for the purpose of pre-training ANNs.

As the introduction to the Schmidhuber article makes clear, it's somewhat difficult to attribute all of the ideas used in ANNs because the literature is diverse and terminology has evolved over time.

  • 1
    $\begingroup$ The paper written by Ballard , has completely different terminologies , and there is not even a sniff of the Autoencoder concept in its entirety. Maybe AE does not have any origins paper. $\endgroup$ Sep 21, 2018 at 10:45
  • 2
    $\begingroup$ The Schmudhuber paper emphasizes that the terminology has changed over time and different people have rediscovered the same topics over and over. It’s not surprising that the author doesn’t use the word “auto encoder” $\endgroup$
    – Sycorax
    Sep 21, 2018 at 13:11

The paper below talks about autoencoder indirectly and dates back to 1986.(which is a year earlier than the paper by Ballard in 1987)

D.E. Rumelhart, G.E. Hinton, and R.J. Williams, "Learning internal representations by error propagation." , Parallel Distributed Processing. Vol 1: Foundations. MIT Press, Cambridge, MA, 1986.

The paper basically describes a novel kind of feedforward network at that time , and its mathematical formalism.


Reviving this thread - In "Neurocomputing" by Robert Hecht-Nielsen @ 1990 there is reference to a 1986 paper by Cottrell/Munro/Zipser that outlines use of a neural network that has the architecture of an autoencoder, and is trained on the identity function, for compression and reconstruction of image data. The term "autoencoder" isn't mentioned, but that's what it is. There is no mention of using it for anomaly detection either. Seems like that usage was discovered later.

  • $\begingroup$ It's best to include the full citations of the papers you're referencing so that others can locate them. Is this the paper? Cottrell, G. W., P. Munro, and D. Zipser. "Image compression by back-propagation." ICS report 8702 (1987). Or does it cite another paper? Please edit to clarify. $\endgroup$
    – Sycorax
    May 17 at 14:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.