Neuron vs. unit in a neural network In the context of machine learning, is there any difference between the terms unit  and neuron? I have read that some people prefer to use the term unit as neurons in an ANN have little in common with neurons in the human brain (see quote below). Is there any other difference?

Quote from Michael A. Nielsen, "Neural Networks and Deep Learning", Determination Press, 2015 (Creative Commons Attribution-NonCommercial 3.0 Unported License.)

The origins of convolutional neural networks go back to the 1970s. But the seminal paper establishing the modern subject of convolutional networks was a 1998 paper, "Gradient-based learning applied to document recognition", by Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. LeCun has since made an interesting remark on the terminology for convolutional nets: "The [biological] neural inspiration in models like convolutional nets is very tenuous. That's why I call them 'convolutional nets' not 'convolutional neural nets', and why we call the nodes 'units' and not 'neurons' ". Despite this remark, convolutional nets use many of the same ideas as the neural networks we've studied up to now: ideas such as backpropagation, gradient descent, regularization, non-linear activation functions, and so on. And so we will follow common practice, and consider them a type of neural network. I will use the terms "convolutional neural network" and "convolutional net(work)" interchangeably. I will also use the terms "[artificial] neuron" and "unit" interchangeably

 A: Let me suggest one scenario (the only one I can think of) where it might be useful to distinguish between "units" (or some similarly generic term) and "neurons." Biologically, a neuron is easy to identify, because it represents a single cell. In terms of neural nets, a neuron or "unit" has typically represented a single object, usually with one activation value, plus an additional threshold or separate input and output values in some cases. Problems arise in distinguishing between a neuron and a "unit" when we take into account the fact that the inputs, outputs, activations and thresholds of biological neurons are often mediated by multiple neurotransmitters and specific subsets of connections on the dendrites - many of which can be modeled as separate units. Then the line between "neuron" and "unit" blurs quickly. As William F. Allman puts it in pp. 65-66, Apprentices of Wonder: Inside the Neural Network Revolution (1989, Bantam Books: New York): 

"An axon may release various amounts of transmitter; a receiving
  dendrite might have varying amounts of receptor; the transmitter
  itself may have different checmical properties and react at different
  rates. And the whole process may be mitigated by the action of various
  enzymes.”

Here's a more thorough treatment from Daniel Gardner (1993, The Neurobiology of Neural Networks, MIT Press: Cambridge, Mass.) (I lost the page number to this, so I can't provide an exact citation):

" First, it has become evident that neurons (both in vertebrates and
  invertebrates) possess rich and complex intrinsic properties. Most
  neurons have multiple channels to different ionic species, and these
  channels can be regulated in a wide variety of ways: They can be
  turned on or off by voltage, molecules, or ions. Some of these
  channels can be active in the absence of external inputs to the cell,
  and endow it with a variety of dynamic properties, such as the ability
  to oscillate (Llinas 1988; Selverston 1988; Yamada et al. 1989). Thus,
  it is not enough to specify the inputs to a neuron to predict its
  outputs; its internal state will also determine its behavior. As a
  consequence, neurons may be better represented as nonlinear dynamic
  systems in their own right. For example, the intrinsic conductances of
  thalamic neurons can allow them to act as linear input/output devices,
  relaying information directly to cortex, but when they are
  hyperpolarized, these conductances cause the neurons to burst,
  significantly transforming their inputs (Llinas 1988). In terms of the
  model neurons that have often been used in artificial neural networks,
  the input/output relationship would need to be represented as a
  function both of voltage and of time.     "Second, the interactions
  between neurons are complex. The differential distribution of synapses
  on complex dendritic trees of neurons can significantly affect the
  nature and intensity of their inputs to a neuron. In addition,
  synapses may have multiple time courses (e.g., initial excitation,
  slower inhibition, and still slower excitation [Getting and Dekin
  1985), and connections may be dynamically reconfigured (e.g., by
  inhibition of specific neurons [Getting and Dekin 1985, or by the
  actions of neuromodulators [Harris-Warrick and Marder 1991; Marder and
  Hooper 19851). Receptors controlling the synaptic response may be
  gated both by the presence of a chemical, such as a neurotransmitter,
  and by voltage, so that the synaptic connections between neurons can
  be affected by their own activity and by the activity of neurons
  impinging on them (discussions of these and other complexities in
  synaptic interactions are found in chapters 2, 3, and 4). Influences
  may occur over a variety of spatial and temporal scales: A
  neuromodulator which is only slowly broken down may affect a very
  large number of neurons in its vicinity over an appreciable period of
  time as it diffuses away from its point of release. Furthermore, such
  compounds are likely to selectively activate those subgroups of
  neurons that have a receptor for that substance. Neuromodulators may
  also have subtle but profound effects on the intrinsic properties of
  neurons, activating or inactivating intrinsic currents and thus
  changing their "electrical personality." Field potentials may alter
  the excitability of neurons in different regions of the brain (Nunez
  1981)."

I've run across other such quotes in the literature with similar detail, but those two should get the point across (Gardner's book may be a good starting point if you want to look into the matter further). In cases where we're dealing with multiple activations, thresholds and the like, it might be helpful to make a distinction between "neurons" and constituent "units" that contribute their own activations and other calculations; there's such bewildering complexity to these matters that I don't think anyone can give a definitive answer as to the best way to model such distinctions. I ran into this problem when trying to implement Fukushima's neocognitrons, in which each neuron has its own separate inhibitory and stimulatory inputs; first I tried modeling them as separate neurons, then as a single neuron with multiple outputs, but I'm still not certain what the optimal choice is. There may be solid computational advantages to modeling many of these various enzymes, neurotransmitters and receptors beyond mere biological plausibility; perhaps there's not; the whole topic is still far afield, even for neuroscientists, who still have much to learn about the purposes of such connections. I suspect such questions will become far more complex and pressing in the future once the field of neuroscience advances, enabling neural net researchers to mimic more of these internal calculations. For the time being it's safe to equate neurons with "units," but that might not be the case once more sophisticated neural nets begin to make practical use of this dizzying array of computations.
A: 
In the context of machine learning, is there any difference between the terms unit and neuron?

They are the same, often called neural unit.
Neurons in an ANN is derived from the McCulloch-Pitts Neurons(MCP neuron), and a MCP neuron is a highly simplified model of a neuron in the human brain.

In 1943 Warren S. McCulloch, a neuroscientist, and Walter Pitts, a logician, published "A logical calculus of the ideas immanent in nervous activity" in the Bulletin of Mathematical Biophysics 5:115-133. In this paper McCulloch and Pitts tried to understand how the brain could produce highly complex patterns by using many basic cells that are connected together. These basic brain cells are called neurons, and McCulloch and Pitts gave a highly simplified model of a neuron in their paper. The McCulloch and Pitts model of a neuron, which we will call an MCP neuron for short, has made an important contribution to the development of artificial neural networks -- which model key features of biological neurons.


The original MCP Neurons had limitations. Additional features were added which allowed them to "learn." The next major development in neural networks was the concept of a perceptron which was introduced by Frank Rosenblatt in 1958. Essentially the perceptron is an MCP neuron where the inputs are first passed through some "preprocessors," which are called association units. These association units detect the presence of certain specific features in the inputs. In fact, as the name suggests, a perceptron was intended to be a pattern recognition device, and the association units correspond to feature or pattern detectors.

