What is the difference between learning and inference? Machine learning research papers often treat learning and inference as two separate tasks, but it is not quite clear to me what the distinction is. In this book for example they use Bayesian statistics for both kinds of tasks, but do not provide a motivation for that distinction. I have several vague ideas what it could be about, but I would like to see a solid definition and perhaps also rebuttals or extensions of my ideas:


*

*The difference between inferring the values of latent variables for a certain data point, and learning a suitable model for the data.

*The difference between extracting variances (inference) and learning the invariances so as to be able to extract variances (by learning the dynamics of the input space/process/world).

*The neuroscientific analogy might be short-term potentiation/depression (memory traces) vs long-term potentiation/depression.

 A: This looks like classic cross-discipline lingo confusion. The OP seems to be using neuroscience-like terminology where the two terms in question may have different connotations. But since Cross Validated generally deals with statistics and maching learning, I'll try answering the question based on the common usage of these terms in those fields.
In classical statistics, inference is simply the act of taking what you know about a sample and making a mathematical statement about the population from which it is (hopefully) representative. From the canonical textbook of Casella & Berger (2002): "The subject of probability theory is the foundation upon which all of statistics is built ... through these models, statisticians are able to draw inferences about populations, inferences based on examination of only a part of the whole". So in statistics, inference is specifically related to p-values, test statistics, and sampling distributions, etc.
As for learning, I think this table from Wasserman's All of Statistics (2003) might be helpful:

A: I agree with Neil G's answer, but perhaps this alternative phrasing also helps:
Consider the setting of a simple Gaussian mixture model. Here we can think of the model parameters as the set of Gaussian components of the mixture model (each of their means and variances, and each one's weight in the mixture).
Given a set of model parameters, inference is the problem of identifying which component was likely to have generated a single given example, usually in the form of a "responsibility" for each component. Here, the latent variables are just the single identifier for which component generated the given vector, and we are inferring which component that was likely to have been. (In this case, inference is simple, though in more complex models it becomes quite complicated.)
Learning is the process of, given a set of samples from the model, identifying the model parameters (or a distribution over model parameters) that best fit the data given: choosing the Gaussians' means, variances, and weightings.
The Expectation-Maximization learning algorithm can be thought of as performing inference for the training set, then learning the best parameters given that inference, then repeating. Inference is often used in the learning process in this way, but it is also of independent interest, e.g. to choose which component generated a given data point in a Gaussian mixture model, to decide on the most likely hidden state in a hidden Markov model, to impute missing values in a more general graphical model, ....
A: Inference is choosing a configuration based on a single input.  Learning is choosing parameters based on some training examples.
In the energy-based model framework (a way of looking at nearly all machine learning architectures), inference chooses a configuration to minimize an energy function while holding the parameters fixed; learning chooses the parameters to minimize the loss function.
As conjugateprior points out, other people use different terminology for the same thing.  For example Bishop, uses "inference" and "decision" to mean learning and inference respectively.  Causal inference means learning.  But whichever terms you decide on, these two concepts are distinct.
The neurological analogy is a pattern of firing neurons is a configuration; a set of link strengths are the parameters.
A: It is strange no one else mentioned this, but you can have inference only in cases where you have a probability distribution. Here to quote Wiki, which quotes Oxford dictionary:
Statistical inference is the process of using data analysis to deduce properties of an underlying probability distribution (Oxford Dictionary of Statistics)
https://en.wikipedia.org/wiki/Statistical_inference
In case of traditional neural networks, k-NN or vanilla SVMs you have no probability density to estimate, nor assumptions about any density, thus, no statistical inference there. Only training/learning. However, for most (all?) statistical procedures, you can use both inference AND learning, since these procedures possess some assumptions about the distribution of population in question.  
