Explanation on a Minsky's critique on statistical learning related to XOR I was listening to the first session of society of Minds by Minsky (2011) and he mentions at some point around minute 48 the following:
"...lots of statistical learning tools is good for lots of applications, but they won't cut it to solve hard problems, where the hypothesis more complicated than seven or eight variables interaction, so most statistical learning people assume that if you get a lot of partial ones then you can look at combinations of ones that have high correlations with the result, you can start combining them and then get better and better, however mathematically if the effect you are looking for depends on exclusive or of several variables, then there is no way to approach that by successive approximations if any one of the variables is missing, any correlation of the phenomenon with the others, anyways that is a long story but I think it is worth complaining about...."
Could somebody please explain me more about this phenomenon or guide me to a reference that I could read about it?
 A: Minksy is famous for criticizing neural networks for their inability to solve the XOR problem. It's possible that is what he's referring to here. Linear statistical relationships are not enough to detect patterns that resemble the XOR function.
http://www.ucs.louisiana.edu/~isb9112/dept/phil341/histconn.html 
A: The point he is making here is that you cannot learn (Ie see a few examples and generalise to the rest) the xor problem  with a (multilayer) Neural network or other stat learning algorithm that takes as input a fixed set of dimensions. With such a statistical learning tool, the best you can do is memorise all responses (since there is no similarity between the examples in terms of the vector space).
The fundamental Problem is that xor is a serial operation (like eg counting,  finding maximum etc) and you cannot generalise assuming a pattern represented as a point in a fixed n dimensional space - since each dimension is implicitly distinct [eg one dimension represents #apples, one dimension represents #oranges etc]. So learning xor for apples and oranges doesn't help you learn xor for bananas and strawberries.
Similar issues arise with translation invariance and rotation invariance in image processing (convolutional networks get around this by hard coding translation invariance into the network).
If you have a statistical learning tool that instead takes  input  in a serial fashion, such as a recurrent Neural net, then XOR and similar problems could potentially be learnt. [ eg you have 1 dimensional input x and previous output y , then you can learn xor of a sequence length n = xor (xor sequence n-1, x)] and similarly the maximum of a sequence ...
A: Well, I don't know about neural nets or what's current in "machine learning", but, assuming he's complaining about the set of statistical algorithms for these cases (which he might not be), it's just wrong. 
In particular, when dependent random variables are modeled, you can, of course, model XOR relationships.  It's not the only example, but consider the multivariate normal (or "Gaussian") with an unknown covariance matrix.  It is possible to estimate these, with sufficient data, and this can be done in a Bayesian framework.  See, for example:
Gaskins, Daniels, "A nonparametric prior for simultaneous covariance estimation", Biometrika, (2013), 100, 1, pp. 125–138, http://dx.doi.org/10.1093/biomet/ass060
Congdon, Bayesian Statistical Modelling, 2nd edition, 2006, Example 5.7, page 177.
Section 4.3.4.2, "Hole Models", pages 148-149, Schabenberger, Gotway, Statistical Methods for Spatial Data Analysis, 2005.
