Neural network not i.i.d Is it not considered correct to use non i.i.d samples in neural networks, for example if I have pixels in an image as observations?
I have read that this is an assumption, but what if this is not the case and the samples are non i.i.d and I want to use the multilayer perceptron, can I do that?
The model works fine with non i.i.d samples, so it this not a strong requirement?
 A: There are several ways to have independency assumptions in neural nets. One is that all your samples are independent, i.e. if you have a data base of 10'000 cat pictures, you assume they have all be taken independently of each other. 
Another is if you want to regress on certain values. Say you want to regress from the cat picture on its size and its weight. If you just minimize the sum of squares of your predictions for both values, you introduce the assumption that weight and size are independent–which is not the case.

The first case can be mathematically seen that your objective functions typically are sums of terms, one for each sample. 
$$
\mathcal{L}(\theta) = \sum_i \ell_i
$$
The sum is nothing but a product in the log domain after observing that $\log (a \cdot b) = \log a + \log b$.
$$
\mathcal{L}(\theta) = \log \prod_i \exp(\ell_i),
$$
which implies that the data samples factorize–they are independent.
The definition of independence: $p(a)p(b) = p(a, b) \Leftrightarrow \text{a is independend of b}$.
For the second case, note that a sum of squares is the log density of a Normal. If you have a large sum over such, this also implies a product over many Normally distributed random variables, and hence implies an independency assumption.
A: Without aiming for the Math: using non independent and/or unequally distributed variables is possible with ANN. You might trigger some side effects by doing so, like with unequally distributed variables having an overly long training phase, getting stuck in (other) local optima (though this is less of an issue in the application case), or not obtaining the optimal error during minimization from a Math point of view. But technically, it is definitely not a hard requirement for input variables of ANN to fulfil those properties - ANNs work fine with such too. Think e.g. of all the deep learning approaches, where people tend to (more or less) throw in whatever information they get hold of, mostly without sophisticated preprocessing.
