how to do decorrelation and covariance equalisation? In order to improve the convergence of a MLP, one should do "mean removal", "decorrelation" and "covariance equalization".
I know how to do mean removal, but how do you do decorrelation  and covariance equalization ?  I didn't find any formulas on that online.
I don't even understand what decorrelation/covariance equalization is doing intuitively. In my view, equalising the covariance between every pair of inputs is basically impossible. For example, the covariance between "interest on German government bonds" and "interest on loans to German companies" is positive while the covariance between "interest rate changes" and "average stock prices" is negative. How do you want to make them equal?    
 A: You can consider different options for your aims. 


*

*Principal Component Analysis (PCA). Given a set of predictors, or inputs, you can extract, using data orthogonalization, a set of components which are independent by construction. Each predictor or input will be represented in each component in a different way, depending on original data structure. You can use MatLab's pca() or princomp() to do the job. I recommend you to visualize data using biplot.

*Deconvolution is used to reverse the effect of a convolution. Suppose to record the voice of a singer, in a bad studio. You have a signal s that contains not only the singer's voice, but also noise, originated from the poor quality of the microphone, and the cables. So you have v + n = s, where v is the voice and n is the noise. You can deconvolve white (wn) noise from s, trying to clean the original voice: s2 = s/wn; hopefully, s2 will be more similar to v than s. You can look at MatLab's function deconv() for this.

*Other transformations, like the Mahalnobis transformation, seems to be useful in some cases (maybe you should take a look here http://www.davidsalomon.name/DC2advertis/DeCorr.pdf).


Clearly, the approach to be adopted strongly depends by the data, and the experiment. Typically, PCA is the option you choose when you need, for example, to do a GLM in which you enter multiple variables. You must be sure that the predictors are uncorrelated, to avoid multicolinearity issues. Deconvolution, instead, is useful when you have signals over time, which need to be 'decorrelated' from another source / another signal.
Hope this helps.
Simone
