What is the difference between Cross Correlation and Mutual Information. What kind of problems can be solved using these measures and when is it appropriate to use one over the other.
Thanks for the comments. To clarify, the question is prompted by an interest in image analysis rather than time series analysis although any enlightenment in that area would also be appreciated
Cross correlation assumes a linear relationship between 2 sets of data. Whereas mutual information only assumes that one value of one dataset says something about the value of the other dataset.
So mutual information makes much weaker assumptions.
A traditional problem solved with mutual information is aligning (registration) of two types of medical images, for example an ultrasound and a x-ray image. (typically, the types of images are called modalities, so the problem is named multi-modal image registration).
For both X-ray and ultrasound, a specific material, say bone, leads to a certain 'brightness' in the image. Whereas some materials lead to a bright x-ray and ultrasound image, for other materials (e.g. fat) it might be the opposite, one is bright, the other is dark. Therefore, it is not the case that bright parts of the X-ray image are also bright parts of the ultrasound.
Therefore, mutual information is still a useful criterion for aligning the images, but cross correlation is not.
Cross correlation is used in time-frequency analysis and is a inner product with a lag-parameter obtained between two functions varying over time, where one function is evaluated at time $t$ and the other is evaluated at time $lag+t$. The cross-correlation theorem relates cross-correlation to the fourier transform of the individual functions, and hence a cros-correlation evaluated in a time domain is connected by this theorem to the spectral properties/frequency domain of the individual functions. Analogues to this exist in other areas like analyzing spatial data for example.
