ICA is quite popular for analyzing brain images (e.g. group ICA). One common assumption/constraint is that the signals in the brain come from "independent spatial sources".
I'm confused about how to express the "independence of spatial sources" mathematically. If the spatial components of brain sources are denoted as long vectors $x_i$ $\forall i$. If two brain spatial sources are independent, what is the mathematical relationship between $x_1$ and $x_2$ then?
I am not familiar with brain images analysis but I am still going to try to help you here. If each of your independent spatial components are denoted as long vectors $x_i$ $\forall i$, then each of your original brain images can be expressed as a linear combination of your independent spatial components.
The fact that your spatial components $x_1$ and $x_2$ are independent means that you can't predict the value taken by $x_1$ at one pixel based on the value taken by $x_2$ at the same pixel. Thus, $p(x_1,x_2) = p(x_1)p(x_2)$.
If you want to further understand the theoretical basis of ICA and how the different algorithms performing ICA such as infomax or FastICA work, I recommend you this review from the creators of the FastICA algorithm.
Tell me if anything is unclear.
Also, ICA is conventionnaly formulated as: $x = As$ where $s$ are your source variables (or independent components), $x$ your observed variables (e.g. your brain pictures), and $A$ the mixing matrix containing coefficients necessary for the above mentioned linear combination.
Independence is something that's hard to formulate mathematically. For example, un-correlatedness is not sufficient for independence (that's why PCA does not perform ICA).
Spatial ICA takes the following formulation:
$$X = A S$$
$X_{\tau\times\upsilon}$ is you data matrix of observed states. $A_{\tau\times\phi}$ is the mixing matrix, which translates the sources into the observed states. $S_{\phi\times\upsilon}$ is the source matrix of independent components. $\upsilon$ is the number of voxels, $\tau$ the number of timesteps and $\phi$ the number of components.
$A$ and $S$ are jointly optimized to reconstruct $X$ while maximizing the independence between the rows of $S$.
One way of defining independence is through non-Gaussianity. This comes from the central limit theorem: the aggregation (mixing) of independent signals is more Gaussian than any of its constituents (unless one of them is Gaussian). See this question What is mean by the non-gaussianity in the independent component analysis(ICA)?.
Two possible measures of non-Gaussinity are kurtosis and negentropy.
Standardized Kurtosis is simply defined as:
$$\kappa = \frac{\mathbb E [(s - \bar s)^4]}{\mathbb E^2 [(s - \bar s)^2]} - 3$$.
We subtract $3$ because Gaussian variables have kurtosis equal to three. We can maximize $\kappa$ through gradient descent, for example.
Negentropy is justified because, among continuous distributions with the same variance, the Gaussian distribution has maximal entropy. We thus try to maximize negative-entropy, which should ensure non-Gaussianity. Estimating negentropy of a signal is not easy, but approximations were proposed, such as:
$$J(x)=\frac{1}{12}(\mathbb E(x^3))^2 - \frac{1}{48}(\kappa(x))^2$$
Which, again, is a differentiable function.
