Correlation between linear regression coefficients Can we get correlation between linear regression coefficients?
I have real data (time series) with different realizations representing the same phenomenon: $\{x_j(t)\} = x_1(t), \dots, x_N(t)$. I model each one with a linear regression model: $f_j(t)=\displaystyle \sum_{i=0}^dc_{ij}\phi_i(t)$, $(j = 1, \dots, N$). $\phi_i(t)$ are Legendre polynomials and are orthogonal.
After estimating the $c_{ij}$ I found correlations between these coefficients wrt $j$, i.e. if I plot (see fig. below) the different $c_{ij}$ function of $j$ I find that $c_{mj}$ and $c_{lj}$, for example, are linear combinations of each other. This happens for all coefficients. Is this normal? What does it mean?

 A: Correlation between linear regression coefficients can occur when terms in your model have very similar effects on the targeted output (in your case, this targeted output is $f$). For example, perhaps an increase in 20% in terms 1 and 3 both yield a 5% increase in $f$. 
In such a case, the fitting routine cannot determine which term is having the biggest effect on $f$, and so can end up confusing several terms. This leads to the coefficients of these terms being highly correlated.
The correlation between the coefficients of different Legendre polynomials may indicate that your behavior is better described by a simpler model.
Let us assume, for example, that your $t$ are very small, such that higher-order terms are negligible. Then we might expect that your Legendre polynomials will reduced to alternating 'linear' and 'constant' terms.
From Wikipedia, the first four Legendre polynomials are:
$1$     
$x$
$\frac{1}{2}(3x^2-1)$
$\frac{1}{2}(5x^3 - 3x)$
For small $x$, the first and third (and fifth, seventh, etc.) terms will reduce to a constant. The other terms will reduce to a constant times $x$. The fitter will not know how to differentiate between the multiple constants or multiple linear terms, and you will end up with very high correlations between your coefficients.
