Consider using Multi-variate Gaussian to approximate $X = [X_1, X_2, ..., X_n]$ and $X_i = [x_{i1}, x_{i2}, x_{i3}, ..., x_{im}]$, so we have n data points and each data point has m features.
Multi-variate Gaussian distribution has a covariance matrix of m by m, does that mean the features has collinearity among them?
Exact collinearity means that one feature is a linear combination of others. Covariance is bilinear; therefore, if $X_2 = a X_1$ (where $a\in \mathbb R$),
$$\mathrm{cov}(X_1, X_2) = a\ \mathrm{cov} (X_1, X_1) = a. $$
Likewise if $X_n$ is some more complicated linear combination of $X_1, \cdots, X_{n-1}$ with coefficients $a_1, \cdots$,
$$\mathrm{cov}(X_i, X_n) = \sum_{j=1,\cdots,n} a_j\ \mathrm{cov}(X_i, X_j) . $$
Since the covariance matrix $\Sigma$ has as as its $i$-th row $\Sigma_{i.}$ the vector $[\mathrm{cov}(X_i,X_1),\cdots,\mathrm{cov}(X_i,X_n)]$, this means the entire $n$-th row will be a linear combination of the previous rows and the covariance matrix is rank-deficient.
By definition, two collinear varaibles are perfectly linearly correlated. All variables that have a correlation of +1 or -1 are collinear. Correlation measures the "linear" joint relationship in the sense of Euclidean (linear) distance, the L^2 norm. So you can use the covariance matrix to find correlations of +/- 1, and those are the collinear variables.
Your best bet would be to obtain the correlation matrix. To do this, standardize the features (subtract the mean and divide by the standard deviation) and then obtain the covariance matrix of the standardized features.
You can now easily examine feature correlation by looking at the off-diagonal elements (as the main diagonal elements will all be 1). If you are interested in the existence of a linear relationship, you can look for values close to 1 or -1. You may wish to verify statistical significance of the correlations of interest before making any claims about there being a linear relationship.
I hope this answers your question.
There are several different collinearity: Collinearity indicates the relationship between two variables. Multi-collinearity means the relationship among more than 2 variables. Full collinearity means at least one variable can be expressed as the linear combination of other variables. We say partial collinearity when the variables are not orthogonal and have no full collinearity relation.
Based on m x m covariance matrix, we can check what kind of collinearity exists among these variables. We know that diagonal elements of covaraince matrix ($\Sigma$) is variance of corresponding random variables, basically they have no information on collinearity, excerpt 0 appears. It is easy to convert $\Sigma$ into correlation matrix $R$. If all of the off-diagonal elements of $\Sigma$ are 0, the random variables are independent or the variables are orthogonal. If there are 1 or -1 in off-diagonal elements in $R$, the corresponding two variables have full collinearity. If the off-diagonal element of $R$ is not -1, 0 and 1, the partial collinearity exist between correspondent variables. If full or partial collinearity exits between any pair of variables, the full multi-collinearity or partial collinearity exists. But no full collinearity does not imply no full multi-collinearity. The full multi-collinearity depends on the rank of $R$. if $R$ is less than full rank, the full multi-collinearity exists. Otherwise, there is no full multi-collinearity.
