Neither are particularly deep. To really dig into the structure of the relationship between two random variables, you need to examine the full copula. In essence, a copula is what remains of the joint distribution between two random variables once their marginal probabilities have been flattened out to uniform. That such a factorization is always possible is due to Sklar's Theorem.
Here is a scatter plot of a real-world data set that has an approximately Gaussian copula. Gaussian copulas represent a fairly simple relationship structure that can be understood purely i terms of the correlation. (To speak to your specific question, yes, correlation is sometimes a good way to understand the relationship between two random variables - when the copula is approximately Gaussian.)
However, many other relationships are possible. From the same real-world project, we also structures like this:
That last one is getting a pretty similar $r^2$ as the first, yet their structure is completely different. Perhaps It's an asymmetrical Gumbel copula, or something other asymmetric copula.
There are infinitely many possible copulas, but these are studied in terms of a finite number of parameterized families. At least two (very important) copulas can be parameterized in terms of their correlation matrices: the Gaussian and the Student's $t$ copulas. But other copulas have other parameters and therefore cannot be understood in terms of correlation or covariance.
So, returning to your question, if you believe that the joint probability distribution between the $k$ random variables in question is well-approximated by a Gaussian or Student's $t$ copula, then yes, correlation is a powerful and sufficient way to understand their structure. But if the copula is from some other family, or unknown, or we're trying to make very general statements about all possible joint probability distributions, then correlation is woefully inadequate.