Covariance versus correlation: which is a "deeper" or more "structural" property of the data? It might seem obvious that the covariance is a "deeper" property of the data generation process (DGP), since normally the specification of a joint distribution is done in terms of its mean vector and covariance matrix. From these, the correlation structure can be deduced. In fact a correlation is mathematically defined as a function of the mean and covariance.
However, it is also possible to define a multivariate DGP in terms of means and correlations. In fact, correlations are interesting enough in that they are bounded between -1 and 1 and are comparable. Thus, it is possible to see them somewhat as a more "structural" property of the data. 
Is covariance, ontologically speaking (if this is possible in statistics), a more structural parameter than correlation? Or are maybe both equally structural?
 A: Neither are particularly deep. To really dig into the structure of the relationship between two random variables, you need to examine the full copula. In essence, a copula is what remains of the joint distribution between two random variables once their marginal probabilities have been flattened out to uniform. That such a factorization is always possible is due to Sklar's Theorem.
Here is a scatter plot of a real-world data set that has an approximately Gaussian copula. Gaussian copulas represent a fairly simple relationship structure that can be understood purely i terms of the correlation. (To speak to  your specific question, yes, correlation is sometimes a good way to understand the relationship between two random variables - when the copula is approximately Gaussian.)

However, many other relationships are possible. From the same real-world project, we also structures like this:

or this:

That last one is getting a pretty similar $r^2$ as the first, yet their structure is completely different. Perhaps It's an asymmetrical Gumbel copula, or something other asymmetric copula.
There are infinitely many possible copulas, but these are studied in terms of a finite number of parameterized families. At least two (very important) copulas can be parameterized in terms of their correlation matrices: the Gaussian and the Student's $t$ copulas. But other copulas have other parameters and therefore cannot be understood in terms of correlation or covariance.
So, returning to your question, if you believe that the joint probability distribution between the $k$ random variables in question is well-approximated by a Gaussian or Student's $t$ copula, then yes, correlation is a powerful and sufficient way to understand their structure. But if the copula is from some other family, or unknown, or we're trying to make very general statements about all possible joint probability distributions, then correlation is woefully inadequate.
