What is the interpretation of the covariance of regression coefficients? The lm function in R can print out the estimated covariance of regression coefficients.  What does this information give us? Can we now interpret the model better or diagnose issues that might be present in the model?
 A: The most basic use of the covariance matrix is to obtain the standard errors of regression estimates. If the researcher is only interested in the standard errors of the individual regression parameters themselves, they can just take the square root of the diagonal to get the individual standard errors. 
However, often times you may be interested in a linear combination of regression parameters. For example, if you have a indicator variable for a given group, you may be interested in the group mean, which would be 
$\beta_0 + \beta_{\rm grp}$. 
Then, to find the standard error for that group's estimated mean, you would have 
$\sqrt{X^\top S X}$, 
where $X$ is a vector of your contrasts and $S$ is the covariance matrix. In our case, if we only have the addition covariate "grp", then $X = (1,1)$ ($1$ for the intercept, $1$ for belonging to the group).
Furthermore, the covariance matrix (or more over, the correlation matrix, which is uniquely identified from the covariance matrix but not vice versa) can be very useful for certain model diagnostics. If two variables are highly correlated, one way to think about it is that the model is having trouble figuring out which variable is responsible for an effect (because they are so closely related). This can be helpful for a whole variety of cases, such as choosing subsets of covariates to use in a predictive model; if two variables are highly correlated, you may only want to use one of the two in your predictive model. 
A: There are two "kinds" of regression coefficients:


*

*"True" regression coefficients (usually denoted $\beta$) that describe the underlying data-generating process of the data. These are fixed numbers, or "parameters." An example would be the speed of light $c$, which (we assume) is always the same everywhere in the accessible universe.

*Estimated regression coefficients (usually denoted denoted $b$ or $\hat \beta$) that are calculated from samples of the data. Samples are collections of random variables, so estimated regression coefficients are also random variables. An example would be an estimate for $c$ obtained in an experiment.


Now think about what covariance means. Take any two random variables $X$ and $Y$. If $\left| \mathrm{Cov}\left(X,Y\right) \right|$ is high, then whenever you draw a large absolute value of $X$ you can also expect to draw a large absolute value of $Y$ in the same direction. Note that "high" here is relative to the amount of variation in $X$ and $Y$, as pointed out in the comments.
The (estimated) covariance of two regression coefficients is the covariance of the estimates, $b$. If the covariance between estimated coefficients $b_1$ and $b_2$ is high, then in any sample where $b_1$ is high, you can also expect $b_2$ to be high. In a more Bayesian sense, $b_1$ contains information about $b_2$.
Note again that "high" is relative. Here "$b_1$ is high" means that "$b_1$ is high relative to its standard error," and their covariance being "high" mean "high relative to the product of their standard errors." One way to smooth out these interpretive hiccups is to standardize each regression input to by dividing by its standard deviation (or two standard deviations in some cases).
One user on this site described $\mathrm{Cov}\left(b_1,b_2\right)$ as "a bit of a fudge," but I don't entirely agree. For one thing, you could use this interpretation to come up with informative priors in Bayesian regression.
As for what this is actually used for, Cliff AB's answer is a good summary.
