Is there a branch of statistics that tries to explain "why" the dataset has certain statistical properties?

Question

Suppose I have a big dataset and I compute some statistical summary of it - e.g., the correlation of one dimension with another.

I think a reasonable question to ask would be "what data points explain this result" - e.g., perhaps, is it because there were two huge outliers that explain the whole correlation? Or are all the points approximately equally important?

Or, say, I'm computing the mean of a response variable, measured at two different values of a parameter. I find that the means are approximately the same. Why is it the same: is it because the distributions of the variable are the same, or is it because the mean in both cases is determined by a big cluster that determines the whole mean and obscures the differences in the rest of the distribution?

I guess, in general, I'm interested in computing sensitivity of a statistical summary (not necessarily a single number) w.r.t. the data points and parameters involved in the computation.

This notion of sensitivity would help me, on one hand, avoid meaningless results (which are entirely explained by abnormal measurements), and on the other hand, it would guide my further exploration.

So: is there a branch of statistics that studies this kind of sensitivity? If yes, what are some useful methods from that branch / what further reading would you recommend? E.g., I imagine, it might be useful for data visualization methods - not just draw the data, but colorize each point according to the sensitivity of some metric w.r.t. that point?

I tried googling for things like "explanatory statistics", "sensitivity of statistical summaries" etc., but did not find a lot. I found Uncertainty and sensitivity analysis but this is not quite what I'm looking for - I'm not interested in sensitivity of one variable of the dataset w.r.t. the other, I'm interested in making deeper answers to questions like "do these two variables correlate".

P.S. Some more googling yielded the keyword "input importance" and some data visualization methods, e.g. http://vis.cs.ucdavis.edu/papers/TVCG_Chan_GSS.pdf . But I crave for advice from experts :)

Answer 1

I am not sure if there is a pithy title for this entire topic but it is certainly an important issue. Maybe "robust statistics" would be a good place to start?

The aptly-named empirical influence function describes how an estimator (e.g., the mean or median) depends on the value of one of the points in its sample. It can also be generalized into the "influence" or sensitivity function, which asks how the estimator's value changes as the distribution of the data shifts.

You could also consider an estimator's breakdown point, which is essentially the proportion of "bogus" values (e.g., arbitrarily large) that an estimator can tolerate. The mean, for example, has a breakdown point of zero because you can completely change its value by subsituting an arbitrarily large positive or negative value for a single point in the data set. On the other hand, the median is very resistent to this sort of 'attack;.

In regression contexts, DDFITS is a diagnostic which asks "how does the prediction for this point change if that is included/excluded in the analysis?" Cook's Distance (or Cook's D) is a similar quantity (they're calculated differently, but are inter-convertible). Leverage is a related quantity but is only affected by the independent variables' values, rather than both the independent and dependent ones.