Over the years, I picked up many Statistics Concepts in by a variety of situations and means. I studied some Statistics maybe for a couple semesters almost 10 years back. But I also picked up concepts while doing Machine Learning work. I only understood things in a narrow scope - I always tried to get away knowing only what I need to know for a project etc.

Consequently, I do not have a big picture understanding of Statistics. For eg. I know vaguely what a t-test is and what a chi-square test is. But I can't relate both these concepts solidly in my head. And without relating each of these concepts I feel they are useless and not as powerful.

I have come to understand that the only way I understand anything is by working on hard problems chosen by experts. It should be hard so that I take some time to work it, also it should be chosen carefully by experts so that each problem contains a 'moral' which takes me one step closer to enlightenment.

So, help me assemble a set of problems to work through to seek out zen through statistics. Scope is all of statistics (regression, ANOVA, t-tests etc, Structural Equations of Latent Variables etc).

  • $\begingroup$ Looks like a good candidate for a Community-Wiki question. $\endgroup$ – chl Apr 12 '11 at 7:52
  • $\begingroup$ @chl, that makes sense. Go ahed if you have the privileges. $\endgroup$ – jason Apr 12 '11 at 14:01
  • 1
    $\begingroup$ I know two excellent series of exercises from Ecole Polytechnique: "Petites classes de statistique", the first one by Christian Robert and the second one by Olivier Cappé. But there are written in French. Type "Petites classes de statistique" in Google and you will find. $\endgroup$ – Stéphane Laurent Jul 15 '12 at 16:39
  • 2
    $\begingroup$ A large number of the questions posted on this very site provide problems ranging from easy to impossible. The replies provide a richer set of answers than any problem set or textbook could possibly offer, because they often represent multiple different attempts at solutions. $\endgroup$ – whuber Jul 25 '12 at 15:19

Mathematical Statistics and Data Analysis, by John A. Rice, Third Edition

If you are struggling to understand how different things in statistical relate to each other, the "glue" that you are missing is an understanding of mathematical statistics.

Rice's textbook provides the theory that justifies most of the statistical tests and methods which are used in introductory statistical courses. For example, it derives the $t$ and $\chi^2$ distributions and describes how they can be used to construct statistical tests. By contrast, most introductory statistical texts list the different distributions and tests but do not derive them.

It is a standard textbook in mathematical statistics. It does not assume much in the way of existing knowledge, but it does require some effort to work through the content (particularly if you are not comfortable in maths). There are lots of examples at the end of the chapters.

No doubt there are other statistical textbooks of equal quality. My key point is that an understanding of mathematical statistics is the thing required in order to understand statistics at a non-superficial level and it is particularly useful if you come from a machine learning background as the basic logic is very different (i.e., proofs rather than learning by trial and error and bake-offs).

  • $\begingroup$ Could you provide a brief description of this textbook or indicate what it covers? $\endgroup$ – chl Jul 25 '12 at 7:32
  • $\begingroup$ I have modified my answer. $\endgroup$ – Tim Jul 29 '12 at 23:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.