21
$\begingroup$

Before I ask my question, let me give you a bit of background about what I know about statistics so that you have a better sense of the types of resources that I'm looking for.

I'm a graduate student in psychology, and as such, I use statistics almost every day. By now I'm familiar with a pretty broad array of techniques, mostly as they are implemented in the general structural equation modeling framework. However, my training has been in the use of these techniques and the interpretation of results – I don't have much knowledge of the formal mathematical foundations of these techniques.

However, increasingly, I've had to read papers from statistics proper. I've found that these papers often assume a working knowledge of mathematical concepts that I don't know much about, such as linear algebra. I have therefore become convinced that if I wish to do more than blindly use the tools that I have been taught, it would be useful for me to learn some of the mathematical basis of statistics.

So, I have two related questions:

  1. What mathematical techniques would be useful for me to know if I want to brush up on the mathematical foundation of statistics? I've encountered linear algebra pretty often, and I'm sure that learning about probability theory would be useful, but are there any other areas of math that would be useful for me to learn about?
  2. What resources (online or in book form) can you recommend to me as someone who wants to know more about the mathematical foundations of statistics?
Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ What math do you already know? $\endgroup$
    – Peter Flom
    Apr 17, 2013 at 17:44
  • $\begingroup$ Very little. I know some light linear algebra as part of learning the multivariate extensions of the GLM. Most of my statistics training has been on a conceptual level, though -- it's been geared towards getting me to understand how to use and interpret results, not necessarily to understand why a certain result (such as the CLT) is true. $\endgroup$
    – Patrick S. Forscher
    Apr 17, 2013 at 18:00
  • 5
    $\begingroup$ Linear algebra, at least some basic calculus, at least a basic course on probability, linear algebra, a little computer simulation, some statistical theory, and maybe some linear algebra. While not critical, some basic programming would be an asset. Actually the questions generated here by students tend to suggest a lot of the kind of background needed. $\endgroup$
    – Glen_b
    Apr 17, 2013 at 23:29
  • 2
    $\begingroup$ Adding to what @Glen_b wrote, multivariable calculus would not hurt. $\endgroup$
    – Alexis
    Mar 15 at 17:37

4 Answers 4

Reset to default
14
$\begingroup$

Maths:

Grinstead & Snell, Introduction to Probability (it's free)

Strang, Introduction to Linear Algebra

Strang, Calculus

Also check out Strang on MIT OpenCourseWare.

Statistical theory (it's more than just maths):

Cox, Principles of Statistical Inference

Cox & Hinkley, Theoretical Statistics

Geisser, Modes of Parametric Statistical Inference

And I second @Andre's Casella & Berger.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks, Scortchi. This looks like a great list, and was exactly the kind of thing that I was looking for (+1). $\endgroup$
    – Patrick S. Forscher
    Apr 17, 2013 at 22:31
  • $\begingroup$ Good. The first three are nearly all the maths I know. And the fourth should be read together with Casella & Berger - very different emphases. $\endgroup$
    – Scortchi - Reinstate Monica ♦
    Apr 17, 2013 at 22:46
  • $\begingroup$ Big ups for Strang's Calculus! $\endgroup$
    – Alexis
    Mar 15 at 17:39
2
$\begingroup$

Have a look at the Mathematical Biostatistics Bootcamp at Coursera.

Improve this answer
$\endgroup$
0
1
$\begingroup$

SEM is (in my opinion) very far removed from traditional probability theory and some basic statistical techniques that extend easily from it (such as point estimation, large sample theory, and Bayesian statistics). I think SEM is the result of a great deal of abstraction from such methods. I furthermore think that the reason why such abstractions were necessary was because of the overwhelming demand to better understand causal inference.

I think a book that would be perfect for someone of your background would be Judea Pearl's Causality. This book specifically addresses SEM as well as multivariate statistics, develops a theory of causality and inference, and is very philosophically sound. It's not a mathematical book, but draws heavily upon logic and counterfactuals, and develops a very precise language for defending statistical models.

I can say from a mathematical background that these results are very sound and do not require an extensive understanding of calculus. I also think it's unrealistic for someone of your pedigree to catch up on the necessary mathematics when you're already a graduate student, that's why there are statisticians!

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Thanks, this looks like a useful resource. However, it looks like this isn't quite written at the level that I want. I already have an abundance of resources on how to draw appropriate conclusions from data. What I'm missing is an understanding of the underlying math. For example, I know in general that ML estimation finds the parameter values that maximize the likelihood of observing the data, but I don't really understand how one finds those parameter values or why different methods of ML work. $\endgroup$
    – Patrick S. Forscher
    Apr 17, 2013 at 22:02
  • 1
    $\begingroup$ This requires calculus: multivariate differentiation, integration, and infinite sequence and series. Additionally, you'll need linear algebra. Once you have that under your belt, you can use any of the basic first year graduate theory texts in probability and inference. The most common one is Casella, Berger's "Statistical Inference". This is a 3 year's commitment at least to get the required maths above and beyond college algebra. You can't "get the math" without calculus. $\endgroup$
    – AdamO
    Apr 17, 2013 at 22:10
  • $\begingroup$ What level of knowledge in calculus is required? I took calculus in high school, but I haven't used it since then. $\endgroup$
    – Patrick S. Forscher
    Apr 17, 2013 at 22:27
  • $\begingroup$ They would be all the same prerequisites as that of an engineering program. Differentiation, Integration, and Infinite Series/Sequence make up a year of introductory calculus. After which, you need basic linear algebra. $\endgroup$
    – AdamO
    Apr 18, 2013 at 1:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.