# Using computer simulations to better understand statistical concepts at the graduate level

Hi I'm taking a graduate course in Statistics and we've been covering Test statistics, and other concepts.

However, I am often able to apply the formulas and develop a sort-of intuition on how stuff works but I am often left with a feeling that perhaps if I backed up my study with simulated experiments I will develop better intuition into the problems at hand.

So, I have been thinking of writing simple simulations to better understand some of the concepts we discuss in class. Now I could use say Java to:

1. Produce a random population with a normal mean and standard deviation.
2. Then take a small sample and try to try to empirically calculate Type-I and Type-II errors.

Now the questions I have are:

1. Is this a legitimate approach to develop intuition?
2. Is there software to do this (SAS?, R?)
3. is this a discipline in Statistics that deals with such programming: experimental statistics?, computational statistics? simulation?
-
I use simulation all the time to try to better understand what's going on. You could use pretty much any programming language or statistical program to do these types of experiments (even Excel). –  John Oct 3 '12 at 19:27
+1, simulations & figures are 2 of the most helpful techniques for building intuition. I've used them commonly to help others & myself understand things. There are lots of answers to CV questions that use sims to illustrate stuff. If you want some links, I could easily list some of my own answers that have used sims in this way. You can also ask a question here on CV in this vein; eg, 'I'm trying to understand _____, but I'm having difficulty, can someone provide an explanation w/ a sim that will make it clearer?' or, 'I did this sim & it suggests that it works this way, is that right?' –  gung Oct 3 '12 at 21:53
In an effort to meet the high standards and expectations of this site, I use simulation (as well as theoretical derivations and illustrations) in every answer that would benefit from it. A large fraction of my recent replies will include some form of simulation, especially almost any reply to a question with the r tag. For examples, you can look through them from the search page if you're interested. –  whuber Oct 3 '12 at 22:24
You'll find lots of great ones by looking through @whuber's answers. Since my answers (& programming abilities) tend to be less sophisticated, they might make a nice 1st step. I use a sim to show that it's hard to use model fit to pick the best link in a GLiM here: difference-between-logit-and-probit-models. Here I use a sim to show how power drops as group sizes become unequal in the t-test: how should one interpret the comparison of means from different sample sizes. –  gung Oct 3 '12 at 22:41
This CV question: explanation-of-statistical-simulation, may also be of interest to readers of this thread. –  gung Nov 6 '12 at 14:24

I like your question but don't have specific answers to 2 and 3? I imagine that software packages like SAS (broadly speaking of SAS products and not just SAS/STAT) may have tools that facilitate simulation but I can't say for certain. I don't think this sort of thing fits as a branch of mathematics or statistics.

Now question 1 is what I would like to focus on. Simulation can help in learning statistics at all levels and can aid in statistical research in general. Indeed there are journals focussed on simulation and computation. Even the FDA is recognizing the imprtance of simulation in designing clinical trials and to help predict outcomes.

In the 1960s Julian Simon taught introductory statistics using simulation as a motivator. Although controversial he later claimed that he was doing resampling (permutation and bootstrap) prior to Efron. He published a book using these ideas in 1969. It certainly lacked the theory and was only a teaching aid and not a new approach to statistical estimation. He did not develop any of the mathematical properties that came with and after Efron.

I think for introductory statistics it is useful to do simulation to demonstrate sampling distributions, show how the central limit theorem comes about and physical simulation through the quincunx demonstrates the DeMoivre - Laplace version of the central limit theorem.

Sometimes it enhances intuition. I think that the Monty Hall problem is puzzling and seemingly paradoxical even to mathematicians like Paul Erdos. But simulating the game is often very convincing. There are many problem in probability that are counterintuitive and simulation can, I think help.

In 1978 when I was working on my PhD in extreme value theory I had an intuitive idea for a limit theorem that I was trying to prove. I struggled with the mathematics. Then I decided to simulate the stochastic process and the simulation "confirmed" my result. This gave me the confidence to push on a prove it.

So even at the graduate level and beyond simulation can be useful in two ways.

1. To help develop intuition as you suggestion in question 1 but also

2. To confirm intuition as I did in my thesis

-
I found someone who has addressed the Monty Hall problem in SAS and R here sas-and-r.blogspot.com/2010/01/… -- enjoy –  user1172468 Oct 4 '12 at 0:38
Susan Holmes at Stanford University put the Monty Hall game simulation on her website several years ago. Thanks for reminding us @user1172468 that many people can and probably are putting simulations on theor websites. –  Michael Chernick Oct 4 '12 at 1:27
2. R would suit you fine. Coding will be quite easy for you if you know Java already (or any other "standard programming language" for that matter).
3. Computational statistics deals with the design of algorithms for implementing statistical methods, probably that is the closest to what you try to describe here.