Whither bootstrapping - can someone provide a simple explanation to get me started? Despite several attempts at reading about bootstrapping, I seem to always hit a brick wall. I wonder if anyone can give a reasonably non-technical definition of bootstrapping?
I know it is not possible in this forum to provide enough detail to enable me to fully understand it, but a gentle push in the right direction with the main goal and mechanism of bootstrapping would be much appreciated! Thanks.
 A: The Wikipedia entry on Bootstrapping is actually very good:
http://en.wikipedia.org/wiki/Bootstrapping_%28statistics%29
The most common reason bootstrapping is applied is when the form of the underlying distribution from which a sample is taken is unknown. Traditionally statisticians assume a normal distribution (for very good reasons related to the central limit theorem), but statistics (such as the standard deviation, confidence intervals, power calculations etc) estimated via normal distribution theory are only strictly valid if the underlying population distribution is normal.
By repeatedly re-sampling the sample itself, bootstrapping enables estimates that are distribution independent. Traditionally each "resample" of the original sample randomly selects the same number of observations as in the original sample. However these are selected with replacement. If the sample has N observations, each bootstrap resample will have N observations, with many of the original sample repeated and many excluded. 
The parameter of interest (eg. odds ratio etc) can then be estimated from each bootstrapped sample. Repeating the bootstrap say 1000 times allows an estimate of the "median" and 95% confidence interval on the statistic (eg odds ratio) by selecting the 2.5th, 50th and 97.5th percentile.
A: The American Scientist recently had a nice article by Cosma Shalizi on the bootstrap which is fairly easy reading and gives you the essentials to grasp the concept.  
A: Very broadly: the intuition, as well as the origin of the name ("pulling oneself up by the bootstraps"), derive from the observation that in using properties of a sample to draw inferences about a population (the "inverse" problem of statistical inference), we expect to err.  To find out the nature of that error, treat the sample itself as a population in its own right and study how your inferential procedure works when you draw samples from it.  That's a "forward" problem: you know all about your sample-qua-population and don't have to guess anything about it.  Your study will suggest (a) the extent to which your inferential procedure may be biased and (b) the size and nature of the statistical error of your procedure.  So, use this information to adjust your original estimates.  In many (but definitely not all) situations, the adjusted bias is asymptotically much lower.
One insight provided by this schematic description is that bootstrapping does not require simulation or repeated subsampling: those just happen to be omnibus, computationally tractable ways to study any kind of statistical procedure when the population is known.  There exist plenty of bootstrap estimates that can be computed mathematically.
This answer owes much to Peter Hall's book "The Bootstrap and Edgeworth Expansion" (Springer 1992), especially his description of the "Main Principle" of bootstrapping.
A: The wiki on bootstrapping gives the following description:

Bootstrapping allows one to gather many alternative versions of the single statistic that would ordinarily be calculated from one sample. For example, assume we are interested in the height of people worldwide. As we cannot measure all the population, we sample only a small part of it. From that sample only one value of a statistic can be obtained, i.e one mean, or one standard deviation etc., and hence we don't see how much that statistic varies. When using bootstrapping, we randomly extract a new sample of n heights out of the N sampled data, where each person can be selected at most t times. By doing this several times, we create a large number of datasets that we might have seen and compute the statistic for each of these datasets. Thus we get an estimate of the distribution of the statistic. The key to the strategy is to create alternative versions of data that "we might have seen".

I will provide more detail if you can clarify what part of the above description you do not understand.
A: I like to think of it as follows: If you obtain a random sample data set from a population, then presumably that sample will have characteristics that roughly match that of the source population. So, if you're interested in obtaining confidence intervals on on a particular feature of the distribution, its skewness for example, you can treat the sample as a pseudo-population from which you can obtain many sets of random pseudo-samples, computing the value of the feature of interest in each. The assumption that the original sample roughly matches the population also means that you can obtain the pseudo-samples by sampling from the pseudo-population "with replacement" (eg. you sample a value, record it, then put it back; thus each value has a chance of being observed multiple times.). Sampling with replacement means that the computed value of the feature of interest will vary from pseudo-sample to pseudo-sample, yielding a distribution of values from which you can compute, say, the 2.5th and 97.5th percentiles to obtain the 95% confidence interval for the value of the feature of interest.
A: Bootstrap is essentially a simulation of repeating experiment; let's say you have a box with balls an want to obtain an average size of a ball -- so you draw some of them, measure and take a mean. Now you want to repeat it to get the distribution, for instance to get a standard deviation -- but you found out that someone stole the box.
What can be done now is to use what you have -- this one series of measurements. The idea is to put the balls to the new box and simulate the original experiment by drawing the same number of balls with replacement -- both to have same sample size and some variability. Now this can be replicated many times to get a series of means which can be finally used to approximate the mean distribution. 
A: 
This is the essence of bootstrapping:
  taking different samples of your data,
  getting a statistic for each sample
  (e.g., the mean, median, correlation,
  regression coefficient, etc.), and
  using the variability in the statistic
  across samples to indicate something
  about the standard error and
  confidence intervals for the
  statistic.
  - Bootstrapping and the boot package in R

