Would you please explain me the difference between Probability distribution and Sampling distribution easily ?
Is that the difference : in probability distribution we have probability for every individual whereas in sampling distribution we get probability for statistic ?
Does sampling distribution require more than 2 observations ?
If you take a sample, each sample having two (or more) observations, from a larger population (or from a probability distribution), the means (and other statistics) of those samples would have a distribution.
This sampling distribution would not be the same distribution as the distribution of the original population. For example, if your original probability distribution was uniform, then the sampling distribution (of the means of the samples) would be approximately normal. [1]
A simple example (and an easy experiment) is to use a single unbiased die as your probability distribution. The probability distribution of a single die roll is uniform, with each number having 1/6th of a chance of occurring. But if you roll multiple dice (a sample) and take the mean, then keep repeating the process then the means of each sample (pair of dice) will have their own distribution. If you repeat often enough, a graph of the means (the samples) should start to look normal, i.e. bell shaped (and not remotely uniform). The sample distribution looks nothing like the original probability distribution.
Taking a single sample of two is not recommended, not because it doesn't have a sampling distribution, but because with only one sample of two observations you don't have enough information to tell (infer) what that sampling distribution looks like. In particular you will not necessarily have "good" estimates of the variance of the sample mean around the population mean, and so you will not be able to tell whether your sample mean is significantly different from any hypothetical population mean.
Rolling just two dice won't give you a very good idea whether the die is biased. If you rolled snake-eyes, your estimate of the mean is 1, and variance zero, and you would conclude that the mean was significantly different from 3.5. But this is a bad estimate of variance because the sample is too small. You don't really have enough data to conclude that the die is biased.
If you repeated the sampling often enough, you could gain a fair impression of what the sampling distribution looks like by plotting the sample statistics (e.g. the means) in a histogram. Even with a sample of only two observations a large number of repeated samples would start to converge to the sampling distribution. This is basically what bootstrapping does. Note that this will not necessarily tell you what the original population distribution looks like.
To continue the example, if you keep throwing samples of dice, and take their means and graph them then you should end up with a nice bell shaped curve centered on 3.5. This is the sampling distribution. You can compare this with the graph of a single die, rolled repeatedly, which should look uniform.
in simple terms to say.... sampling distribution is the distribution obtained from the samples means or other statistics from the sample datas... where as the probability distribution is the distribution based on the parameters from the population.
