The reasoning behind the question:

The p-values depend on researcher's plan and intentions

Assume we are using frequentist paradigm

Let's say that a researcher wants to find out if an American's average height differs significantly from 170 cm. He decides to collect as many volunteers to measure as will come to his office until the end of the week. 87 people turn up in total. If he then decides to run a standard analysis for the sample size of 87, the results will be wrong. The p-values will be unreliable. This is because the p-value is conditioned on the data that is unobserved, but theoretically possible to collect. It was possible to get a different sample size (resulting in a different sampling distribution for the average height), so in reality, he should condition on other sample sizes as well, such as 45, 86, 88, or 101. This obviously becomes complicated.

For more details read for example:

The question:

I was wondering how this extends to the significance of statistical models. Suppose you find some data on housing prices on the internet, such as the famous Boston Dataset. You decide to estimate house prices using variables such as house location, age, etc., and use for this purpose linear regression. The estimates for the coefficients come up along with their significance. How does one interpret them given the problem above? This dataset could have had a different number of observations. Should we consider the intentions of the people who gathered the data? Will the p-values for the coefficients mean the same thing in the same model regardless of who calculated the model?



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.