Skip to main content
Replaced questionably-related tags with more descriptive and accurate tags.
Link

Can one leave out data from researchresearch because it is not significant?

edited tags
Link
whuber
  • 333.5k
  • 63
  • 792
  • 1.3k
deleted 1134 characters in body
Source Link
whuber
  • 333.5k
  • 63
  • 792
  • 1.3k

I've encountered this sentence while reading an article on sciencemag.org.

In the end, responses from just 7600 researchers in 12 countries were included because the remaining data were not considered statistically significant.

Is this a proper way to do research? To leave out results because they were not considered statistically significant?

EDIT: This is discussed on p.104-105 of the full report from Eurodoc:

While conducting data cleaning procedures, the Eurodoc survey experts’ team decided to run a power test analysis. Based on the assumption of fully completed questionnaires which will result in a multi-normal distribution, a power test for estimation of the confidence interval was used. This was done to test the accuracy of the data. It was decided to accept maximum a 6% error-level at a 95% confidence interval. A loss of 16% of the sampling size resulted in a sample of 12 participating countries with 7,600 participants. The report ended up including 610 doctoral candidates from Austria, 301 from Belgium, 324 from Croatia, 654 from Finland, 1,126 from France, 1,165 from Germany, 583 from the Netherlands, 755 from Norway, 907 from Portugal, 246 from Slovenia, 399 from Spain, and 491 from Sweden. A consolidated view indicates that challenges reside on how to access the target group rather than with the technological feasibility of the survey.

I've encountered this sentence while reading an article on sciencemag.org.

In the end, responses from just 7600 researchers in 12 countries were included because the remaining data were not considered statistically significant.

Is this a proper way to do research? To leave out results because they were not considered statistically significant?

EDIT: This is discussed on p.104-105 of the full report from Eurodoc:

While conducting data cleaning procedures, the Eurodoc survey experts’ team decided to run a power test analysis. Based on the assumption of fully completed questionnaires which will result in a multi-normal distribution, a power test for estimation of the confidence interval was used. This was done to test the accuracy of the data. It was decided to accept maximum a 6% error-level at a 95% confidence interval. A loss of 16% of the sampling size resulted in a sample of 12 participating countries with 7,600 participants. The report ended up including 610 doctoral candidates from Austria, 301 from Belgium, 324 from Croatia, 654 from Finland, 1,126 from France, 1,165 from Germany, 583 from the Netherlands, 755 from Norway, 907 from Portugal, 246 from Slovenia, 399 from Spain, and 491 from Sweden. A consolidated view indicates that challenges reside on how to access the target group rather than with the technological feasibility of the survey.

I've encountered this sentence while reading an article on sciencemag.org.

In the end, responses from just 7600 researchers in 12 countries were included because the remaining data were not considered statistically significant.

Is this a proper way to do research? To leave out results because they were not considered statistically significant?

Loading
Tweeted twitter.com/#!/StackStats/status/119713559616815104
deleted 1 characters in body; edited title
Source Link
user88
user88
Loading
Source Link
upabove
  • 3.2k
  • 10
  • 35
  • 39
Loading