After finishing a data collection season, there's a huge dataset containing n>>1 variables (columns) and a few different observations divided by group (rows; let's say cases and controls). The task is to find variables with "LARGE" differences between cases and controls, but before starting any test, there's a special feature to consider about the dataset:

Descriptive statistics show that most of the variables have very small dispersion (i.e., all observations are very similar, so standard dev. ≈ 0)..., although a few of them (approx. 1% of all variables) do show higher SD values. Then, although differences between cases and controls could -in principle- be detected across many variables, the fact that the majority of these variables do not show large statistical dispersion makes them unatractive. That is to say, small between-group differences would not be interesting, although they were statistically significant... So:

Should only variables with high dispersion be chosen to compare groups in statistical tests? That is to say: is it tricky, if one chooses only the subset of the top-10 variables with large statistical dispersion values, and then performes group comparisons (in order to avoid doing tests whose results would have no value)? How would Bonferroni work in this case? Would the multiple testing correction take into account only the number of tests performed, with the top-10 subset variables?

Any suggestions and/or references on this topic would be appreciated. Thanks in advance.

After finishing a data collection season, there's a huge dataset containing n>>1 variables (columns) and a few different observations divided by group (rows; let's say cases and controls). The task is to find variables with "LARGE" differences between cases and controls, but before starting any test, there's a special feature to consider about the dataset:

Descriptive statistics show that most of the variables have very small dispersion (i.e., all observations are very similar, so standard dev. ≈ 0)..., although a few of them (approx. 1% of all variables) do show higher SD values. Then, although differences between cases and controls could -in principle- be detected across many variables, the fact that the majority of these variables do not show large statistical dispersion makes them unatractive. That is to say, small between-group differences would not be interesting, although they were statistically significant... So:

Should only variables with high dispersion be chosen to compare groups in statistical tests? That is to say: is it tricky, if one chooses only the subset of the top-10 variables with large statistical dispersion values, and then performes group comparisons (in order to avoid doing tests whose results would have no value)? How would Bonferroni work in this case? Would the multiple testing correction take into account only the number of tests performed, with the top-10 subset variables?

Any suggestions and/or references on this topic would be appreciated. Thanks in advance.

A question about the (mis?)use of multiple testing corrections.

After finishing a data collection season, there's a huge dataset containing n>>1 variables (columns) and a few different observations divided by group (rows; let's say cases and controls). (All variables have the same units; they were obtained from a huge set of measurements.) The task is to find variables with "LARGE" differences between cases and controls, but before starting any test, there's a special feature to consider about the dataset:

Descriptive statistics show that most of the variables have very small dispersion (i.e., all observations are very similar, so standard dev. ≈ 0)..., although a few of them (approx. 1% of all variables) do show higher SD values. Then, although differences between cases and controls could -in principle- be detected across many variables, the fact that the majority of these variables do not show large statistical dispersion makes them unatractive. That is to say, small between-group differences would not be interesting, although they were statistically significant... So:

Should only variables with high dispersion be chosen to compare groups in statistical tests? That is to say: is it tricky, if one chooses only the subset of the top-10 variables with large statistical dispersion values, and then performes group comparisons (in order to avoid doing tests whose results would have no value)? Is it legitimate to work only with this subset and discard the rest of the dataset?

BTW, by doing this, several tests would be avoided (i.e., variables with small dispersion would not be tested for differences)..., is this a technically fair way to reduce multiple testing restrictions? (Instead of looking at all the variables, only those showing the highest variability would be assessed for differences among groups.)

Any suggestions and/or references on this topic would be appreciated. Thanks in advance.

After finishing a data collection season, there's a huge dataset containing n>>1 variables (columns) and a few different observations divided by group (rows; let's say cases and controls). The task is to find variables with "LARGE" differences between cases and controls, but before starting any test, there's a special feature to consider about the dataset:

Descriptive statistics show that most of the variables have very small dispersion (i.e., all observations are very similar, so standard dev. ≈ 0)..., although a few of them (approx. 1% of all variables) do show higher SD values. Then, although differences between cases and controls could -in principle- be detected across many variables, the fact that the majority of these variables do not show large statistical dispersion makes them unatractive. That is to say, small between-group differences would not be interesting, although they were statistically significant... So:

Should only variables with high dispersion be chosen to compare groups in statistical tests? That is to say: is it tricky, if one chooses only the subset of the top-10 variables with large statistical dispersion values, and then performes group comparisons (in order to avoid doing tests whose results would have no value)? How would Bonferroni work in this case? Would the multiple testing correction take into account only the number of tests performed, with the top-10 subset variables?

Any suggestions and/or references on this topic would be appreciated. Thanks in advance.

A question about the (mis?)use of multiple testing corrections.

After finishing a data collection season, there's a huge dataset containing n>>1 variables (columns) and a few different observations divided by group (rows; let's say cases and controls). The task is to find variables with "LARGE" differences between cases and controls, but before starting any test, there's a special feature to consider about the dataset:

Descriptive statistics show that most of the variables have very small dispersion (i.e., all observations are very similar, so standard dev. ≈ 0)..., although a few of them (approx. 1% of all variables) do show higher SD values. Then, although differences between cases and controls could -in principle- be detected across many variables, the fact that the majority of these variables do not show large statistical dispersion makes them unatractive. That is to say, small between-group differences would not be interesting, although they were statistically significant... So:

Should only variables with high dispersion be chosen to compare groups in statistical tests? That is to say: is it tricky, if one chooses only the subset of the top-10 variables with large statistical dispersion values, and then performes group comparisons (in order to avoid doing tests whose results would have no value)? How would Bonferroni work in this case? Would the multiple testing correction take into account only the number of tests performed, with the top-10 subset variables?

Any suggestions and/or references on this topic would be appreciated. Thanks in advance.

A question about the (mis?)use of multiple testing corrections.

After finishing a data collection season, there's a huge dataset containing n>>1 variables (columns) and a few different observations divided by group (rows; let's say cases and controls). (All variables have the same units; they were obtained from a huge set of measurements.) The task is to find variables with "LARGE" differences between cases and controls, but before starting any test, there's a special feature to consider about the dataset:

Descriptive statistics show that most of the variables have very small dispersion (i.e., all observations are very similar, so standard dev. ≈ 0)..., although a few of them (approx. 1% of all variables) do show higher SD values. Then, although differences between cases and controls could -in principle- be detected across many variables, the fact that the majority of these variables do not show large statistical dispersion makes them unatractive. That is to say, small between-group differences would not be interesting, although they were statistically significant... So:

Should only variables with high dispersion be chosen to compare groups in statistical tests? That is to say: is it tricky, if one chooses only the subset of the top-10 variables with large statistical dispersion values, and then performes group comparisons (in order to avoid doing tests whose results would have no value)? Is it legitimate to work only with this subset and discard the rest of the dataset?

BTW, by doing this, several tests would be avoided (i.e., variables with small dispersion would not be tested for differences)..., is this a technically fair way to reduce multiple testing restrictions? (Instead of looking at all the variables, only those showing the highest variability would be assessed for differences among groups.)

Any suggestions and/or references on this topic would be appreciated. Thanks in advance.