Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.
3
votes
2answers
235 views
What might be a clear, practical definition for a “family of hypotheses” (with respect to familywise error rate)?
When trying to evaluate what constitutes a family of hypotheses within an experiment/project/analysis, I've found "similar in purpose" and "similar in content" given as guidelines for delimiting ...
4
votes
2answers
800 views
Multiple comparisons with ANOVA including one between and one within-subject effect
I'm trying to look at multiple comparisons (across levels of my between subjects factor) for a model with one between-subject factor and one within-subject factor. I was trying to use TukeyHSD on an ...
6
votes
1answer
184 views
What kind of statistical method do I need for comparing element concentrations?
I have assayed the concentrations of various metal elements in
different anatomic regions of two strains of mice. Now, for each
element, in each region, I want to find whether there
is any difference ...
4
votes
1answer
98 views
Mean of one group vs. the rest of the sample
I have data from a survey with Likert-type questions (Rate how strongly you feel about this issue on a scale from 1 to 10).
We know the respondents home town (as well as some other information, such ...
5
votes
2answers
470 views
Understanding MANOVA in case of a single predictor
I'm trying to understand the statistical analysis I saw in a clinical study. They measured performance of 3 groups of subjects with a series of performance measures (A, B, C, ..., N). The objective ...
4
votes
2answers
207 views
How to deal with identical p-values with the Benjamini-Hochberg method for correcting for multiple testing
I'm testing for mutations in DNA and I'm using the Benjamini-Hochberg method to modify the threshold I'm testing my p-values against. The method is basically to rank the p-values and compare them to a ...
4
votes
2answers
213 views
How to test for association between proportions with multiple variables?
I have tested 100 bacterial isolates for resistance to 6 antibiotics (tetracycline, erythromycin, kanamycin, streptomycin, gentamicin, chloramphenicol). The result for any antibiotic could be either ...
5
votes
1answer
363 views
If a factor variable is to be dropped in model selection, should all levels be dropped simultaneously? If so, why?
In answer to a previous question factor pooling in model selection was discussed.
If a factor or categorical variable is to be dropped in model selection, should all levels be dropped simultaneously? ...
5
votes
1answer
676 views
Is there a way to compare linear regression slopes by permutation tests?
I have some species $\times$ year matrices and analyse the temporal dynamics by regressing distance between years on $\sqrt{ {\rm time \ lag} }$. Since the data points are not independent and since I ...
2
votes
0answers
121 views
Identifying significantly different groups in a logistic regression model
In my experiment, I compare the resistance of 12 different seed types to seed beetles. I have 30 replications (i.e. 30 times a single beetle larva developing in a single seed) for each variety and ...
7
votes
2answers
189 views
Large number of correlations with MRI region of interest (ROI) variables: What adjustment should I perform?
In our group, we have measured grey matter volumes of about 20 regions of interest (ROI) and frequency of cannabis use. Now, we would like find out whether cannabis use is associated with grey matter ...
0
votes
2answers
121 views
Probability of accidentally producing significant results
I have a problem, which I think should be very common, but I mostly have been taught stochastic at university and most of my statistics knowledge is self-taught. Hence I have no clue on how to look ...
4
votes
2answers
179 views
Multiple comparisons for parameter significance in a sparse high dimensional regression model
I have a regularized linear regression model with a large number of parameters (~100,000) that has been optimized using a sparse fitting algorithm (coordinate descent with early stopping). The model ...
4
votes
1answer
140 views
How to compare two groups when there are many outcome variables?
Suppose we have two patient groups, healthy and ill patients, and we measure the chemical concentrations of multiple chemicals. Now we want to determine which chemical concentrations are different for ...
2
votes
3answers
322 views
Sidak correction
For variables A (product research) and B (product purchase), I am proposing to do three levels of analysis as follows:
Test 1: Overall:
The goal here is to see if there is a relationship between A ...
11
votes
3answers
5k views
Is there a nonparametric equivalent of Tukey HSD?
I am using JMP to examine differences in vegetation cover in growth form groups (trees, shrubs, forbs, etc.) before and after three treatments with a control.
My sample size is small (n=5) and most of ...
10
votes
1answer
248 views
Why there is usually no correction for multiple comparisons in Bayesian statistics
How would you explain to an only moderately statistically literate scientific reviewer that adjustment for multiple comparisons is not necessary when sampling from a Bayesian posterior?
1
vote
1answer
248 views
Can you still use the least significant difference test even if the F test is not significant?
Can you still used the least significant difference test even if the F test is not significant?
5
votes
5answers
2k views
When is multiple comparison correction necessary?
I have a sort of philosophical question about when multiple comparison correction is necessary.
I am measuring a continuous time varying signal (at discrete time points). Seperate events take place ...
1
vote
2answers
140 views
How to examine effect of time over four time points across two experimental conditions?
My experiment entails comparing a mock-infected group with a virus-infected group across 4 different time points (3,5,8,10 weeks) looking at the proportion of immune cells.
I have already compared ...
2
votes
2answers
613 views
How to calculate overall rankings/ratings from many pairwise comparisons?
I have a number of categories and a bunch of data about pairwise comparisons between those two categories; how do I learn information about overall rankings/ratings/etc. of all the categories?
Let me ...
4
votes
4answers
506 views
Why is it a “statistical sin” to run a large number of correlations and only report the statistically significant ones?
I recently read the following statistical 'sin' here:
Something I see a surprising amount in conference papers and even journals is making multiple comparisons (e.g. of bivariate correlations) and ...
7
votes
1answer
225 views
What article or book clearly states that one can't use protected t-tests in within subjects ANOVA?
I have been asked to use protected paired samples t-tests in an analysis. The requester states that if I do not use the overall MSe from my (1 factor with four levels) within-subjects ANOVA when ...
1
vote
0answers
36 views
Metric for cross-comparing vectors of multiple values for a few class labels
There are 20 individuals who get tested for a particular task on 3 consecutive days. They are considered to be iid. These results are stored in a 60 element vector.
There is a similar test on 3 ...
1
vote
1answer
1k views
How to calculate Sidak correction by hand?
I can calculate Sidak correction of the p value in Excel but is there a quick way to do it by hand?
I am looking for something similar to calculating the margin of error by hand, i.e. dividing 1 ...
0
votes
0answers
76 views
Maximum number of significance tests
I have a data set comprising of responses from approximately 385 people on several variables such as their shopping habits, mode of transport to the shopping centre etc. I want to do Pearson ...
1
vote
1answer
323 views
Bonferroni for outlier detection?
I am reading a book on time series analysis and I am having problems understanding the section about outlier detection.
The authors say that when you want to know whether at a certain time $T$ there ...
0
votes
1answer
1k views
How to compare effect of time across two groups on ten dependent variables?
I have one categorical independent variable with two levels (Group 1 and Group 2). These groups were measured at two times on 10 dependent variables, therefore, I have 2 sets of 10 variables for Time ...
1
vote
0answers
104 views
Fisher's criterion distribution
My task is to show that Fisher's criterion distribution when the null hypothesis is true follows $F(k-m,N-k)$ when df increases. Here $k-m$ is number of restriction.
So I can write a null hypothesis ...
4
votes
2answers
214 views
How can I compare the relative effects between values of a categorical IV?
I am a social science phd student trying to figure out a statistical test for small conference presentation I am working on. However, I realized I don't know how to run the model I want. Some ...
2
votes
0answers
107 views
Statistical significance for correlated significance tests
I refer to this article http://www.scientificamerican.com/article.cfm?id=the-mind-reading-salmon.
Is there a p-value equivalent for a set of significance tests, especially in the case where some of ...
3
votes
2answers
5k views
Threshold for correlation coefficient to indicate statistical significance of a correlation in a correlation matrix
I have computed a correlation matrix of a data set which contains 455 data points, each data point containing 14 characteristics. So the dimension of the correlation matrix is 14 x 14.
I was ...
3
votes
2answers
224 views
Can we combine false discovery rate and principal components analysis?
I am trying to get myself more savvy with literature on False Discovery Rates (FDR). At its original incarnation, FDR assumes that the tests being compared are independent. Several authors have ...
5
votes
2answers
751 views
Are t tests of coefficients in multiple regression post hoc tests?
In multiple regression, if a global F test is significant, then are t tests (or Wald tests) for the coefficients considered to be multiple comparisons and post hoc tests and should they be adjusted?
1
vote
1answer
87 views
Identifying largest proportion in a set based on a common set of companies
To run a certain activity, a population of companies use a finite set of inputs. There are 8 different inputs, but each company may not use all of them. For instance, company 1 uses inputs A, B and ...
4
votes
1answer
1k views
False discovery rate & q-values: how are q-values to be interpreted when rank of p-values is altered?
I am trying to wrap my head around False Discovery Rate, and its associated q-value; I am new to this technique, but it seems quite promising for my needs.
One sticking point I keep coming across and ...
7
votes
2answers
755 views
Correcting for multiple comparisons in a within subjects / repeated measures ANOVA; excessively conservative?
It strikes me that the available corrections for multiple comparisons in the context of a repeated measures ANOVA are excessively conservative. Is this actually the case? If so, what are some ...
1
vote
2answers
423 views
R planned comparisons in Zelig negative binomial regression
I have this model:
model <- zelig(dv~(product*intervention), model = "negbin", data = data)
intervention has two levels: neutral(=0), treatment(=1)
product ...
7
votes
2answers
283 views
Comparison of time series sets
I have three sets of time-series data I am looking to compare. They have been taken on 3 separate periods of about 12 days. They are the average, maximum and minimum of head counts taken in a college ...
2
votes
0answers
131 views
How to compare results from two groups?
I'm getting confused with the analysis of one of my experiments. Here is how the results look like.
There were P participants in the experiments and they were split into two groups P1 and P2. Every ...
3
votes
1answer
471 views
Can I have an example of pairwise comparisons of data against a within-subject factor?
Here is an ANOVA model with one between-subject factor (condition; 4 levels) and one within-subject factor (trial_seq; 20 levels).
...
6
votes
4answers
860 views
If a statistics doesn't reveal a significance do I have to calculate power for it?
Following the design and data described in this question, I did a simple one-way within-subjects repeated-measures (RM) ANOVA and found some significant p-values. I then applied non-orthogonal ...
0
votes
1answer
150 views
Correlation analysis of parameters
I have a data table detailing various histological parameters among 3 groups of patients- DLE, SCLE and ACLE.
I would like to know how we could do an analysis on the strenghth of the association of ...
5
votes
2answers
391 views
Which statistical test should I use for my experiment on aggressive interactions in killifish?
I am doing a project on sexual selection (male-male competition) in the turquoise killifish Nothobranchius furzeri.
There are two morphs of male in the population from which my fish are obtained ...
4
votes
2answers
548 views
Uncorrected pairwise p-values for one-way ANOVA?
I'm performing a relatively simple one-way ANOVA as part of a class exercise. I'd like to get pairwise uncorrected p-values out of R, so I can do a sequential FDR test in another package (I realize ...
7
votes
1answer
149 views
Inverse of false discovery rate (FDR)
Suppose we have a set $S$ consisting of $p$ features, and a subset $S_+$ of the features are positive. If $Q$ is any subset of $S$, define the false positive rate as the proportion of features in $Q$ ...
3
votes
3answers
707 views
False discovery rate from permutation testing?
I have conducted a search for genetic interactions using a simple dosage model:
Y ~ A + B + AB
where Y is the phenotype, in this case, gene expression values and A and B are vectors of genotype ...
3
votes
1answer
109 views
Calculating the value of attributes of cards in a card game
I hope this is a stats question :) There's a game (RftG) with cards, each of which has various attributes (generally bonuses of some kind) and a price. I'm wondering if there's some kind of technique ...
1
vote
1answer
356 views
Odds ratios multiple comparisons
I have to assess for each product p, the odds ratio associated (success/failure). The data are in this table:
...
3
votes
1answer
1k views
Comparing multiple time series in R
If I have many time series that I'd like to compare to see if there are relationships between the variables, (I have several dependent variables and many more independent variables) how might I go ...
