Signals situations where one is concerned about achieving intended power and size when more than one hypothesis test is performed.

learn more… | top users | synonyms (2)

0
votes
1answer
11 views

Multiple Comparisons test strategy

I have a test data with 3452 subjects , I have 5 different algorithms that detect the presence or absence of a disease. ...
-1
votes
0answers
12 views

Relationship between observations [on hold]

My question is how can test the hypothesis H0 H0 : Relationship between observations. How to get this? Thanks.
0
votes
0answers
15 views

Pairwise test after performing chisq. test

In R I have data which is ...
1
vote
2answers
38 views

Appropriate test for comparison of 50+ groups

I am currently working with a data set containing several hundreds of thousands of instances for which I am trying to find the most appropriate analysis. The goal is to determine whether there are ...
0
votes
1answer
43 views

Neural network: two output vectors?

Architecture: I have a CNN which does some classification for me. The output layer y consists of a vector $\vec{y}$ which is of dimension $(1, 1000)$, so it has 1.000 neurons in total (the weight ...
0
votes
0answers
12 views

difference in difference cross section or panel

Hello so i am doing some econometric on inflation targeting. What do you recommend I use if I am doing difference in difference with CPI quarterly: the method of collecting data on should it be cross ...
1
vote
0answers
5 views

Comparing two groups with different size on multiple continuous variables [duplicate]

I want to compare two groups on 9 continuous variables to see whether mean scores on these variables differ between the two groups. However, my samples have a very different size: one group includes ...
0
votes
0answers
20 views

Multiple comparison FDR for positively correlated p-values and test statistics

I have the following scenario: I have sampled two subjects on three days. I am looking for a consistent and statistically significant difference between two collections of measurements (call them A ...
1
vote
1answer
42 views

Interpretting non-contiguous significance letters from Tukey HSD test in the multcomp package? [closed]

I've only ever seen the compact letter display for Tukey's HSD give contiguous letter groupings but like in the following example this package is giving non-contiguous groupings (ie 'cde' is ...
1
vote
0answers
13 views

Testing for the ordering of several means

We have delineated 4 groups into a population and have a theory according to which the distributions of a certain characteristic of this popular are ordered in the sense that $\mu_i > \mu_{i+1}$ ...
0
votes
0answers
19 views

When shall I use multiple comparison adjustment if I perform a series of Chi-square?

I am presenting some descriptive statistics about % of access to a service in different regions (12) of a country in my bar graph over two time period points. As the results are based on 12 2X2 tables ...
0
votes
0answers
19 views

Multiple t-tests on group averages vs multi-level model

I did an experiment with N participants, each tested with 10 stimuli with type 1 and 2 variants. 10 instances of each stimulus. I want to see if response to item type 1 was significantly different ...
0
votes
0answers
4 views

Unexpected results with non-parametric multiple comparison of means (R package nparcomp functions nparcomp and mtcp) [migrated]

I am currently trying to perform a non-parametric simultaneous relative contrast effects using the R package nparcomp (https://cran.r-project.org/web/packages/nparcomp/index.html). I am getting some ...
1
vote
0answers
30 views

p-value in interpreting path coefficients in path modeling of PLS-SEM

I'm beginner at statistics, so my question may seem to have non sense for some. I was trying to interpret a PLS-SEM results, which is a kind of path modeling. The results of path coefficients are ...
1
vote
1answer
26 views

Multiple tests plus correction versus ANOVA - is one always better?

A colleague was using three pair-wise t-tests plus Bonferroni correction to detect any differences in the means of three groups. My first response was to tell him to use an ANOVA instead, since my ...
1
vote
0answers
22 views

How to discriminate among different post hoc friedman tests

I had to carry out several Friedman test comparing different yields. I also wanted to conduct a Post-hoc test to distinguish among different systems. I used the R function ...
0
votes
0answers
8 views

Approach for determining winner in A/B test with more than 2 version?

I've heard people take the top performing variation (say there are 4 variations total), and then do either a t-test or a chi-square test against each of the other versions. My issue with this is ...
0
votes
1answer
24 views

Analysis of experiment without using MANOVA

My dissertation topic is on differences in body image in different dance and different skill levels. My IV is dance type (Dancehall, Street, Contemporary, Ballet, Other and non-dancer) my DV's are 5 ...
0
votes
0answers
26 views

Any alternatives to Kruskal-Wallis and post-hoc tests?

I have non-parametric data comparing people's sense of community scores who use different modes of transport to travel I want to see if there are significant differences between the three modes of ...
3
votes
0answers
36 views

Is doing several hypothesis tests and choosing the significant ones reliable?

Case 1 Suppose I wanted to test the association between variable 'a' and variable 'b1' (i.e., I did only one hypothesis test), and I reported that 'a' is significantly associated with b1 with 95% ...
0
votes
1answer
23 views

Can I run several independent samples test to compare two stochastic algorithms?

I want to compare performance of two stochastic optimization algorithms: A1 and A2. Performance is defined as the output of a run. The lower the output is, the better is the solution the algorithm ...
0
votes
0answers
12 views

How to make a comparison across group of variables with different sizes?

I'm making a quantitative analysis for a research project. In this analysis, I have found one or more key theme which are evaluated using a value from 0 to 1. Themes are linked to categories. That is: ...
0
votes
0answers
41 views

Using logistic regression on groups of candidate SNPs in genetic study

I'm working with a large number of candidate SNPs (about 55) in a smallish sample (n=157), looking at a binary sub-diagnosis of schizophrenia (my dependent variable). Basically I am trying to ...
1
vote
0answers
35 views

Pairwise vs listwise deletion of missing data in regression?

I was wondering what would be the difference between using the pairwise versus the listwise option in a multiple regression? I have a dependent variable (reaction time) and several predictors ...
0
votes
0answers
18 views

Conover and Nemenyi post hoc give different results

Using the Wine tasting data set as provided http://www.r-statistics.com/2010/02/post-hoc-analysis-for-friedmans-test-r-code i did Conover and Nemenyi post-hoc tests. They though give different ...
3
votes
1answer
71 views

Same estimates but different p-values in tukey post hoc test (lmer)

I'm pretty new in using lmer and be confused about different p-values in Tukey post hoc tests associated with exactly the same estimates. I built a linear mixed ...
1
vote
0answers
28 views

Price comparison statistics and data visualisation

I'm getting into learning statistics and was hoping you could give me some pointers on where to go with this experiment. Basically I'm trying to compare prices for same/similar items across a variety ...
0
votes
0answers
13 views

What's the difference between FWER and FWE?

I don't understand if there is a difference between FWER and FWE and if so, which is this difference. I'm reading the book "Permutation Tests for Complex Data: Theory, Applications and ...
0
votes
0answers
19 views

Normalized strength of schedule ranking system in R

What is the process of creating a pairwise ranking system in R? Specifically looking for the iterative ranking of margin of victory determined by strength of schedule to normalize for opponents ...
0
votes
0answers
5 views

Comparing the progress in various features in two groups

My question might not seem so mature, coz I have never understood statistics. I had two groups of students whose voices were recorded in a pre and a post test. Between the pre and the post test I ...
0
votes
0answers
20 views

Multiple One-sample t-tests alpha inflation

I am examining whether the observed activation level differs from the target activation level at 7 contraction intensities in 4 muscles. I am considering using multiple one-sample t-tests (28 in ...
3
votes
0answers
37 views

Non normally distributed data and comparing correlation

I have data from a research that was conducted by me, the data refer to four conditions made of two levels of time (pretest and post-test) and two levels of group (control and intervention). ...
0
votes
1answer
46 views

Testing multiple hypotheses simultaneously

Let $X_{ij}$ be data modeled by$X_{ij} \stackrel{\text{iid}}{\sim}N(\theta_j,\sigma_j^2), i=1,2,...,50, j=1,2,...,10$. We test $H_{0j}:\theta_j=0 $ against $H_{1j} : \theta_j\neq0$ for $j=1,2,...,10$ ...
0
votes
0answers
15 views

How to obtain pairwise comparisons as Significant (S) or Non-significant (N) in Randomised Complete Block Design?

In this problem, I am testing effects of 3 insecticides on a variety of rice. To obtain data, 4 plots have been used (of equal area each). As plots can be different in soil fertility, sheltering from ...
0
votes
0answers
28 views

Case study: Testing two random effect variables separately but not together

There was a recent study published in the news, "Texting insincerely: The role of the period in text messaging." http://www.sciencedirect.com/science/article/pii/S0747563215302181 It reported the ...
0
votes
0answers
9 views

What's the best way to implement multiple intervention analysis using causalimpact ?

One of my tasks in to analyse cumulative effect of multiple interventions in a different time period. I was told that the best way to obtain right effect would be perform first intervention and ...
1
vote
0answers
20 views

Should I change interaction_average and/or covariate_average with mcp function from multcomp [closed]

I am using Dunnett's test to compare treatments to a control in an ANCOVA model that I build. Sometimes, however, the final ANCOVA model contains an iteraction term. I can still perform Dunnett's test ...
0
votes
0answers
14 views

R: Dunnett's test gives z and t value

I am using R for statistical analysis of my data. For some experiments I have to construct a linear mixed effect model using the lmer function from the lme4 package. For other experiments an ANCOVA ...
0
votes
0answers
29 views

FDR correction for multiple comparisons with Pearson's correlation and linear regression

I'm trying to correlate a measurement (survival after treatment with 'x') with gene expression data for > 10,000 genes across ~ 30 samples (cell lines) and I think I need to correct the significance ...
1
vote
1answer
21 views

multiple comparison for discrete nonparametric distributions

I have N=24 controls and M=10 patients. (M can be 20 later) Each subject has one 3D image, each element of the image is a discrete value (integer) ranging from 0 to 4 [0,1,2,3,4]. It's like a score. ...
1
vote
1answer
23 views

Problems with multiple one-way ANOVAs

Could you please help me how I can conclude if I would like to know the difference between Treatment 1 and Treatment 2: I have three groups. Treatment 1, Treatment 2 and Control. I used multiple ...
3
votes
1answer
34 views

What is the correct terminology for one/two-tailed p-values and how do I apply the Holm-Bonferroni correction?

I have been struggling to reconcile my (very basic) understanding of P-values with the approach of one of my colleagues and it appears to come down to interpretation of p-value terminology. This is ...
1
vote
0answers
13 views

Check if a difference between paired means within two populations is constant

I have a data from two experiments E1 and E2. Samples in both of them were divided into paired groups so the first group ...
2
votes
0answers
31 views

Bayesian association testing and simulation

I read some Bayesian references recently and came up with some basic or naive questions about Bayesian inference. My research question is to test the association between predictors and outcome, and it ...
0
votes
0answers
7 views

Comparing 2 treatment groups in a randomized experiment

Good morning, I am drafting a randomized experiment testing the effects of A. an increase in quantity of kcal assumed in meals B. an increase in the quality of meals (quality defined both in terms ...
0
votes
0answers
30 views

Interpreting Results from Multiple Comparisons and Joint Significance test

I am conducting a study to test whether the manufacturing quality of a machine has an effect on its failure time. I am using a hazard model with a dummy variable to identify machines manufactured with ...
1
vote
0answers
33 views

How to compare these distributions?

I have two different data frames , one per condition, where each one of them has 3 columns: specie : a qualitative value population@4days : population calculated for that specie at 4 days ...
0
votes
1answer
34 views

contrast matrix for multiple comparisons in the model

In the following model I want to create a contrast matrix (K) corresponding to the global null hypothesis that all interaction terms present in the model are zero and test both the global hypothesis ...
2
votes
0answers
47 views

p-value distribution and the applicability of different multiple testing correction methods

I came across this issue as I was analysing some genomic data sets looking for differential DNA methylation. I have generated p-values from the comparisons I am making, and am now looking into the ...
0
votes
0answers
15 views

Multiple comparisons when comparing groups with multiple outcomes

My question is similar to the one here: How to adjust for multiple comparisons with 2 groups where several outcomes were measured? However in this question they used the same test. What if the ...