Questions tagged [statistical-significance]
Statistical significance is a characteristic of a statistic viewed in light of a null hypothesis and a given significance level. It reflects whether the statistic belongs to the rejection region (is statistically significant) or the acceptance region (is not statistically significant).
6,319
questions
0
votes
1
answer
41
views
If you change the sign of a two tailed test does the p value remain the same?
If the results of a t test are reversed are the p values the same?
I am running an experiment:
Treatment: sample size - 5k and metric value - 0.57 Control: sample size - 5k and metric value - 0.62
The ...
- 23
0
votes
0
answers
7
views
Statistical test for aggregated proportion difference in difference
I would like to statistically test if difference in difference estimate is significant, but I do not know what test to use. I am trying to answer a question if war is affecting wellbeing. I have ...
- 123
2
votes
1
answer
25
views
How to calculate the significance of a distance correlation
I have two sets of data points and want to know the correlation.
I.e., my Hypothesis is "The two set of data points are correlated".
The data points are (if at all) non-linear correlated.
...
- 182
1
vote
3
answers
36
views
Comparing two groups' answers to 10 Likert questions
I’m having two different groups complete the same survey (they differ in terms of their profession) consisting of 10, 5-point Likert style questions asking how much they agree or disagree that a given ...
- 11
0
votes
0
answers
19
views
Appropriate Statistical Test to Determine Internal Cohesion of a Categorical Data
I have a dataset that is composed of mainly binary variables, each row represents a patient, and each column represents a symptom of a disease. The patient would tick 1 (if they experience the ...
2
votes
1
answer
59
views
What references assert that the effect size of a nonsignificant covariate may be high?
An answer to a FAQ notes:
The effect size of a covariate may be high, even if it is not significant.
Does anyone know of papers I can reference to support that assertion?
FWIW, I do understand why ...
- 783
0
votes
0
answers
21
views
Test for whether method decreases the metric or not
we have created a software for a given task. What we wanted to achieve is 1) to speed up the process 2) without lowering the quality of the work. Now I would like to run a test to see if 2) stands. My ...
0
votes
0
answers
45
views
+50
Paired Comparison on $n$ Tests, Repeated $k$ Times Each
Suppose I test two subjects on $n$ pass/fail (Bernoulli) tasks, $k$ times each—for a total of $n\cdot k$ tests per subject. I want to compare the subjects to see if there is a statistically ...
- 392
0
votes
1
answer
22
views
Losing significance when adding variables to hierarchical regression model
I have two hierarchical models with continuous variables. In the first block, one of the variables is significant. However, in the second block, when I add three more variables the first variable ...
1
vote
1
answer
26
views
Batch effect significance
I'm doing a project where i study bending angles in plants, with different genotypes and treatments. I have done the experiment 4 times, so i have 4 different batches. I'm trying to take into account ...
- 11
0
votes
0
answers
50
views
Is Fisher's exact test good for this use?
I have a table that looks something like this:
group 1
group 2
group 3
group 4
region1
50
217
340
53
region2
32
23
36
42
:-------
:-------:
:-------:
:-------:
-------:
male
40
200
36
42
female
...
- 11
0
votes
0
answers
13
views
How can I compare model performance across datasets of varying sizes?
I have a person wearing 2 sensors. I create two models, one using Sensor-1 and other using Sensor-2 data
I have multiple people repeating the same experiment with varying numbers. How do I a ...
- 31
5
votes
2
answers
1k
views
Can I use p-values without hypothesis?
I have been working on my Master's Thesis which included a survey and its results have been put through various tests. The unusual thing about my Thesis is that I have no hypotheses at all, making it ...
- 61
1
vote
0
answers
34
views
Does anyone recognize this significance test between two regression betas?
I came across the following test for statistical significance between two betas (predictors) from a multiple regression model. Note that $R^2$ is the model coefficient of determination, $r$ is the ...
- 163
0
votes
0
answers
13
views
What are the differences and similarities between Reductionism and Holism in data analysis in terms of data storage and data visualization?
I am currently studying Network Analysis and have met the terminologies of Reductionism and Holism and I have read many articles on the Internet about them but I wonder the differences as well as ...
- 3,185
0
votes
0
answers
29
views
How to statistically validate samplers? Is Chi-Square test relevant for synthetic data?
I am developing sampling procedures for different kinds of discrete objects (rankings, sets, etc.). In most cases I know the theoretical distribution of the objects I'm sampling. My idea is thus to ...
- 1
1
vote
1
answer
59
views
Chi-Square test for independence for field failure data
I'm analyzing the product field failure data where each product can have one or more failures (Failure A, Failure B, Failure C, Failure D etc.) My goal is to understand the correlation between failure ...
- 687
2
votes
0
answers
52
views
Which test to perform?
Design:
We tested mixtures of 12 varieties of species A with species B in multiple growing seasons using an RCBD (Randomized Complete Block Design). The content of species B in the mixture was ...
- 21
0
votes
0
answers
46
views
Significant study is not shown as such in the meta-analysis [closed]
[]
https://pubmed.ncbi.nlm.nih.gov/29300423/ significant results from this study. Seino 2018
1This caught my eye. In a meta for protein consumption and muscle mass gain in older adults, the study ...
- 101
6
votes
1
answer
132
views
Permutation test for exponential null hypothesis: really bad?
Having found nice formulas for testing the null hypothesis under exponentially-distributed samples, I wanted to see how well permutation tests could do the job. And the answer, assuming no mistakes, ...
- 1,070
0
votes
0
answers
3
views
Loan Data: Bucket recoveries 1-D array
Some context: When someone defaults on their loan, we keep track of the recoveries as a percentage of the exposure (loan amount), we have a limited time T (legally) to collect recoveries, those ...
4
votes
2
answers
122
views
How do I include these variables within my logistic regression analysis?
So my logistic regression model aims to predict the likelihood of a severe accident (yes/no) based on characteristics of drivers, car types and other factors. I have data which has all this ...
- 81
2
votes
1
answer
35
views
Statistical test for a biology experiment when plants were grown in 3 different solution concentrations
I have conducted an experiment for my Biology Internal Assessment in the IB Diploma Course where I grew Phaseolus coccineus beans for 14 days. This is the overview of my experiment:
Independent ...
- 21
0
votes
0
answers
21
views
Choosing an appropriate statistical test in Multi-Reader Multi-Case (MRMC) Scenario
I am looking for the most suitable statistical test for the following situation:
I am in a multi-reader multi-case (MRMC) scenario where 6 readers will perform binary classification on the same 165 ...
- 1
0
votes
0
answers
34
views
GLM or GLMM design for different tasks
I am conducting a study involving 70 participants diagnosed with Mild Cognitive Impairment (MCI) and 75 without MCI. All participants were engaged in taxonomic semantic, thematic semantic, and ...
0
votes
1
answer
36
views
How to perform a 2 sample Kolmogorov-Smirnov test controlling for a type II error instead of type 1
I have four completely different sets of two samples and want to check for each set if both samples come from the same distribution, so four Kolmogorov-Smirnov tests total. If it's going to be ...
- 11
4
votes
2
answers
248
views
Can I perform logistic regression or any other type of regression on this dataset?
My data set contains information regarding the number of road accidents separated by gender, age, area, and other factors. Can I perform logistic regression analysis using this dataset to predict how ...
- 81
0
votes
0
answers
7
views
In APC-Detrended Analysis: Testing for Significant Difference in Cohort Variable Coefficients, Beyond Z-Test
I have conducted an APC-Detrended analysis.
Among the results, I want to test whether there is a significant difference in the coefficient values of the Cohort variable especially between the two ...
3
votes
1
answer
55
views
Significance testing on Generalized Additive Mixed Models (GAMMs) - mgcv::gam
I am wondering whether anyone has any insight on comparing the performance of two GAMMs?
Specifically, I want to compare two models:
A "nested model", one with a smoothed age term ...
- 61
1
vote
1
answer
69
views
Overlapping confidence intervals of two proportions
According to to G. Cumming and S. Finch in their paper "Inference by Eye" the 95% confidence intervals of two independent means can overlap and the difference between the two means can still ...
- 77
1
vote
1
answer
49
views
FDR correction - different number of p-values and models/hypotheses being tested
I have $3$ predictors:
$1$ binary (prenatal alcohol exposure)
$1$ categorical with $4$ categories (drinking patterns e.g., light stable, heavy reducer)
$1$ continuous (total drinks consumed during ...
- 11
1
vote
0
answers
34
views
Can a t-test be used to compare means obtained from samples of the same batch but tested in two different labs? [closed]
I have two datasets containing size measurements. The initial dataset originates from six different aliquots drawn from a large batch of a product (a suspension with particles). Each sample was ...
- 11
4
votes
1
answer
118
views
Hypothesis tests for Rayleigh variables
Given samples from two Rayleigh-distributed random variables with unknown parameters, $X \sim R(\sigma_x), Y \sim R(\sigma_y)$, what tests can we use to determine if and to what extent their ...
- 1,070
0
votes
0
answers
42
views
Interpretation of Statistical Tests and the Importance of Statistical Power [duplicate]
I was planning on running a statistical test for hypothesis testing, but was confused if statistical power is important once a test is run.
Looking at this confusion matrix, one would ideally set ...
8
votes
3
answers
757
views
Non-significant results when running Kruskal-Wallis, significant results when running Dwass-Steel-Critchlow-Flinger pairwise comparisons
I am writing my Master's thesis on adult's performance on cognitive and linguistic measures. I have four age groups and am investigating if there is an age difference in the performance on summarized ...
- 91
0
votes
1
answer
46
views
How to statistically discover significant process change effectiveness?
I am currently working on a project where I need to assess the effectiveness of changes made in a production process. Our initial success rate was 50%, and after making some alterations, we've ...
- 301
6
votes
3
answers
217
views
Are post-hoc adjusted analyses more valid than initial analyses?
Many papers design a methodology to detect a difference between $2$ groups regarding a primary outcome. This result is then given a $p$ value to denote statistical significance. Some papers then go on ...
- 363
0
votes
0
answers
61
views
How do I proof or disprove that the size of a document influences classification accuracy?
I have a multinomial Naive Bayes model that classifies a document into 1 of 178 classes. This goes well about 50% of the time, so it's pretty good (for my use case) considering the large set of ...
0
votes
1
answer
47
views
Hypothesis test on binary preference [duplicate]
I have two product designs and I want to see if there is a significant difference between which people like which design the best. I did a survey where I showed a photo of each design and asked ...
- 101
1
vote
1
answer
36
views
Statistical significance of correlations from confidence intervals... making my brain hurt a little
Quick question (hopefully)...
If you have a correlation of -0.256 with a 99% confidence interval of -0.541 to 0.185,
OR, a correlation of -0.336 with a 99% CI of -0.597 to 0.125...
That essentially ...
0
votes
0
answers
38
views
is my p-value significant at 3.9e-11? [duplicate]
I know that if it says < 3.9e-11 it is significant because it is less than. But this time it has no < symbol, just the value of 3.9-11. Does this mean its not less than or is it still ...
- 1
3
votes
1
answer
34
views
Sample size estimate for difference of hazard ratios
I'm planning a Kaplan-Meier analysis and need to determine an appropriate sample size for the study. I have two treatment arms and want to show that there is a differential biomarker effect depending ...
- 8,490
0
votes
1
answer
152
views
What do these Wilcoxon values tell me?
I did a pre and post-test with a small number of students ($n=12$).These values came after a science intervention that took $4$ weeks. I used a Wilcoxon test for 2 related samples as there was not a ...
2
votes
1
answer
34
views
Barnard's unconditional test gives different results when switching rows and columns?
I'm really new to statistical analysis, and I've been playing around with exact tests for 2x2 contingency tables. I've noticed that for the Barnard test (implementation with ...
- 23
6
votes
2
answers
265
views
Comparing the mean of biological replicates that are not normally distributed, low N for replicates
I work in cell biology where the field standard is now to do 3 biological replicates of experiments. However, each data point is extremely laborious to get and we receive very little statistics ...
- 61
1
vote
0
answers
55
views
How to identify statistical significance in a difference between means of grouped survey questions
I'm looking at some survey data where the questions were on a Likert scale. For each question I get what percentage respond positive/neutral/negative. There are on the order of 30,000 survey ...
- 191
1
vote
0
answers
9
views
heterogenous treatment effects - a question
So I conducted a framing experiment - 4000 people received either framing A or B, and donated $ to a charity if they wanted to.
Now I want to check if framing A or B had specific efffects on ...
0
votes
0
answers
44
views
Hypothesis Testing - Varying p-value issue in my MWU
I have two machines (A & B) in our production line. A is older, B is very new. Both perform the same tasks, except that machine B is advanced and will perform 1 task less than the no of tasks in ...
- 3
1
vote
1
answer
28
views
Unsure of statistics needed while working with replicates
I apologize if the question is considered too broad, but I'm hoping to get at least a basic starting point as I'm somewhat stuck with how to approach this.
I am working with community ecological data-...
- 21
0
votes
0
answers
22
views
What is the best method for imputing binary (or integral/count) data?
I have a longitudinal dataset, and I want to create a composite score by including five healthy lifestyles to measure the overall lifestyle over time (use as predictor). Each lifestyle is a binary ...
- 113