Questions tagged [statistical-significance]
Statistical significance is a characteristic of a statistic viewed in light of a null hypothesis and a given significance level. It reflects whether the statistic belongs to the rejection region (is statistically significant) or the acceptance region (is not statistically significant).
6,500 questions
2
votes
2
answers
94
views
Median equality test for non-normal distributions
Despite many articles affirm that we can use the Wilcoxon-Mann-Whitney test to compare medians when the assumption of normality is violated, the Wilcoxon-Mann-Whitney test does not compare medians ...
- 444
1
vote
1
answer
25
views
Can we include post index variables in a logistics regression
Can we include post index event in logistic regression if teh event is before the end point?
I have to do a regression to understand which patients stay on medication. I have taken the index date as ...
- 11
0
votes
0
answers
13
views
What is the correct frequentist interpretation of hypothesis testing? [duplicate]
I understand the frequentist interpretation of a confidence interval with a confidence level of X% (I think): Imagine taking many random samples and following the procedure to calculate a CI for all ...
- 101
2
votes
1
answer
41
views
Randomization validity based on a modulus function
I want to run an AB test on a platform that only supports assigning variants based on address_id. Each user can have multiple address_ids in our platform i.e each user_id can be mapped to multiple ...
- 21
1
vote
0
answers
38
views
Interpretation of log-transformed explanatory variable in Poisson regression
I need to interpret a coefficient in a Poisson model that is log-transformed. Let's call the coefficient log(1+EO). The coefficient estimate is 0.0651.
The mean of EO (i.e., not log transformed) is 0....
- 11
0
votes
2
answers
31
views
On choice of y-axis range for visualization
Let's say that a disease has low prevalence. The prevalence is the proportion which should range from 0% to 100%. I want to visualize the trend of prevalence over the time. (Let's assume that ...
- 1,465
0
votes
0
answers
8
views
Evaluating the match between real data distribution and human-estimated interval
I would like to know what the best significance tests and metrics are to determine the fit of distributions in the following experiment:
Given a product feature value x, participants in a study are ...
1
vote
1
answer
43
views
How can I determine significant differences within factors and interaction effects?
My study is structured as such (false information, same idea):
Factor A: 3 levels (3 bacterial strains)
Factor B: 3 levels (3 antimicrobials)
Factor C: 2 levels (growth stage of bacteria: early, old)
...
3
votes
1
answer
74
views
Is random slope meaningful when the relevant fixed effect is not significant?
I am building a liner mixed model with the following syntax:
$$
y \sim \text{run} + (1 + \text{run} \mid \text{subjects})
$$
The summary is as follows:
The fixed effect of $\text{run}$ is not ...
- 113
0
votes
0
answers
12
views
How to define power in case of multiple tests that complements with family-wise error rate?
Family-wise error rate is defined as the probability of rejecting at least one true null hypothesis, that is, the probability of committing at least one Type-1 error.
How do we define the power in ...
- 211
0
votes
0
answers
14
views
Magnitude and directionality of bootstrapped differences between data in a 2d space
TL;DR: Are the blue and orange bootstrapped distributions in the plot different? And what is the directionality of that difference?
I'm analyzing two sets of multivariate time series using something ...
- 1
0
votes
0
answers
39
views
Test between model output over a binary variable
I have a set of features that includes a binary variable (called $B$) and some others ( binary or numerical variables, let's these covariates $X$).
I have a target variable that can be either binary ...
12
votes
6
answers
4k
views
Why isn't a confidence level of anything >50% "good enough"?
I took a graduate-level stat course so I feel really dumb asking this, but I'm having trouble with the meaning behind 95% confidence. (Please be nice to me.)
I currently running an A/B test on two ...
- 319
0
votes
0
answers
14
views
Comparing two (derived) quantities for which uncertainty is known
I need to determine if two quantities, A and 𝐵, with their associated uncertainties, $u_A$ and $u_B$, are statistically different.
Both quantities have been derived from measurements.
The ...
8
votes
3
answers
613
views
How to Interpret Statistically Non-Significant Estimates and Rule Out Large Effects?
I'm working on a regression analysis and have obtained a point estimate that is statistically non-significant. Economically, a non-significant result makes sense in my context, but I want to ensure ...
- 3,209
0
votes
0
answers
10
views
Is the significance test in GMM more reliable than it in ML, especially when the regularity conditions are not met?
Maximum likelihood vs generalized method of moments
From the answer to the above question, we can draw a conclusion that GMM and ML can get the same parameters' estimators when the model is exactly ...
- 41
3
votes
1
answer
32
views
Testing for significance where results are expressed as a ratio to an in group control
I originally asked this question in the biology stack exchange site, but realise that it might be more relevant here in the stats site:
I have a large dataset that represents a library of yeast ...
4
votes
1
answer
123
views
How do I test if a population change is statistically significant?
I have data showing different statistics for all parishes in the country for 2014 and 2024 (I actually have data for all the years in between as well, but I am comparing these two years). The data ...
- 43
1
vote
0
answers
22
views
What statistical tests can I use in R to ascertain the statistical significance of clusters obtained using DIANA (DIvisive ANAlysis Clustering)?
I have a data set of cuticular hydrocarbons (CHCs) from 60 samples of aphids collected across mid, early and late activity season on three different plants species that they fed on. The CHCs can ...
- 11
0
votes
1
answer
45
views
What does getting a non-significant log(theta) mean?
I am running a simple model in R testing if fish density (nr of fish per volume) depends on the depth in the area. There are a lot of areas with zero fish so I use a ...
2
votes
1
answer
88
views
Measure the significance of differences in internal distances in two groups
I am a linguist studying grammatical variation in 169 female and 169 male Norwegian authors (using a treebank), based on eight inflectional and syntactic properties. The eight properties are eight ...
- 21
0
votes
1
answer
37
views
How to adjust a variable by age, sex and BMI
I'm working on a database with almost 300 patients, of which I have their age, sex, BMI and their level of diabetes (which, for the purpose of the study, is stratified into mild, intermediate or ...
3
votes
2
answers
186
views
Asymptotic consistency of the Clopper-Pearson interval
I have a series of random variables $X_n \sim Bin(n,p)$ and I want to claim that for every confidence level $c \in (0,1)$ $ClopperPearsonLower(X_n, n, c)$ converges in probability to $p$, that is for ...
- 235
5
votes
2
answers
76
views
How to know what effect size is excluded by a negative trial
I am trying to get my head around how best to think about "negative" studies in Medicine. For background, I'm a medical doctor in Critical Care, not a statistician, and I'm not a researcher. ...
- 63
0
votes
1
answer
28
views
advice on statistical method
I just want some opinions on the best way to analyse some data as I am not very savy with statistics.
I have been collecting abundance data for microbes in soil. Gene copy numbers.
I have two forest ...
- 21
1
vote
1
answer
35
views
Why estimates of data via residuals has 50/50 effect on significance compared to original data?
I am generating data according to the following model
In the real dataset, variable I is unknown and the goal is to study the relationship between I and D, B11, to see if it is significantly non-zero....
- 425
1
vote
1
answer
48
views
Epsilon parameter of the tost() function in the 'equivalence' R package
This is my first time using an equivalency test.
I am using the tost() function in the 'equivalence' R package, and I want to test the hypothesis that these two groups are equivalent:
...
- 111
2
votes
3
answers
79
views
Is there a way to determine if there is a statistically significant difference between measurements obtained by two different methods?
Here's my situation. I have 259 samples from a data set, and each sample has a value calculated by two different methods. Both methods are trying to describe the same phenomenon, and ideally when a ...
- 85
1
vote
1
answer
43
views
Can statistical tests be applied to analyze temperature anomalies in vaccine refrigerators across different shifts and days?
Background:
I'm analyzing the vaccine refrigerator's internal temperature data collected during COVID pandemic (2021-2022) from all over an country. I measured the internal temperature of the 1077 ...
- 23
0
votes
0
answers
21
views
Which test is appropriate to test the significance of odds ratios?
I'm new to statistics, and I'm wonderig if there is a best test to use in this situation. I have some odds ratios of age buckets and chance of getting cancer, and I want to test if the odds ratio of ...
- 11
1
vote
1
answer
63
views
Determining statistical significance
I am trying to determine the statistical significance of my results at a 95 percent confidence level. I have the coefficients and the standard errors. I wanted to check if the following statement is ...
0
votes
0
answers
21
views
Hausman test p-value jumps to 1 after adding control variable?
I want to test whether a fixed or a random effects model is better suited for my data. I am including all variables which don't make the p-value jump, estimate both models and the Hausman test gives ...
- 21
1
vote
0
answers
46
views
How to assess a tolerance interval study when data needs to be transformed using Johnson transformation
I'm doing a variable data analysis in which I need to perform a tolerance interval assessment to make sure that my population is meeting the Lower specification limit (LSL) of 6 lbf. To be able to ...
- 11
2
votes
1
answer
58
views
How to compare ranks for individual items across multiple lists using statistical tests?
I have multiple ranked lists for the same items, I want to compare the ranks of a specific item across two subsets of the lists and assess the statistical significance, what statistical tests I should ...
- 123
3
votes
1
answer
72
views
Two ways of meta-analysis: Which one is preferred?
Suppose that I have $k=1, 2, 3...K$ studies with the same set of a continuous independent variable $Y$ and dependent variables $X$. The effect between studies may be different, and I want to run a ...
0
votes
0
answers
28
views
Why does the sign on the variable changes with the lag while running ARDL model?
The dependent variable of the model is mortality rate, while independent variable is mean: Mean year of schooling, and GDP for the Gross domestic product, percentage change. While they are all either ...
1
vote
1
answer
69
views
Using Friedman Test for Time-of-Day Analysis on Vaccine Fridge Data?
I worked on a project monitoring vaccine fridge temperatures, collecting data every 30 seconds over two years. I want to compare high temperature anomalies (above 8°C) across different times of the ...
- 23
0
votes
1
answer
24
views
Primary Care Physician Visits resulting in Hospital Visits - Statistical Measure
I'm looking for some guidance to help with statistical significance for the below problem (which may need to be reframed).
Problem Statement: Are PCP (primary care physician) visits driving more ...
- 1
1
vote
0
answers
35
views
How can for two groups one variable be significantly different but another not if both measure same construct?
Overview:
Two sets of ordinal data collected using two different measures for usability
These two measures are proven to be correlated (also confirmed for this dataset)
Each dataset can be divided ...
- 123
1
vote
2
answers
24
views
What is an appropriate statistical test of significance of whether the result of one medical exam has affected the participation in a second exam?
I have a number of participants of a medical exam. After getting their result, they then have the option to have another, more detailed, exam. Visually, it seems that if a participant receives a high ...
- 13
1
vote
1
answer
53
views
Appropriate significance test for small sample size, unclear (non-normal) distribution
Given data:
30 datapoints in the form of usability scores (System Usability Scale)
2 groups within data (Group 1: 17, group 2: 13, independent, unequal sizes)
Objective:
See if there is any ...
- 123
0
votes
0
answers
23
views
How to test for "No significant difference"? [duplicate]
my company is in game development. We are trying to test if adding some more ads in specific places will effect the players engagement badly (decrease playtime, retention,...).
Can you guys advise me ...
0
votes
1
answer
18
views
Extraction of significant samples after paired measures
I have 2 sets of paired values of time: A and B. I know that B is significantly higher than
A using a Wilcoxon test (I have a small population) but I want to know which samples have a significantly ...
- 1
0
votes
0
answers
94
views
How to determine or show if two solutions to linear regression are different
Assume that I have a system and that I can measure both the inputs and outputs of and that I can formulate a solution I want in the form of linear regression: $A \vec{x}=\vec{b}$. Further assume that ...
- 516
1
vote
1
answer
52
views
What statistical method to use for binary independent variable and continuous dependent variable
I have a dataset that has columns item, week, and units sold. I have another dataset that has columns group, week, and flag.
The weeks for both datasets lie in the same range. Units sold is continuous ...
- 199
0
votes
0
answers
22
views
How do I quantify the significance of a percentage ranked data set?
I'm considering seven different forecast models for a given data set, over a number of forecast iterations, and quantifying the number of times each forecast model was the most accurate overall.
I ...
1
vote
2
answers
88
views
What hypothesis test should be used for two categorical variables in Likert scale?
I'll give an example.
Let's say I conducted a health survey in a hospital. I asked two questions. The first question is: 'I take medication to sleep well' and the second is: 'I feel unwell in the ...
- 11
1
vote
2
answers
64
views
Misunderstanding p-value [duplicate]
I read that the p-value is the likelihood that a number is equal to or greater than the actual observed number if the null hypothesis is correct 1 . I understand the null hypothesis means that there’s ...
- 111
1
vote
1
answer
38
views
Survey-adjusted standard errors for a compound annual growth rate
I am trying to calculate a standard error for a compound annual growth rate. Both the begin and final estimate are from a complex survey. I am using the following formula to calculate the growth rate:...
- 113
0
votes
0
answers
23
views
Testing whether mean sample difference between two groups is different in two different samples
Suppose in a medical study there are the following students recruited to the study:
300 Japanese males, 500 Japanese females
600 Australian males, 700 Australian females
The particular study is about ...
