Questions tagged [statistical-significance]
Statistical significance is a characteristic of a statistic viewed in light of a null hypothesis and a given significance level. It reflects whether the statistic belongs to the rejection region (is statistically significant) or the acceptance region (is not statistically significant).
6,500 questions
1
vote
1
answer
252
views
Assessing Logistic Regression and Determining if Splines Are Appropriate
I'm working on building a logistic model which will be used to estimate the probability that an account will skip on their monthly payment. My dataset roughly includes 50,000 observations with 15% of ...
CommunityBot
- 1
0
votes
0
answers
51
views
Am I going about statistical significance testing for weighted categorical survey data the right way?
I created a (fictional) dataset that contains individuals' car colors and their level of agreement to the question "I enjoy driving fast". The survey includes three car colors (red, black, ...
- 1
0
votes
1
answer
45
views
Is there a standard way to quantify & test the benefit to one covariate by including a second covariate?
For a linear model (or glm), is there a standard statistic (and associated test) to quantify whether inclusion of a covariate increases the strength of the association between another covariate and ...
- 419
4
votes
4
answers
544
views
How to permute p-values?
Recently, my colleague encountered a problem while working with a dataset, df_a, which contains gene data. Each row represents a ...
- 153
13
votes
3
answers
44k
views
Significant predictors become non-significant in multiple logistic regression
When I analyze my variables in two separate (univariate) logistic regression models, I get the following:
...
0
votes
1
answer
47
views
Bradley's liberal criterion
I'm reading an article where the authors utilized Bradley's liberal criterion to estimate the robustness of the F statistic in the context of post hoc tests.
The problem here is that they said the ...
- 5,294
5
votes
2
answers
214
views
How to include a nominal independent variable two or three times in a regression model?
So my logistic regression model aims to predict the likelihood of a severe accident (yes/no) based on the characteristics of drivers, car types and other factors. I have data which has all this ...
CommunityBot
- 1
0
votes
0
answers
117
views
Which significance test to use for a difference of means? How to be sure?
[Updated] I have two data sets containing all official recorded games played in the USA for this one sport. One data set is for the women's teams and the other for the men's teams. I'm trying to ...
3
votes
1
answer
5k
views
Reporting regression equations for non-significant results
Do you report the regression equation for non-significant multiple regressions if you have reported them for significant regressions in the same work?
My paper has six multiple regressions in it. ...
- 128k
0
votes
2
answers
53
views
Suitable statistical test
I have 2 groups - people that were ever active on my website and people who were never active. Activity is a metric based on some actions - both groups of people visited the website.
Each group used a ...
- 809
9
votes
3
answers
29k
views
What if an overall ANOVA is not significant, but specific contrasts are?
Study Design: I have a 2x3 factorial design, 2 levels of Time (2050 or 2100) by 3 levels of information (None/Control, Moderate, Extreme).
I set up some very specific contrasts when analyzing this ...
- 1
4
votes
1
answer
3k
views
Test of significance for glmer
I would like to test the effect of a treatment ("crop") on species richness. I would rather use a glm for richness as it is a kind of count data.
Besides, I have a nested sampling design (5 values ...
CommunityBot
- 1
1
vote
0
answers
316
views
How to compare the similarity between two particle size distributions?
I have been trying to find an answer for this question.
I have two soil particle size distributions, which both have 4 categories.
I should do a statistical test to find out if they are statistically ...
0
votes
1
answer
285
views
How to test significance between rates of occurrences
Without going into domain specific information, I have a data set in which I'm testing the rate of occurrences of an event like so:
Event 1
# of trials: 378
# of occurrences: 109
# of occurrences ...
CommunityBot
- 1
4
votes
3
answers
259
views
Statistical testing on non-random sample?
I am working with a non-random sample in an observational study but would like to do statistical testing to show certain trends and processes. Statistical tests assume random sampling, which is not ...
- 17
3
votes
1
answer
227
views
What is the variance of residuals in logistic regression?
Recently I read Wald test for logistic regression this post, which states why do we use Wald test in logistic regression and no t-test instead. It says, that the main reason behind it, is that for ...
CommunityBot
- 1
0
votes
1
answer
271
views
What statistical test is appropriate? Binary dependent variable, categorical independent variable(s)
I'm trying to understand the relationship based on survey data between one binary dependent variable (Did Give a Bribe for a Vaccine) and where the respondent lives (Urban, Peri-urban and Rural) - ...
CommunityBot
- 1
0
votes
0
answers
16
views
Estimate SD[Z=X*Y] which one is correct?
Given we have a sample of two variables X and Y with sample size n. I want to calculate the standard deviation of Z = X*Y. I don't know which of the two options bellow are correct?
Option 1:
Simply ...
- 193
0
votes
0
answers
22
views
Model comparison and experimentation for a thesis result
We are conducting a study to compare the accuracy of two computer vision models:
Model A: Trained on a non-augmented dataset of 11,200 real-world images.
Model B: Trained on an augmented dataset ...
- 101
3
votes
4
answers
143
views
Parametric hypothesis testing for non-normal data
Are there any methods to make parametric hypothesis testing assuming that data is sampled from a known but non-normal continuous distribution?
I'm glad to see a solution to any particular ...
- 19k
3
votes
2
answers
305
views
T-test and ANOVA suggesting different results?
I'd like to start off by saying I am very much a beginner to statistical analyses, so I might be missing some key ideas here.
Context: My study involves an experimental group and control group that ...
CommunityBot
- 1
3
votes
4
answers
1k
views
Correct Interpretation for One-tailed Hypothesis with Inverse Result
I am currently doing a research and have this hypothesis for a variable:
HA: "Variable A has a significant positive influence on Variable B"
However, the result was actually negative and ...
- 17.2k
1
vote
1
answer
57
views
What level of statistical significance to choose for test of equaility of seemingly unrelated regression coefficients?
It is possible to compare effects from models estimated as seemingly unrelated [1] [2] [3]. DVs may be related, IVs may be related, or both may be true. In any event, analysts test for equality of ...
- 326
2
votes
4
answers
539
views
Using successive hypothesis tests to test the magnitude of the difference
I don't understand why hypothesis tests only test whether there is a difference in statistics, but not how big the difference is. Is there something wrong with the following approach?
Let $X_s = \{ ...
- 41
0
votes
0
answers
23
views
Determining number of observations be within confidence threshold
I have a model which has an accuracy of $A$. It makes a prediction with $Y$ confidence about a number of samples. Samples are observed many times, but not all are observed equally. So my data looks ...
- 51
1
vote
2
answers
80
views
A test to check whether two sets of proportions are _not different_
Is there a test to check whether two small sets of proportions are significantly not different?
For example, consider the two sets of proportions
...
- 5,294
0
votes
1
answer
1k
views
How to compare two time-series datasets?
I am trying to compare the financial performance of green bonds and conventional bonds by looking at their historical bond yields, bond prices etc. individually over a certain period of one year. I ...
CommunityBot
- 1
0
votes
1
answer
359
views
Not sure if i should be doing a type III anova
I was initially using the following anova formula in R
aov.out <- aov(column_name ~ core * ethnicity, data=mydata)
However i noticed that my dataset is quite unbalanced :
In light of that I tried ...
CommunityBot
- 1
1
vote
0
answers
36
views
Accounting for Retention Patterns in YouTube thumbnail A/B/C Testing
I am using the YouTube thumbnail A/B/C testing feature and encountering results that are implausible – I am trying to figure out why. One theory is that YouTube is not taking retention into account in ...
- 11
0
votes
1
answer
587
views
How to perform log-rank test correctly on IPTW weighted groups?
Thank you very much for your attetion!
I am working on an observation study with time-to-event data. The data has multiple covariates say V1-V5. I want to evaluate the treatment effect, so I used IPTW ...
CommunityBot
- 1
0
votes
1
answer
234
views
Comparing time series trends of non-linear count data across multiple firms in R
I have data on the number of patent applications per year for 20 years for ten different firms. I am trying to answer the question, "did the number (count) of applications evolve differentially across ...
CommunityBot
- 1
2
votes
1
answer
41
views
Is power of a test lower bounded by the significance level? [duplicate]
Here's an example that makes me ask this question. Let the significance level be $\alpha$.
Suppose we have a normal distribution with a known standard deviation $\sigma = 1$.
$H_0$: The mean of the ...
- 86.5k
2
votes
1
answer
344
views
How do I determine whether there is a significant difference between two sets of price data?
I am conducting a study where I am collecting prices from two groups of funeral homes in a particular state: in the first group, the homes I collected prices from are owned by families, and in the ...
CommunityBot
- 1
2
votes
1
answer
50
views
AB test design in eCommerce - group split per user and statistic aggregation per item
Is it statistically correct/viable to run an A/B test where the split into groups is per user (randomly assigned) and the statistic is aggregated per store item?
Lets narrow down the issue into a ...
- 5,294
2
votes
2
answers
515
views
test of significance for two groups of classification results
I will edit this question for more specific questions.
I want to predict the presence of multiple, binary questionnaire items with a classifier, such as SVM. I do this via 10 fold nested cross ...
CommunityBot
- 1
1
vote
1
answer
3k
views
How to interpret Jensen Alpha statistical significance?
When you regress portfolio excess returns against relative benchmark excess return you get a model in which the beta (slope) could be interpreted as the one you get from the CAPM, that is systemic ...
CommunityBot
- 1
2
votes
1
answer
711
views
How can I compare whether classifiers outcome is significantly different?
How can I compare whether classifiers outcome is significantly different?
I have a relatively small data set which I'm trying to classify. My training set consists of 24 items come from two different ...
- 10.3k
1
vote
1
answer
216
views
How to show that method X is more reliable than method Y?
I compute a statistic s using two methods, X and Y. In order to measure the reliability of X and Y, I found split half reliability measures (alpha) using a bootstraping technique. So, for each of ...
CommunityBot
- 1
2
votes
3
answers
79
views
Is there a way to determine if there is a statistically significant difference between measurements obtained by two different methods?
Here's my situation. I have 259 samples from a data set, and each sample has a value calculated by two different methods. Both methods are trying to describe the same phenomenon, and ideally when a ...
- 82.8k
0
votes
2
answers
229
views
Method for detecting significant increase in values?
I have a list of values which start and continue close to 0 before increasing towards the end (see attached plot of these values). I am interested in the value at which this increase starts - I could ...
- 17.2k
0
votes
1
answer
210
views
Statistical Test to test two sets of variables
If I have a two sets of variables and I want to find out if there is any significant relationship between them is there a statistical test to do this? e.g. Set 1: Columns A-D are paired data and Set 2:...
CommunityBot
- 1
1
vote
1
answer
280
views
How to test proportion of each factor level against mean proportion across all levels (binary outcome)
I have a dataset with the factor region (4 levels) and a binary variable outcome (0/1). Here in wide format.
...
CommunityBot
- 1
7
votes
3
answers
5k
views
Asymptotic power
I have found the term "asymptotic power of a statistical test" only related to the Kolmogorov-Smirnov test (to be precise: asyptotic power = 1). What does this term acctually mean? In my ...
- 82.8k
2
votes
1
answer
755
views
How to set an upper threshold for a property that is calculated from experimental results
I can't really place this problem theoretically, and I wonder if anyone could please point me to relevant posts/literature or provide some advice.
A property $X$ of a sample can be measured using ...
- 4,487
1
vote
2
answers
78
views
Does every finding in medical statistics need to be validated in an independent dataset?
I am performing a medical statistics analysis about 500 independent variables and 1 dependent variables. The sample size is 1200. The relation between the variables and outcomes are weak (say, diet ...
- 5,294
0
votes
1
answer
334
views
How to test if there's a statistically significant difference between two counts, based on relative percentile?
How to test if there's a statistically significant difference between two counts?
I'm struggling to test if two ratios on the same row of my dataset are significantly different from each other. I need ...
CommunityBot
- 1
1
vote
2
answers
329
views
Why do we calculate the family-wise Type-I error rate by assuming statistical independence of pair-wise comparisons?
$FWER = 1-\big(1-\alpha\big)^m$
The above formula is typically used when calculating the $FWER$ where alpha is the criterion applied by the researcher to each test and $m$ is the number of tests.
...
1
vote
2
answers
662
views
Comparing differences across conditions
I measured a variable during pre- and post-sessions and my research question is whether the change from pre to post is different across various conditions (e.g. (c=1,a=1), (c=2,a=1), (c=1,a=2), (c=2,a=...
CommunityBot
- 1
3
votes
1
answer
665
views
Comparing the AUC of two models by using a combination of nested cross-validation and bootstrapping
Main question:
I have an imbalanced binary labeled dataset (6% positive labels) and two different methods of training a predictive model for binary classification (e.g. Tree Model vs. Neural Network), ...
CommunityBot
- 1
1
vote
4
answers
3k
views
Direction of relationship in 2x2 contingency tables
I've been studying contingency table analysis and got to experiment a little bit with Fisher's exact test and Fisher's power test. Now I want to be able to determine the direction of the relationships....
- 101