A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

Filter by
Sorted by
Tagged with
3
votes
1answer
23 views

Benjamini-Hochberg procedure with equally ranked p-values

I struggling with Benjamini-Hochberg correction due to equally ranked p-values and I would gladly appreciate some help. My problem is as follow: I've got 11 comparisons to do and the threshold is set ...
0
votes
0answers
8 views

What test to use to measure the effectivity of a filtration device? [closed]

I'm researching on the effectivity of a filtration device that we created for a group project. I'm wondering what statistical test to use. Our test measured these parameters before the water is ...
1
vote
0answers
16 views

Statistical Signifance Test for Regression Models

I have two systems A and B both of which are regression models. I predict N items, using system A and system B using the same query points (X). I want to check if system A comes from the same ...
1
vote
0answers
29 views

Interpret Interaction Effects in Negative Binomial Regression

I have two questions regarding the interpretation of interaction effects in a negative binomial regression model. The model uses the number of people arriving at a hotel per hour as the dependent ...
0
votes
1answer
21 views

What statistical significance test should I use in this situation with two variables?

I have two variables: Wind speed (MPH) and passing yards in the NFL. I want to test if wind speed has a statistically significant effect on a player's passing yards. I used independent t-test from ...
3
votes
1answer
37 views

What does X²(1, N=100) =6.83, p =0.009 mean?

I am reading scientific research (Link) and stumbled upon the following notation: X²(1, N=100)=6.83, p =0.009. An explanation is given as follows: ...
2
votes
1answer
26 views

Which statistical test : Anova or T-Test for between subject experiment (repeated measures ) with one IV with two levels( control and treatment)

It's a between-subject design with a control group and a treatment group with 50 people each. Each participant undergoes 100 trails. (repeated measures). So the only within-subject factor is the ...
1
vote
0answers
19 views

Tests for significant difference between prediction errors of machine learning regression models

Are there any recommended statistical tests for significant differences between cross-validation (CV) estimates of prediction error of machine learning regression models? I have 5 different random ...
0
votes
1answer
19 views

comparing two regression coefficients obtained by same elastic net regression model

I have 10 responses and 20 predictors for which I measured values in two conditions. I ran elastic model for each response at each condition separately. As a result, I will have two association ...
0
votes
0answers
17 views

statistic significance of proportions

Let's assume someone buys a bag of smarties, and finds that it includes 22 brown candies 19 red candies 16 yellow candies 15 blue candies 8 green candies How can we use a test of significance to ...
1
vote
0answers
25 views

Which effects and which interactions are significant?

I'm performing an experiment with five main effects resp. 6-7 since these main effects influences are studied on 20 sensors (which can be grouped into 3 ...
3
votes
2answers
148 views
+50

Statistical test with violation of independence assumption

My objective is Checking of the significance of differences of two linear regression methods on one database based on the robustness measure I used (which I called robustness as only a name). The ...
2
votes
0answers
26 views

Explainability of a model

I am trying to answer this question: This question is actually based on the Mackay plot here: the simpler the model, the more it places its mass towards the left However I'm not quite able to arrive ...
1
vote
0answers
7 views

Backward feature elimination removes significant one-sided associations?

I’m fitting data to a linear regression model. In the end, I want a model that only includes significant IVs with p values <= 0.05, so I’m looping through and at each iteration removing the IV with ...
0
votes
0answers
13 views

How to compute Net Reclassification Improvement or Alternatives?

I am working on a binary classification problem with ~5k records and class proportion of 33:67. I have 60 features/variables in my dataset and finally I have come to about 10 variables based on ...
0
votes
0answers
17 views

Valid significance testing of coefficients when performing model selection

I want to know how I can validly perform permutation (or bootstrap) significance testing of model coefficients when I am also using the same dataset to perform model selection. My model is not the ...
1
vote
1answer
32 views

How to compare and evaluate models for a new feature?

I am working on a binary classification where I have 4712 records with Label 1 being 1554 records and Label 0 being 3558 records. When I tried multiple models based on 6,7 and 8 features, I see the ...
4
votes
2answers
102 views

Can the null hypothesis of a t-test of “no significant difference” be used as a result?

I am not so familiar with statistics, since I am a computer science student. I hope it is alright to ask this question here. I am currently working towards the end of a thesis. I compared the User ...
2
votes
0answers
10 views

Intuition behind null model for disparity filter algorithm

I'm trying to better understand the disparity filter algorithm for weighted networks: Original paper: https://www.pnas.org/content/106/16/6483 Wikipedia link: https://en.wikipedia.org/wiki/...
0
votes
1answer
12 views

Statistical Test for Staffing Agency Selection of Employees

I am looking for the statistical test to measure the selection pattern from a pool of people. The staffing agency receives requests from a contract and then selects a person to fill the order. The ...
1
vote
1answer
23 views

Which statistical tests can I use to analyse significant difference between the means of non-random samples? [duplicate]

I want to compare the means of non-random samples to population mean. However, most standard tests (eg. t-tests, ANOVA, Welch test etc.) are based on the assumption that samples are randomly obtained ...
2
votes
0answers
21 views

Statistical significant of average time series shapes

Let's say we have a number of time series that come from different observations of related events. For example, 10 similar events centred around the end of a quarter describing measures taken the 100 ...
1
vote
1answer
35 views

Hypothesis testing for apriori market basket analysis

I'm wondering if there is a possibility to perform hypothesis tests to show that the resulting rules from an apriori market basket analysis are statistically significant? Maybe I'm missing the forest ...
0
votes
0answers
23 views

significant evolution

I know the number of customers who come every day into a shop. I need to calculate the evolution of the number of customers over two year (2801 customers in 2018, 2999 customers in 2019 -> 7% of ...
5
votes
1answer
128 views

A bit confused on p values on ttest in Stata output

In Stata the output of a ttest shows p values as: Pr(T<t), Pr(|T|>|t|) and Pr(T>t) I understand the two sides to be the tails of the test and the middle ...
1
vote
1answer
14 views

Chi Square vs Spearman correlation

I have a set of 2 ordinal categorial variables and I want to see the relationship between them. Chi square shows a significant value while the spearman correlation is non significant. What does it ...
0
votes
0answers
15 views

How to transform two non-normal samples to perform parametric test (t test)?

I have two samples of different size (Ratio = 1:12) which are non-normal in nature. The samples are sensor values from Machine working in normal condition and when it is about to fail. I have ...
0
votes
0answers
12 views

Can I assume data points follow Logistic Distribution with distances from the decision boundary in Logistic Regression?

While I was doing some researches on Logistic Regression, I read a post saying that distances from the decision boundary in Logistic Regression follow Logistic Distribution. I am not sure if the ...
0
votes
1answer
24 views

The way to evaluate level of significance for coefficient of eta correlation?

How I can check the significance level of eta correlation? To assay to see the significance of Pearson correlation, we compare the coefficient with r table, but for eta correlation what can I do?
0
votes
0answers
16 views

Paired T-Test Question

If you have two paired groups (group A and B) and want to determine the difference in the two means, you would use a t test for two related populations. However, what if you want to determine the % ...
3
votes
1answer
44 views

Why and how to match variables using logistic regression?

I have a dataset of ~4.7K records focused on binary classification with 60 features. class 1 is of 1554 records and class 2 is ...
2
votes
1answer
69 views

What to report when p-values of Standardized and Unstandardized Beta disagree?

I am testing the following models. Volumes were purposely log10 transformed to examine brain allometry. ...
1
vote
1answer
46 views

R Displaying Incorrect Significance Level?

In the following output, predictor variable has a p-value of 0.001809 but is indicated to be significant at the 0.001 level as it has '**' displayed next to it. Surely given that the p-value is ...
4
votes
2answers
224 views

What should be statistically significant game score in football and basketball to say that one team stronger?

4 teams play 2 games: football and basketball. (2 games just to better understand theory) Final result of a football game is 2:1 Final result of basketball game is 101:96 What should be the result ...
1
vote
1answer
43 views

When to drop correlated features?

I have a dataset focused on binary classification with 60 features and 5k records. Am trying to 1) find the risk factors using statsmodel logistic regression (I do this because it's important to ...
0
votes
0answers
9 views

Assessing the significance of a change in two surveys

I have the following case: My company hired a company to conduct two surveys each time asking about 800 people whether they know my company or not (binary result). The first survey was conducted ...
0
votes
0answers
18 views

control variable, industry

I'm doing a master thesis on the effect of CSR contracting on CSR performance. My model is based on a multiples regression by OLS. I have regressed CSR level on CSR contracting and used several ...
2
votes
1answer
25 views

How to select best feature set?

I am currently using feature selection approaches like filter, wrapper, embedded etc. All these methods give different set of features and I rank them based on their frequency of occurrence in other ...
0
votes
1answer
19 views

Do MCAT scores really predict med school performance?

I am looking for correlations between MCAT data and performance in medical school. I am given MCAT scores from several hundred anonymized students and their final grades in five classes in medical ...
1
vote
0answers
25 views

Assumptions for Univariate Analysis

I am working a binary classification problem. To identify the risk factors, I would first like to do univariate analysis and then do ...
0
votes
1answer
30 views

How to generate Odds Ratio table using Python? [closed]

I am working on a binary classification problem and am trying to do univariate analysis to get their p-values and risk ratios. ...
0
votes
0answers
15 views

What is accuracy rejection curve

I am trying to understand accuracy rejection curve from this paper . It defines ARC as "An accuracy rejection curve (ARC)is a function representing the accuracy of a classifier as a function of its ...
4
votes
3answers
754 views

Why significant variable doesn't improve model performance?

I have a binary classification problem with 5K records and 60+ features/columns/variables. dataset is slightly imbalanced (or not) with 33:67 class proportion What I did was 1st) Run a logistic ...
2
votes
0answers
27 views

Sample size estimation with known mean (optimal value), unknown stdv

quick question on sample size estimation. Let's say i play a game of poker, where i have 2 options, play or don't play. In the long run, the optimal strategy is to play 15% of all times, dependent on ...
7
votes
2answers
414 views

How can the F-test reject the null hypothesis while the KS test does not?

Suppose I have two datasets, $\mathbf{a}$ and $\mathbf{b}$. I want to test whether the two datasets are different in a statistically significant way. To compute the F-test, I take the ratio of the ...
0
votes
1answer
14 views

SPSS Binary Logistic Regression - intercept (baseline) model not significant

I am working with a binary logistic regression in SPSS and my current issue is that my baseline model is not significant (0.440) (Block 0 "Variables in the Equation), my Omnibus-test however is and ...
0
votes
0answers
6 views

Hoeffding-like bound when drawing clusters from a global distribution

Imagine you have some large global population which you would like to make measurements about. This population is grouped into some large number of clusters. Lets say it is infeasible to sample ...
0
votes
1answer
27 views

Validate hypotheses from individual questions

A survey was conducted that includes 3 questions related to the same hypothesis that we want to validate. These 3 questions are essentially the same question, formulated in three different ways. They ...
1
vote
0answers
18 views

How to make sense of logit summary output?

I ran a logit model using statsmodel api available in Python. I have few questions on how to make sense of these. What's the difference between summary and summary2 output? Why is the AIC and BIC ...
1
vote
1answer
40 views

how is confounders different from interaction terms?

Let me define the problem space. I am working a binary classification problem. I am trying to build a causal model as well as predictive model. My aim is to find list of significant features (based ...