A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

# Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

4,466 questions
Filter by
Sorted by
Tagged with
23 views

### Benjamini-Hochberg procedure with equally ranked p-values

I struggling with Benjamini-Hochberg correction due to equally ranked p-values and I would gladly appreciate some help. My problem is as follow: I've got 11 comparisons to do and the threshold is set ...
8 views

### What test to use to measure the effectivity of a filtration device? [closed]

I'm researching on the effectivity of a filtration device that we created for a group project. I'm wondering what statistical test to use. Our test measured these parameters before the water is ...
16 views

### Statistical Signifance Test for Regression Models

I have two systems A and B both of which are regression models. I predict N items, using system A and system B using the same query points (X). I want to check if system A comes from the same ...
29 views

### Interpret Interaction Effects in Negative Binomial Regression

I have two questions regarding the interpretation of interaction effects in a negative binomial regression model. The model uses the number of people arriving at a hotel per hour as the dependent ...
21 views

### What statistical significance test should I use in this situation with two variables?

I have two variables: Wind speed (MPH) and passing yards in the NFL. I want to test if wind speed has a statistically significant effect on a player's passing yards. I used independent t-test from ...
37 views

### What does X²(1, N=100) =6.83, p =0.009 mean?

I am reading scientific research (Link) and stumbled upon the following notation: X²(1, N=100)=6.83, p =0.009. An explanation is given as follows: ...
26 views

### Which statistical test : Anova or T-Test for between subject experiment (repeated measures ) with one IV with two levels( control and treatment)

It's a between-subject design with a control group and a treatment group with 50 people each. Each participant undergoes 100 trails. (repeated measures). So the only within-subject factor is the ...
19 views

### Tests for significant difference between prediction errors of machine learning regression models

Are there any recommended statistical tests for significant differences between cross-validation (CV) estimates of prediction error of machine learning regression models? I have 5 different random ...
19 views

### comparing two regression coefficients obtained by same elastic net regression model

I have 10 responses and 20 predictors for which I measured values in two conditions. I ran elastic model for each response at each condition separately. As a result, I will have two association ...
17 views

### statistic significance of proportions

Let's assume someone buys a bag of smarties, and finds that it includes 22 brown candies 19 red candies 16 yellow candies 15 blue candies 8 green candies How can we use a test of significance to ...
25 views

### Which effects and which interactions are significant?

I'm performing an experiment with five main effects resp. 6-7 since these main effects influences are studied on 20 sensors (which can be grouped into 3 ...
148 views
+50

### Statistical test with violation of independence assumption

My objective is Checking of the significance of differences of two linear regression methods on one database based on the robustness measure I used (which I called robustness as only a name). The ...
26 views

### Explainability of a model

I am trying to answer this question: This question is actually based on the Mackay plot here: the simpler the model, the more it places its mass towards the left However I'm not quite able to arrive ...
7 views

### Backward feature elimination removes significant one-sided associations?

I’m fitting data to a linear regression model. In the end, I want a model that only includes significant IVs with p values <= 0.05, so I’m looping through and at each iteration removing the IV with ...
13 views

### How to compute Net Reclassification Improvement or Alternatives?

I am working on a binary classification problem with ~5k records and class proportion of 33:67. I have 60 features/variables in my dataset and finally I have come to about 10 variables based on ...
17 views

### Valid significance testing of coefficients when performing model selection

I want to know how I can validly perform permutation (or bootstrap) significance testing of model coefficients when I am also using the same dataset to perform model selection. My model is not the ...
32 views

### How to compare and evaluate models for a new feature?

I am working on a binary classification where I have 4712 records with Label 1 being 1554 records and Label 0 being 3558 records. When I tried multiple models based on 6,7 and 8 features, I see the ...
102 views

### Can the null hypothesis of a t-test of “no significant difference” be used as a result?

I am not so familiar with statistics, since I am a computer science student. I hope it is alright to ask this question here. I am currently working towards the end of a thesis. I compared the User ...
10 views

### Intuition behind null model for disparity filter algorithm

I'm trying to better understand the disparity filter algorithm for weighted networks: Original paper: https://www.pnas.org/content/106/16/6483 Wikipedia link: https://en.wikipedia.org/wiki/...
12 views

### Statistical Test for Staffing Agency Selection of Employees

I am looking for the statistical test to measure the selection pattern from a pool of people. The staffing agency receives requests from a contract and then selects a person to fill the order. The ...
23 views

### Which statistical tests can I use to analyse significant difference between the means of non-random samples? [duplicate]

I want to compare the means of non-random samples to population mean. However, most standard tests (eg. t-tests, ANOVA, Welch test etc.) are based on the assumption that samples are randomly obtained ...
21 views

### Statistical significant of average time series shapes

Let's say we have a number of time series that come from different observations of related events. For example, 10 similar events centred around the end of a quarter describing measures taken the 100 ...
35 views

### Hypothesis testing for apriori market basket analysis

I'm wondering if there is a possibility to perform hypothesis tests to show that the resulting rules from an apriori market basket analysis are statistically significant? Maybe I'm missing the forest ...
23 views

### significant evolution

I know the number of customers who come every day into a shop. I need to calculate the evolution of the number of customers over two year (2801 customers in 2018, 2999 customers in 2019 -> 7% of ...
128 views

### A bit confused on p values on ttest in Stata output

In Stata the output of a ttest shows p values as: Pr(T<t), Pr(|T|>|t|) and Pr(T>t) I understand the two sides to be the tails of the test and the middle ...
14 views

### Chi Square vs Spearman correlation

I have a set of 2 ordinal categorial variables and I want to see the relationship between them. Chi square shows a significant value while the spearman correlation is non significant. What does it ...
15 views

### How to transform two non-normal samples to perform parametric test (t test)?

I have two samples of different size (Ratio = 1:12) which are non-normal in nature. The samples are sensor values from Machine working in normal condition and when it is about to fail. I have ...
12 views

### Can I assume data points follow Logistic Distribution with distances from the decision boundary in Logistic Regression?

While I was doing some researches on Logistic Regression, I read a post saying that distances from the decision boundary in Logistic Regression follow Logistic Distribution. I am not sure if the ...
24 views

### The way to evaluate level of significance for coefficient of eta correlation?

How I can check the significance level of eta correlation? To assay to see the significance of Pearson correlation, we compare the coefficient with r table, but for eta correlation what can I do?
16 views

### Paired T-Test Question

If you have two paired groups (group A and B) and want to determine the difference in the two means, you would use a t test for two related populations. However, what if you want to determine the % ...
44 views

### Why and how to match variables using logistic regression?

I have a dataset of ~4.7K records focused on binary classification with 60 features. class 1 is of 1554 records and class 2 is ...
69 views

### What to report when p-values of Standardized and Unstandardized Beta disagree?

I am testing the following models. Volumes were purposely log10 transformed to examine brain allometry. ...
46 views

### R Displaying Incorrect Significance Level?

In the following output, predictor variable has a p-value of 0.001809 but is indicated to be significant at the 0.001 level as it has '**' displayed next to it. Surely given that the p-value is ...
224 views

### What should be statistically significant game score in football and basketball to say that one team stronger?

4 teams play 2 games: football and basketball. (2 games just to better understand theory) Final result of a football game is 2:1 Final result of basketball game is 101:96 What should be the result ...
43 views

### When to drop correlated features?

I have a dataset focused on binary classification with 60 features and 5k records. Am trying to 1) find the risk factors using statsmodel logistic regression (I do this because it's important to ...
9 views

### Assessing the significance of a change in two surveys

I have the following case: My company hired a company to conduct two surveys each time asking about 800 people whether they know my company or not (binary result). The first survey was conducted ...
18 views

### control variable, industry

I'm doing a master thesis on the effect of CSR contracting on CSR performance. My model is based on a multiples regression by OLS. I have regressed CSR level on CSR contracting and used several ...
25 views

### How to select best feature set?

I am currently using feature selection approaches like filter, wrapper, embedded etc. All these methods give different set of features and I rank them based on their frequency of occurrence in other ...
19 views

### Do MCAT scores really predict med school performance?

I am looking for correlations between MCAT data and performance in medical school. I am given MCAT scores from several hundred anonymized students and their final grades in five classes in medical ...
25 views

### Assumptions for Univariate Analysis

I am working a binary classification problem. To identify the risk factors, I would first like to do univariate analysis and then do ...
30 views

### How to generate Odds Ratio table using Python? [closed]

I am working on a binary classification problem and am trying to do univariate analysis to get their p-values and risk ratios. ...
15 views

### What is accuracy rejection curve

I am trying to understand accuracy rejection curve from this paper . It defines ARC as "An accuracy rejection curve (ARC)is a function representing the accuracy of a classifier as a function of its ...
754 views

### Why significant variable doesn't improve model performance?

I have a binary classification problem with 5K records and 60+ features/columns/variables. dataset is slightly imbalanced (or not) with 33:67 class proportion What I did was 1st) Run a logistic ...
27 views

### Sample size estimation with known mean (optimal value), unknown stdv

quick question on sample size estimation. Let's say i play a game of poker, where i have 2 options, play or don't play. In the long run, the optimal strategy is to play 15% of all times, dependent on ...
414 views

### How can the F-test reject the null hypothesis while the KS test does not?

Suppose I have two datasets, $\mathbf{a}$ and $\mathbf{b}$. I want to test whether the two datasets are different in a statistically significant way. To compute the F-test, I take the ratio of the ...
14 views

### SPSS Binary Logistic Regression - intercept (baseline) model not significant

I am working with a binary logistic regression in SPSS and my current issue is that my baseline model is not significant (0.440) (Block 0 "Variables in the Equation), my Omnibus-test however is and ...
6 views

### Hoeffding-like bound when drawing clusters from a global distribution

Imagine you have some large global population which you would like to make measurements about. This population is grouped into some large number of clusters. Lets say it is infeasible to sample ...
27 views

### Validate hypotheses from individual questions

A survey was conducted that includes 3 questions related to the same hypothesis that we want to validate. These 3 questions are essentially the same question, formulated in three different ways. They ...