# Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

4,754 questions
Filter by
Sorted by
Tagged with
11 views

### Can a predictor have a significant effect on non-signicant difference?

Let's say I estimate $d_t = b + \epsilon_t$, where $d$ stands for some difference between two variables and $b$ is a constant. I estimate this model $i$ times for $i$ different subjects (say firms, ...
8 views

### Statistically express differences in regional distribution of accident characterisitcs

I have a some data containing counts and shares of bicycle collisions per region and variable. All collisions have been reported in the timespan of one year. The regions do not have the same ...
20 views

### Parametric hypothesis testing for non-normal data

Are there any methods to make parametric hypothesis testing assuming that data is sampled from a known but non-normal continuous distribution? I'm glad to see a solution to any particular ...
12 views

### Get best performing feature

A data set that looks like this ...
29 views

### Statistical test for significant different between two temperature depth profile

I have two sea water temperature column from two different location and I want to test if they are significantly different. KS test and MANOVA test classify the water depth profile into different ...
14 views

### Correcting correlation coefficient significance (p-value) for dependent samples within Monte Carlo analysis

I would like to perform hypothesis testing on the significance of the correlation coefficient (Pearson's correlation coefficient) between variable $a$ and variable $b$. I do not have access to such ...
16 views

### detecting correlation between and significance of non-linear model parameters

I have a set of observed data y,obs that I am modelling using a non-linear model (currently in EXCEL though I also have a Python model). There are 4 input ...
8 views

### Statistically test differences of model performance on two different datasets

Suppose I have an SVM and a NB classifier that try to predict ownership of a particular text. For each of the algorithms, I train a model on 10 disjoint sets of training data. Then, I test them on ...
14 views

### FDR control in model prediction

I classified positives from negatives using, e.g., linear discriminant analysis (LDA), and perform false discovery rate (FDR) ...
13 views

### Why don't we consider pre-test scores when using one way ANOVA?

Newbie here. I'm trying to determine whether there are any statistically significant differences between the means of four independent groups as follows: ...
10 views

### LRT and Manually Finding Significance when Wilks Theorem isn't Valid

Hello and thank you for taking the time. I'm performing an LRT for a likelihood distribution which violates the regularity conditions for Wilks theorem and wald intervals. I'm running a monte-carlo ...
10 views

### What is the best method to found if results of different model have significant relationship or not?

I have 3 models data for estimating an objective (for example precipitation). My supervisor told me I need to check if the data of these three models have significantly similar or not using ...
59 views

### How can a hypothesis test be done for means of two groups?

I have managed to arrive to the following two tables, can a hypothesis test be used to test the significant difference between the means? EDIT: I am watching this video: https://www.youtube.com/...
12 views

### Achieved significance level (ASL) vs p-value

Statement I have came across the achieved significance level recently. The rejection of the null hypothesis in a permutation test that I read about was based on the ASL. Question This has made me ...
30 views

### Large sample size creating significant variables

I'm currently working with a large sample size (around 400,000 observations) where I did a t-test and the p-value turned out to be less than 0.001 for all variables. This ofcourse is not something ...
49 views

15 views

### What kind of test do i need to run to calculate the probability of this difference?

I have a sample of trials, with a positive result in some fraction 1/n trials and a predicted parameter of 1/p. What kind of test do I need to do to determine the probability that the difference ...
16 views

### Stuck on confidence interval analysis for GLM

I'm really new to R, and sort of stats in general. I've been using a GLM to analyse a series of data, which is basically tracking how concentrations of DNA change over time, when different treatments ...
25 views

### How to interpret p-values in terms of correlation?

I have one datamatrix and a single column(let's say it fvector). I had find the spearman's correlation of each column from datamatrix with fvector i had setted the parameters as p <0.05 and i got ...
4 views

### Minitab assistant (geometric distribution) [closed]

I have to do a homework for the university. I need to find the geometric distribution in an array without knowing a (probability) it can be found in Minitab ? For example there are some data : 2, 3, ...
30 views

### Comparing significance of variables between each other

I would like to determine the significance of my variables in my model and compare them with each other. I have 6 explanatory variables. Here is my summary of the model: ...
8 views

### Likert scale vs Dichotomous. Please suggest

As per my understanding dichotomous response (Agree vs Disagree) has much better Power than Likert scale, because it is possible to do Fisher's exact test on 2-way contingency table, while such tests ...
64 views

### Inter-rater agreement (raters >2) of non-random items

I ran a survey where participants (P) had to choose one category out of six (C1-C6) for each of five items. The data looks like this: ...
10 views

### Next step after building GLMM logistic regresion: LRT, ANOVA?

I am a biologist in the middle of their PhD journey and I'm constantly learning new things about statistics but there are some things that are difficult for me to understand. Until now I had dealt ...
16 views

I am analyzing a questionnaire survey data about online learning receptibility across different age groups. I have 4 age groups, and each has its % acceptance of the online learning susceptibility. ...
10 views

### percentage difference between two group when one group contains only positive cases

I have two groups $i=1,2$, with different total cases $n_1$ and $n_2$ on which the number of positive cases is $x_1<n_1$ and $x_2=n_2$. So there is no variability in the second group. I wonder how ...
6 views

### Why do we calculate the family-wise Type-I error rate by assuming statistical independence of pair-wise comparisons?

$FWER = 1-\big(1-\alpha\big)^m$ The above formula is typically used when calculating the $FWER$ where alpha is the criterion applied by the researcher to each test and $m$ is the number of tests. ...
17 views

### Comparing return distributions

Taking the log and/or normalizing (subtracting the mean and dividing by the standard deviation) stock returns still does not eliminate the dependence or heterogeneity. E.g. I cannot assume that each ...
20 views

### Choosing significance level if data less normal

If my data is not a very good approximation of normality, but I continue to run a t-test, will choosing a lower significance level offset some of the error caused by the fact that the data is not ...
2 views

### Why does one part of the interaction term turn insignificant when the interaction term is added to the model?

Suppose I am trying to figure out whether a bigger shoe-size makes you happy. In my model, I also control for gender. In model 1, there is no interaction term. All coefficients are significant. The ...
21 views

### Statistical methods to profile clusters

I am working on a clustering exercise and now I need to generate business insights by profiling the clusters(provide a brief description of each cluster. Eg : Cluster 1 mostly consists of high ...
13 views

### Choosing sample sizes when working with different sensitivity levels

I am currently ramping up a project to determine the rate of immunity in a population with SARS-CoV-2 using the new Roche Diagnostice test kits. Now, we will be testing people over the course of 6 ...
5 views

### Statistical test for ecpm metric

we ran a banner campaign, and we have been ask to analyse the results based on ecpm for 2 different groups. Ecpm is: (revenue/total impressions)*1000 They didn't set up an a/b test so the 2 groups of ...
9 views

### SPSS finding the significands between age_group and primary device of choice?

I am trying to find a correlation between age group and primary device of choice, and then also compute a statistical significant. Here's my output in SPSS, I have grouped the data in this way: ...
22 views

### Is t-test for coefficients of linear regression two tailed?

What is the type of t-test used in the significance testing of coefficients/weights of linear regression? Is it one tailed or two tailed?
7 views

### Effect size to use in Table One

I am currently doing an observational study, cross-sectional study comparing diseased and non-diseased patients. The sample size is rather big at 40K observations and when I used t-test for continuous ...
5 views

### May I use t-test for time serial dataset compare when residual normality has confirmed?

I have a question about time serial data analysis. I got data which, observed population change through time points. (From T2 ~ T12) What I want is that, comparing population proportion changes ...
21 views

### Testing for potential discrimination

Apologies, I know this is incredibly simple but it's been a while since I was in school! I'd like to run a significance test on the (dummy) data in the table attached to ascertain whether there has ...
18 views

### nonparametric tests for random forest regression

Ive built a random forest model which predicts temperature, humidity and energy based on 5 inputs. I obtain an R2 value of 0.92, CV(RMSE) of 14, and errors of 3.51%, 4.22%, 7.23% for each variable ...
13 views

### Interpretation when Information Criteria and F-Test Disagree

I used SPSS (v. 25) conducted a pair of nested multtlevel models (aka linear mixed models) where the outcome criterion is itself also nested within participants. I followed Singer and Willett's (2003) ...
14 views

### Calculating the optimal sample size before running a test

I want to run an experiment to test the outcomes of variations A and B. I'm designing the experiment but I want to know what's the minimum sample size to obtain a result of 10% difference between ...
34 views

### Simple algorithm to detect change point in time series

Apologies if this question has already been asked, but a lot of similar questions are regarding R or complex algorithms that I don't want. I have numerous 2-dimensional time series, eacch plotting ...
13 views

### goodness of fit statistic independant of the fixed effects

I am using glm model to fit data. I want to check if my fit is good. My model is usually the classical model with only fixed effects. In order to do so, I use goodness of fit tools like residual ...
19 views

### Analyzing collapsed data for statistical significance

What is the proper way to analyze data that is only available in a collapsed format for statistical significance? I am dealing with a dataset that contains at the observation level the arithmetic sum ...
104 views

### Testing of hypothesis for the linearity of a data? PCA suggested, but how do we design a statistical test using it? [closed]

Suppose we're given the data set $\{x_1 \dots x_n\}$ in $\mathbb{R}^D$ the $D$-dimensional Euclidean space, and assume this data has intrinsic dimension $d < D.$ N.B. this just means that data is ...
18 views

### Interpreting overlap of bootstrapped confidence intervals

Assuming two samples of numeric values for two groups of unequal group sizes (e.g. 100 opinion scores collected from group A and 15 opinion scores collected from group B), I understand that non-...
9 views

### Justifying the judgement with a systematic approach/theories

I would like to give a context of what I did. 1) Let's say there are two dictionaries (dict A and dict B) each containing a list of words/terms as shown below. 2) Now my task is to find matching ...
30 views

### Statistical Testing of Non-linear Model Parameters

I have produced three non-linear 4-parameter logistic curves based on some experimental observations using three different measurement devices. Each curve shows the measurement of density decline ...