# Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

4,752 questions
Filter by
Sorted by
Tagged with
8 views

### Correcting correlation coefficient significance (p-value) for dependent samples within Monte Carlo analysis

I would like to perform hypothesis testing on the significance of correlation coefficient (Pearson's correlation coefficient) between variable $a$ and variable $b$. I do not have access to such ...
4 views

### detecting correlation between and significance of non-linear model parameters

I have a set of observed data y,obs that I am modelling using a non-linear model (currently in EXCEL though I also have a Python model). There are 4 input ...
6 views

### Statistically test differences of model performance on two different datasets

Suppose I have an SVM and a NB classifier that try to predict ownership of a particular text. For each of the algorithms, I train a model on 10 disjoint sets of training data. Then, I test them on ...
13 views

### FDR control in model prediction

I classified positives from negatives using, e.g., linear discriminant analysis (LDA), and perform false discovery rate (FDR) ...
13 views

### Why don't we consider pre-test scores when using one way ANOVA?

Newbie here. I'm trying to determine whether there are any statistically significant differences between the means of four independent groups as follows: ...
10 views

### LRT and Manually Finding Significance when Wilks Theorem isn't Valid

Hello and thank you for taking the time. I'm performing an LRT for a likelihood distribution which violates the regularity conditions for Wilks theorem and wald intervals. I'm running a monte-carlo ...
10 views

### What is the best method to found if results of different model have significant relationship or not?

I have 3 models data for estimating an objective (for example precipitation). My supervisor told me I need to check if the data of these three models have significantly similar or not using ...
56 views

### How can a hypothesis test be done for means of two groups?

I have managed to arrive to the following two tables, can a hypothesis test be used to test the significant difference between the means? EDIT: I am watching this video: https://www.youtube.com/...
11 views

### Achieved significance level (ASL) vs p-value

Statement I have came across the achieved significance level recently. The rejection of the null hypothesis in a permutation test that I read about was based on the ASL. Question This has made me ...
26 views

### Large sample size creating significant variables

I'm currently working with a large sample size (around 400,000 observations) where I did a t-test and the p-value turned out to be less than 0.001 for all variables. This ofcourse is not something ...
48 views

15 views

### What kind of test do i need to run to calculate the probability of this difference?

I have a sample of trials, with a positive result in some fraction 1/n trials and a predicted parameter of 1/p. What kind of test do I need to do to determine the probability that the difference ...
16 views

### Stuck on confidence interval analysis for GLM

I'm really new to R, and sort of stats in general. I've been using a GLM to analyse a series of data, which is basically tracking how concentrations of DNA change over time, when different treatments ...
24 views

### How to interpret p-values in terms of correlation?

I have one datamatrix and a single column(let's say it fvector). I had find the spearman's correlation of each column from datamatrix with fvector i had setted the parameters as p <0.05 and i got ...
4 views

### Minitab assistant (geometric distribution) [closed]

I have to do a homework for the university. I need to find the geometric distribution in an array without knowing a (probability) it can be found in Minitab ? For example there are some data : 2, 3, ...
29 views

### Comparing significance of variables between each other

I would like to determine the significance of my variables in my model and compare them with each other. I have 6 explanatory variables. Here is my summary of the model: ...
8 views

### Likert scale vs Dichotomous. Please suggest

As per my understanding dichotomous response (Agree vs Disagree) has much better Power than Likert scale, because it is possible to do Fisher's exact test on 2-way contingency table, while such tests ...
63 views

### Inter-rater agreement (raters >2) of non-random items

I ran a survey where participants (P) had to choose one category out of six (C1-C6) for each of five items. The data looks like this: ...
10 views

### Next step after building GLMM logistic regresion: LRT, ANOVA?

I am a biologist in the middle of their PhD journey and I'm constantly learning new things about statistics but there are some things that are difficult for me to understand. Until now I had dealt ...
16 views

I am analyzing a questionnaire survey data about online learning receptibility across different age groups. I have 4 age groups, and each has its % acceptance of the online learning susceptibility. ...
10 views

### percentage difference between two group when one group contains only positive cases

I have two groups $i=1,2$, with different total cases $n_1$ and $n_2$ on which the number of positive cases is $x_1<n_1$ and $x_2=n_2$. So there is no variability in the second group. I wonder how ...
6 views

### Why do we calculate the family-wise Type-I error rate by assuming statistical independence of pair-wise comparisons?

$FWER = 1-\big(1-\alpha\big)^m$ The above formula is typically used when calculating the $FWER$ where alpha is the criterion applied by the researcher to each test and $m$ is the number of tests. ...
16 views

### Comparing return distributions

Taking the log and/or normalizing (subtracting the mean and dividing by the standard deviation) stock returns still does not eliminate the dependence or heterogeneity. E.g. I cannot assume that each ...
20 views

### Choosing significance level if data less normal

If my data is not a very good approximation of normality, but I continue to run a t-test, will choosing a lower significance level offset some of the error caused by the fact that the data is not ...
2 views

### Why does one part of the interaction term turn insignificant when the interaction term is added to the model?

Suppose I am trying to figure out whether a bigger shoe-size makes you happy. In my model, I also control for gender. In model 1, there is no interaction term. All coefficients are significant. The ...
20 views

### Statistical methods to profile clusters

I am working on a clustering exercise and now I need to generate business insights by profiling the clusters(provide a brief description of each cluster. Eg : Cluster 1 mostly consists of high ...
13 views

### Choosing sample sizes when working with different sensitivity levels

I am currently ramping up a project to determine the rate of immunity in a population with SARS-CoV-2 using the new Roche Diagnostice test kits. Now, we will be testing people over the course of 6 ...
5 views

### Statistical test for ecpm metric

we ran a banner campaign, and we have been ask to analyse the results based on ecpm for 2 different groups. Ecpm is: (revenue/total impressions)*1000 They didn't set up an a/b test so the 2 groups of ...
9 views

### SPSS finding the significands between age_group and primary device of choice?

I am trying to find a correlation between age group and primary device of choice, and then also compute a statistical significant. Here's my output in SPSS, I have grouped the data in this way: ...
22 views

### Is t-test for coefficients of linear regression two tailed?

What is the type of t-test used in the significance testing of coefficients/weights of linear regression? Is it one tailed or two tailed?
7 views

### Effect size to use in Table One

I am currently doing an observational study, cross-sectional study comparing diseased and non-diseased patients. The sample size is rather big at 40K observations and when I used t-test for continuous ...
5 views

### May I use t-test for time serial dataset compare when residual normality has confirmed?

I have a question about time serial data analysis. I got data which, observed population change through time points. (From T2 ~ T12) What I want is that, comparing population proportion changes ...
20 views

### Testing for potential discrimination

Apologies, I know this is incredibly simple but it's been a while since I was in school! I'd like to run a significance test on the (dummy) data in the table attached to ascertain whether there has ...
18 views

### nonparametric tests for random forest regression

Ive built a random forest model which predicts temperature, humidity and energy based on 5 inputs. I obtain an R2 value of 0.92, CV(RMSE) of 14, and errors of 3.51%, 4.22%, 7.23% for each variable ...
13 views

### Interpretation when Information Criteria and F-Test Disagree

I used SPSS (v. 25) conducted a pair of nested multtlevel models (aka linear mixed models) where the outcome criterion is itself also nested within participants. I followed Singer and Willett's (2003) ...
14 views

### Calculating the optimal sample size before running a test

I want to run an experiment to test the outcomes of variations A and B. I'm designing the experiment but I want to know what's the minimum sample size to obtain a result of 10% difference between ...
34 views

### Simple algorithm to detect change point in time series

Apologies if this question has already been asked, but a lot of similar questions are regarding R or complex algorithms that I don't want. I have numerous 2-dimensional time series, eacch plotting ...
13 views

### goodness of fit statistic independant of the fixed effects

I am using glm model to fit data. I want to check if my fit is good. My model is usually the classical model with only fixed effects. In order to do so, I use goodness of fit tools like residual ...
19 views

### Analyzing collapsed data for statistical significance

What is the proper way to analyze data that is only available in a collapsed format for statistical significance? I am dealing with a dataset that contains at the observation level the arithmetic sum ...
104 views

### Testing of hypothesis for the linearity of a data? PCA suggested, but how do we design a statistical test using it? [closed]

Suppose we're given the data set $\{x_1 \dots x_n\}$ in $\mathbb{R}^D$ the $D$-dimensional Euclidean space, and assume this data has intrinsic dimension $d < D.$ N.B. this just means that data is ...
18 views

### Interpreting overlap of bootstrapped confidence intervals

Assuming two samples of numeric values for two groups of unequal group sizes (e.g. 100 opinion scores collected from group A and 15 opinion scores collected from group B), I understand that non-...
9 views

### Justifying the judgement with a systematic approach/theories

I would like to give a context of what I did. 1) Let's say there are two dictionaries (dict A and dict B) each containing a list of words/terms as shown below. 2) Now my task is to find matching ...
30 views

### Statistical Testing of Non-linear Model Parameters

I have produced three non-linear 4-parameter logistic curves based on some experimental observations using three different measurement devices. Each curve shows the measurement of density decline ...
17 views

### How to determine if updating a variable will result in a target output within some confidence interval?

I'm trying to optimize a manufacturing process for work, and was stuck when trying to estimate our confidence in a settings update procedure. We are trying to configure an instrument setting (x) from ...
26 views

### Why I can't do an ANOVA on PC-scores? [closed]

I have recently received the critique on a submitted manuscript that it is problematic that I did multiple (non-independent) testing on the pc-scores of a PCA analysis using an ANOVA. I did an ...
13 views

### Bootstrapping a RMSE

I am trying to compute a bootstrapped distribution of the root mean squared errors to 1 of a distribution "A". However, I am not sure how to do this. My approach was to resample the distribution "A" ...
13 views

### Test for significant differences for Temperature and Salinity obtained from depth profiles [closed]

I’m working on modeling the fish catch with the physical oceanographic variables. Trying to find the relation between the catch and the physical oceanographic variables (Temperature and Salinity). ...
54 views

### Multiple variables significant

I'm working on a regression of the price of the car compared to other factors. I can upload the data here but unsure it's necessary. The summary gives me: ...