Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

4
votes
1answer
143 views

What does “statistically insignificantly worse” mean?

I am reviewing a paper where I found this sentence repeatedly: A is statistically insignificantly worse than B I consider this to be confusing since, for me at least, it is not immediately clear ...
1
vote
0answers
15 views

Significance testing between two sets of Pearson's correlation coefficients

I have been reading through questions here to find an answer to my question but didn't find a satisfactory one. Here is my scenario: I have a system computing an algorithm on each of 100 units (so ...
1
vote
1answer
23 views

Testing the hypothesis on clustering

I have a number of samples. For each, there is a time course of multivariate data defined, with $t$ timepoints ($t < 50$) and $n$ variables ($n > 100$). We have noted that the time courses of a ...
1
vote
2answers
57 views

Test if two binomial distributions are statistically different from each other

I have three groups of data, each with a binomial distribution (i.e. each group has elements that are either success or failure). I do not have a predicted probability of success, but instead can ...
2
votes
1answer
87 views

Can we skip the lower order terms in interactions? [duplicate]

This question is about three-way interaction and the possibility of applying without second lower terms with keeping the main variables in the equation not like the other questions. In fact the other ...
0
votes
0answers
18 views

Imputation model: Pooled model is insignificant. How to interpret?

I have ordinal data on three IVs ranging from 1 to 5 as below: IV1: Not at all Important - Very Important IV2: Not at all Satisfied - Very Satisfied IV3: Performs much Worse - Performs much better ...
1
vote
0answers
27 views

How to test for a home bias in a data set?

I'm trying to think how I can test a data set I have, for transactions between buyers and sellers, for a home bias. My hypothesis is that buyers would rather buy from a merchant from their home ...
1
vote
0answers
40 views

Which post-hoc comparison to use and why

Hopefully this is now a much clearer picture of the question I want to answer: For each of 60 different plans I ran 35 simulation replications of my Agent Based Model (ABM) simulation. Then for each ...
1
vote
1answer
30 views

How can a distribution of cross-validated $R^2$ scores be used to determine whether one model is significantly better than another?

I have two models, A and B. I have performed 10-fold cross-validation on both of them, so now I have 10 $R^2$ scores for each. How can I determine whether one is significantly better than the other? ...
2
votes
1answer
45 views

Relative increase between two pairs of samples

Consider the following samples from four distributions: $a = \{20, 2, 200\} \qquad\qquad c = \{1, 10, 100\}$ $b = \{22, 2.2, 220\} \qquad\quad\:\,d = \{1.2, 12, 120\}$ I would like to say that the ...
1
vote
0answers
6 views

permutation test with a distribution of correct values

I want to test my classifier performance using a permutation test. I know how to test one actual value against a null distribution of values generated by randomly reassigning labels to my two classes. ...
1
vote
0answers
12 views

Likelihood Ratio as statistical test for Transfer Entropy with MatLab

I estimated a Transfer Entropy value TE(XY). Now I want to establish the statistical significance of the estimated value. Therefore I used the method of shuffled surrogates to estimate a shuffled ...
0
votes
0answers
22 views

main effect and interaction effect

I faced difficulty with my data analysis output. I had two factors with split plot design. Separately, the main effect of the two factors on soil pH was significant. But their interaction effect was ...
0
votes
0answers
11 views

How to calculate whether a change in expenditure is statisically significant

Sorry, this is probably very basic... I have data on expenditure over 3 years. This expenditure is made up of 2 products L and H. L has been increasing and H has been decreasing, so overall L+H is ...
23
votes
4answers
739 views

Is the “hybrid” between Fisher and Neyman-Pearson approaches to statistical testing really an “incoherent mishmash”?

There exists a certain school of thought according to which the most widespread approach to statistical testing is a "hybrid" between two approaches: that of Fisher and that of Neyman-Pearson; these ...
0
votes
1answer
15 views

Non-response bias (many waves, time span between waves not consistent)

I want to ask regarding non-response bias. I have sent my questionnaire on-line to SMEs in Malaysia. However, I have sent reminders many times (around 5 times). However, now I cannot recall when the ...
0
votes
0answers
25 views

Correcting for multiple testing on non-independent sliding windows

Question Values taken from adjecent windows in a sliding windows are corrolated. If I calculate a p-value from each window, how can I correct for the fact I have tested many windows, given that the ...
2
votes
2answers
46 views

Confidence intervals which contain 1 but the z test and p value are significant

I calculated the confidence intervals for the regression coefficients using the formula: SE X 1.96. I then took this value away from the regression coefficient to get the lower confidence level and ...
0
votes
0answers
15 views

How to down weight correlations in my microarray analysis?

Background: I have been tasked in one part of my analysis to reproduce a method used in another study as follows in bullet points form: Microarray data from a number of time points Calculate ...
0
votes
1answer
70 views

How can a regression be significant yet both predictors be non-significant? [duplicate]

I know this question has been asked in a slightly different form here. But my question differs and because of the forum rules I can't post on that thread. I have 2 independent variables, n=32, highly ...
0
votes
1answer
60 views

High Standard Deviation for Leave one out cross-validation?

I am using the leave one out cross-validation technique to evaluate my model. If the prediction on the test sample is right the output is 1 otherwise 0. So I have a array of N samples with 0's and 1's ...
0
votes
0answers
24 views

$J$ statistics to $p$-value of $J$ statistics an vice versa

I am not entirely sure if I should ask this question here, but does EViews has a function of converting J statistics to p-value of Jstats, or other way around. I am running several GMM estimations ...
0
votes
0answers
12 views

How To Determine Association Plot in R and p-value

I have created a contingency table and an association plot in R and I am not quite sure how to interpret it. Here is my contingency table: ...
0
votes
0answers
14 views

Statistical test Comparison of

I have some ordinal data (no. of exits from opiate treatment, by type (planned unplanned exits etc.). I want to know if last years figures have a statistical probability of being different from ...
0
votes
0answers
26 views

Bootstrapping results for p-values of trading rule profit significance

I want to obtain p-values for the significance of trading rule profits as described by LeBaron and Brock (1992). The author's use what is developed by Efron (1973) called Bootstrapping. It goes like ...
1
vote
0answers
19 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
0
votes
0answers
41 views

How to obtain t-value for significance of median

I collected analysts’ forecasts and calculated bias and accuracy such as: $bias=actual-forecast$ $accuracy=abs(actual-forecast)$ What I am wondering is now on how to calculate the t-statistic to ...
1
vote
0answers
16 views

Comparing 2 classifiers with different feature vectors based on same training and test data

Let us assume we have two classifiers C1 and C2 which are based on different feature vectors, ...
0
votes
0answers
21 views

3 way mixed ANOVA & pairwise comparisons effect size

im in desperate need of some help for my data analysis assignment! I have a 3 way mixed anova- where I have 2 sig main effects, one significant two way interaction and the sig three way ...
3
votes
1answer
34 views

Using a binomial approach to calculate significance of multiple significance tests

I have 10 data sets with normal distributions and carrying out significance tests on each one. With this many data sets, there is a good chance that some will be significant by chance. I am going to ...
0
votes
0answers
16 views

Comparing two paired data sets

I am trying to see compare tow paired data sets which do not have normal distributions, and I am not sure how to do that. Can I use t-test? I would appreciate your help very much. This what they look ...
2
votes
1answer
35 views

What are the relative merits of testing significance of a regression coefficient

If I have several hundred coefficients generated by running multiple variable regression model (keeping it as broad as possible by not specifying the nature of the predictors and outcome variable), it ...
2
votes
2answers
54 views

Which property of count data make mean-variance dependency?

I have read about the fact that, there is dependency of variance on mean of count data.In most of cases they do variance stabilization transfomration as preprocessing step of data modeling. I wonder, ...
0
votes
0answers
12 views

How do I perform a statistical analysis on an intervention based study?

I am trying to get my head around the best way to analyse a study. The study comprises of persons that get given a number of up to 10 different interventions, whereby hospital admission frequency is ...
1
vote
0answers
34 views

Test means of populations with lots of zeros, is sampling the way to go?

I am trying to test whether means of two populations with lots of zeros are different. Here is the following python code example: ...
1
vote
0answers
67 views

MHT: Pre-Selecting statistical tests without Bias

Summary: The last formula boxed in red (which is a modified log likelihood from logistic regression) is a special non-differentiable loss function that is adapted to contain a Bonferroni correction in ...
12
votes
5answers
216 views

Does $r$-squared have a $p$-value?

I seem to have confused myself trying to understand if a $r$-squared value also has a $p$-value. As I understand it, in linear correlation with a set of data points $r$ can have a value ranging from ...
0
votes
0answers
32 views

compare strength algorithms across different data sets

Reflecting on this paper. The use of a statistical significance test is required in order to compare the algorithms across different data sets. In that paper and other papers, the number of data ...
0
votes
0answers
13 views

are the percentage differences significantly different from oneanother?

morning, say i have 5 wheat varieties undergoing 2 treatment. when counting the seeds of the replicates the percentage difference between the means of the 2 treatments is 10% for variety 1, 30% for 2, ...
0
votes
1answer
22 views

Statistical tests

Can someone please explain the difference between difference test and correlation test? I am confused because e.g. if a mean of one group (males) is significantly different from the other (females), ...
0
votes
3answers
40 views

How to test if the change in difference between 2 distributions is significant?

I have 2 different conditions of 'behaviour state' and 2 different people. I'm trying to find a way to test if the difference in distributions (which I'm assuming are approximately normal) changes ...
0
votes
0answers
15 views

Non Significant MANOVA but significant OLS regression

I have data which is showing a $p$-value of $0.15$ significance value within a MANOVA (including a significant planned contrast). However, when I dummy code and put it into an OLS regression, the ...
0
votes
1answer
46 views

Finding distribution function [duplicate]

Let $X_1,X_2,...$ denote iid random variables such that $X_1$ is continuously distributed with density $p(x)$ and distribution function $F(x)$ where $F(0) = 0, F(x) > 0$ for all $x > 0$, and ...
0
votes
2answers
27 views

Comparison between Two Groups

One of my demographic variable is age. Age is measured as continues data not categorical. If I want to test differences between two groups to determine whether there is significant difference between ...
0
votes
0answers
13 views

Response Rate in Research

I want to ask about response rate. I have distributed 2500 posts. However, I have received back around 30. Then, I sent 200 posts as reminders with appealing letter. I have received one or two only. ...
2
votes
0answers
38 views

Inference test when data points are means

General background I've administered two types of questions to students multiple times. Type 1 I've administered six times, and type 2 I've administered five times. The two questions get at the same ...
0
votes
0answers
29 views

Chi-squared test with scipy: what's the difference between chi2_contingency and chisquare?

I'd like to run a chi-squared test in Python with scipy. I've created code to do this, but I don't know if what I'm doing is right, because the scipy docs are quite sparse. Background first: I have ...
0
votes
3answers
93 views

Is Maximum Likelihood Estimation (MLE) a parametric approach?

There are two main probabilistic approaches to novelty detection: parametric and non-parametric. The non-parametric approach assumes that the distribution or density function is derived from the ...
0
votes
0answers
45 views

Statistic for count process and measurements

I am facing some problems concerning which statistical approach to use for my measurements. I have a sensors which counts how many times an events occured. I had to characterize two lots of sensors, ...
2
votes
0answers
14 views

Motivating likelihood ratio test vs Wald test for paper reviewer

I've got back reviews for a paper I've submitted, with the following problem. I have two logistic regression models, say y ~ A, and y ~ A + B, where B is a factor with several levels. I have ...