Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

0
votes
0answers
7 views

probability life similarities in a population cross section

I have just read up on Bouchard's Minnesota Twins study that turned up some amazing identical similarities in the lives of some identical twins reared apart. Both had same jobs, married and divorced ...
0
votes
0answers
17 views

Significance of linear combination of two insignificant regression parameters

I'm running a model with an interaction term, of the general form: $Y=b_1X_1+b_2X_2+b_3X_1X_2$, whereby $X_2$ is a dummy variable and $X_1$ is continuous. Neither $X_1$ nor $X_2$ turn out as ...
2
votes
0answers
28 views

Testing correlation between two variables when there are other variables involved

I want to see if there is any correlation between number of pages in a paper and the citation it receives. If I just plot number of pages vs number of citations of a paper, I might get a correlation, ...
4
votes
1answer
143 views

What does “statistically insignificantly worse” mean?

I am reviewing a paper where I found this sentence repeatedly: A is statistically insignificantly worse than B I consider this to be confusing since, for me at least, it is not immediately clear ...
1
vote
0answers
16 views

Significance testing between two sets of Pearson's correlation coefficients

I have been reading through questions here to find an answer to my question but didn't find a satisfactory one. Here is my scenario: I have a system computing an algorithm on each of 100 units (so ...
1
vote
1answer
24 views

Testing the hypothesis on clustering

I have a number of samples. For each, there is a time course of multivariate data defined, with $t$ timepoints ($t < 50$) and $n$ variables ($n > 100$). We have noted that the time courses of a ...
1
vote
2answers
61 views

Test if two binomial distributions are statistically different from each other

I have three groups of data, each with a binomial distribution (i.e. each group has elements that are either success or failure). I do not have a predicted probability of success, but instead can ...
2
votes
1answer
98 views

Can we skip the lower order terms in interactions? [duplicate]

This question is about three-way interaction and the possibility of applying without second lower terms with keeping the main variables in the equation not like the other questions. In fact the other ...
0
votes
1answer
33 views

Imputation model: Pooled model is insignificant. How to interpret?

I have ordinal data on three IVs ranging from 1 to 5 as below: IV1: Not at all Important - Very Important IV2: Not at all Satisfied - Very Satisfied IV3: Performs much Worse - Performs much better ...
1
vote
0answers
27 views

How to test for a home bias in a data set?

I'm trying to think how I can test a data set I have, for transactions between buyers and sellers, for a home bias. My hypothesis is that buyers would rather buy from a merchant from their home ...
1
vote
0answers
40 views

Which post-hoc comparison to use and why

Hopefully this is now a much clearer picture of the question I want to answer: For each of 60 different plans I ran 35 simulation replications of my Agent Based Model (ABM) simulation. Then for each ...
1
vote
1answer
30 views

How can a distribution of cross-validated $R^2$ scores be used to determine whether one model is significantly better than another?

I have two models, A and B. I have performed 10-fold cross-validation on both of them, so now I have 10 $R^2$ scores for each. How can I determine whether one is significantly better than the other? ...
2
votes
1answer
45 views

Relative increase between two pairs of samples

Consider the following samples from four distributions: $a = \{20, 2, 200\} \qquad\qquad c = \{1, 10, 100\}$ $b = \{22, 2.2, 220\} \qquad\quad\:\,d = \{1.2, 12, 120\}$ I would like to say that the ...
1
vote
0answers
6 views

permutation test with a distribution of correct values

I want to test my classifier performance using a permutation test. I know how to test one actual value against a null distribution of values generated by randomly reassigning labels to my two classes. ...
1
vote
0answers
13 views

Likelihood Ratio as statistical test for Transfer Entropy with MatLab

I estimated a Transfer Entropy value TE(XY). Now I want to establish the statistical significance of the estimated value. Therefore I used the method of shuffled surrogates to estimate a shuffled ...
0
votes
0answers
22 views

main effect and interaction effect

I faced difficulty with my data analysis output. I had two factors with split plot design. Separately, the main effect of the two factors on soil pH was significant. But their interaction effect was ...
0
votes
0answers
11 views

How to calculate whether a change in expenditure is statisically significant

Sorry, this is probably very basic... I have data on expenditure over 3 years. This expenditure is made up of 2 products L and H. L has been increasing and H has been decreasing, so overall L+H is ...
23
votes
4answers
745 views

Is the “hybrid” between Fisher and Neyman-Pearson approaches to statistical testing really an “incoherent mishmash”?

There exists a certain school of thought according to which the most widespread approach to statistical testing is a "hybrid" between two approaches: that of Fisher and that of Neyman-Pearson; these ...
0
votes
1answer
15 views

Non-response bias (many waves, time span between waves not consistent)

I want to ask regarding non-response bias. I have sent my questionnaire on-line to SMEs in Malaysia. However, I have sent reminders many times (around 5 times). However, now I cannot recall when the ...
0
votes
0answers
25 views

Correcting for multiple testing on non-independent sliding windows

Question Values taken from adjecent windows in a sliding windows are corrolated. If I calculate a p-value from each window, how can I correct for the fact I have tested many windows, given that the ...
2
votes
2answers
46 views

Confidence intervals which contain 1 but the z test and p value are significant

I calculated the confidence intervals for the regression coefficients using the formula: SE X 1.96. I then took this value away from the regression coefficient to get the lower confidence level and ...
0
votes
0answers
15 views

How to down weight correlations in my microarray analysis?

Background: I have been tasked in one part of my analysis to reproduce a method used in another study as follows in bullet points form: Microarray data from a number of time points Calculate ...
0
votes
1answer
71 views

How can a regression be significant yet both predictors be non-significant? [duplicate]

I know this question has been asked in a slightly different form here. But my question differs and because of the forum rules I can't post on that thread. I have 2 independent variables, n=32, highly ...
0
votes
1answer
63 views

High Standard Deviation for Leave one out cross-validation?

I am using the leave one out cross-validation technique to evaluate my model. If the prediction on the test sample is right the output is 1 otherwise 0. So I have a array of N samples with 0's and 1's ...
0
votes
0answers
24 views

$J$ statistics to $p$-value of $J$ statistics an vice versa

I am not entirely sure if I should ask this question here, but does EViews has a function of converting J statistics to p-value of Jstats, or other way around. I am running several GMM estimations ...
0
votes
0answers
12 views

How To Determine Association Plot in R and p-value

I have created a contingency table and an association plot in R and I am not quite sure how to interpret it. Here is my contingency table: ...
0
votes
0answers
14 views

Statistical test Comparison of

I have some ordinal data (no. of exits from opiate treatment, by type (planned unplanned exits etc.). I want to know if last years figures have a statistical probability of being different from ...
0
votes
0answers
26 views

Bootstrapping results for p-values of trading rule profit significance

I want to obtain p-values for the significance of trading rule profits as described by LeBaron and Brock (1992). The author's use what is developed by Efron (1973) called Bootstrapping. It goes like ...
1
vote
0answers
19 views

Measure difference in answers on survey depending on item response options

For my thesis I sent out a questionnaire that has 10 questions. I set defaults: the first seven question had the default set to the desired answer and the last three questions had the default set to ...
0
votes
0answers
41 views

How to obtain t-value for significance of median

I collected analysts’ forecasts and calculated bias and accuracy such as: $bias=actual-forecast$ $accuracy=abs(actual-forecast)$ What I am wondering is now on how to calculate the t-statistic to ...
1
vote
0answers
16 views

Comparing 2 classifiers with different feature vectors based on same training and test data

Let us assume we have two classifiers C1 and C2 which are based on different feature vectors, ...
0
votes
0answers
22 views

3 way mixed ANOVA & pairwise comparisons effect size

im in desperate need of some help for my data analysis assignment! I have a 3 way mixed anova- where I have 2 sig main effects, one significant two way interaction and the sig three way ...
3
votes
1answer
34 views

Using a binomial approach to calculate significance of multiple significance tests

I have 10 data sets with normal distributions and carrying out significance tests on each one. With this many data sets, there is a good chance that some will be significant by chance. I am going to ...
0
votes
0answers
16 views

Comparing two paired data sets

I am trying to see compare tow paired data sets which do not have normal distributions, and I am not sure how to do that. Can I use t-test? I would appreciate your help very much. This what they look ...
2
votes
1answer
36 views

What are the relative merits of testing significance of a regression coefficient

If I have several hundred coefficients generated by running multiple variable regression model (keeping it as broad as possible by not specifying the nature of the predictors and outcome variable), it ...
2
votes
2answers
54 views

Which property of count data make mean-variance dependency?

I have read about the fact that, there is dependency of variance on mean of count data.In most of cases they do variance stabilization transfomration as preprocessing step of data modeling. I wonder, ...
0
votes
0answers
12 views

How do I perform a statistical analysis on an intervention based study?

I am trying to get my head around the best way to analyse a study. The study comprises of persons that get given a number of up to 10 different interventions, whereby hospital admission frequency is ...
1
vote
0answers
34 views

Test means of populations with lots of zeros, is sampling the way to go?

I am trying to test whether means of two populations with lots of zeros are different. Here is the following python code example: ...
1
vote
0answers
67 views

MHT: Pre-Selecting statistical tests without Bias

Summary: The last formula boxed in red (which is a modified log likelihood from logistic regression) is a special non-differentiable loss function that is adapted to contain a Bonferroni correction in ...
12
votes
5answers
216 views

Does $r$-squared have a $p$-value?

I seem to have confused myself trying to understand if a $r$-squared value also has a $p$-value. As I understand it, in linear correlation with a set of data points $r$ can have a value ranging from ...
0
votes
0answers
32 views

compare strength algorithms across different data sets

Reflecting on this paper. The use of a statistical significance test is required in order to compare the algorithms across different data sets. In that paper and other papers, the number of data ...
0
votes
0answers
13 views

are the percentage differences significantly different from oneanother?

morning, say i have 5 wheat varieties undergoing 2 treatment. when counting the seeds of the replicates the percentage difference between the means of the 2 treatments is 10% for variety 1, 30% for 2, ...
0
votes
1answer
22 views

Statistical tests

Can someone please explain the difference between difference test and correlation test? I am confused because e.g. if a mean of one group (males) is significantly different from the other (females), ...
0
votes
3answers
41 views

How to test if the change in difference between 2 distributions is significant?

I have 2 different conditions of 'behaviour state' and 2 different people. I'm trying to find a way to test if the difference in distributions (which I'm assuming are approximately normal) changes ...
0
votes
0answers
15 views

Non Significant MANOVA but significant OLS regression

I have data which is showing a $p$-value of $0.15$ significance value within a MANOVA (including a significant planned contrast). However, when I dummy code and put it into an OLS regression, the ...
0
votes
1answer
46 views

Finding distribution function [duplicate]

Let $X_1,X_2,...$ denote iid random variables such that $X_1$ is continuously distributed with density $p(x)$ and distribution function $F(x)$ where $F(0) = 0, F(x) > 0$ for all $x > 0$, and ...
0
votes
2answers
27 views

Comparison between Two Groups

One of my demographic variable is age. Age is measured as continues data not categorical. If I want to test differences between two groups to determine whether there is significant difference between ...
0
votes
0answers
13 views

Response Rate in Research

I want to ask about response rate. I have distributed 2500 posts. However, I have received back around 30. Then, I sent 200 posts as reminders with appealing letter. I have received one or two only. ...
2
votes
0answers
38 views

Inference test when data points are means

General background I've administered two types of questions to students multiple times. Type 1 I've administered six times, and type 2 I've administered five times. The two questions get at the same ...
0
votes
0answers
30 views

Chi-squared test with scipy: what's the difference between chi2_contingency and chisquare?

I'd like to run a chi-squared test in Python with scipy. I've created code to do this, but I don't know if what I'm doing is right, because the scipy docs are quite sparse. Background first: I have ...