Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

1
vote
0answers
4 views

What is the purpose of specifying a “control” in an A/B/N split test?

I define control as the currently accepted, best known variation. The test to be performed would introduce n new variations. Each variation will be served randomly for the duration of the test. I am ...
0
votes
0answers
6 views

How to extend independent significance features test to multiple classes?

I have found this post about feature selection using the independent significance features test. There is also an implementation provided. Unfortunately it only works for 2 classes but I have 4 ...
1
vote
0answers
25 views

Statistical significance between group sums

How can we state that there is a significant difference between total amounts of donations of two sample groups in the following scenario. Control group A has donated lots of smaller denominations ...
1
vote
1answer
24 views

Which statistical analysis to use to compare the level of similarity between two large samples?

I'm writing a small speech recognition prototype as my side project, which matches pre-recorded words of the speaker. So now I'm thinking of comparing two sets of data (outcome of FFT) which are two ...
1
vote
0answers
28 views

t-test in large, unequal sample size + additional info such as effect size

I have 2 samples, named "1" and "3": (1) n = 1282, mean = 14.77, sd = 8.27 (3) n = 5360, mean = 15.88, sd = 8.55 Their distribution look: ...
1
vote
2answers
84 views

significance test for yes/no data in R

I have the following data format: ...
1
vote
0answers
27 views

Comparing more than two populations

I have a dataset of around 20 financial advisors and their customers. The customers have been divided into groups based on type of investment, as well as the size of their investment and whether the ...
1
vote
0answers
18 views

Combining data from different locations to test difference in populations

I have (irregularly spaced) times series concentration data for four locations and am trying to discern whether or not there has been a change in concentrations after a particular event. To avoid ...
-3
votes
0answers
29 views

Why is the reported p-value in papers is so less than 0.05 in wilicoxon test? [duplicate]

0 down vote favorite I have been testing my two vectors using wilicoxon signed rank test and kolmogorov smirnov test to evaluate their significance of difference. I am using matlab fr this purpose ...
0
votes
0answers
6 views

Statistical significance of difference between samples given total success rate when test was applied by group

I am comparing treatments that select members to receive some email, and we are measuring the response to that email. The emails are similar, but are sent to a different group of members and a ...
0
votes
0answers
10 views

Ordered Probit Interpretation

My dependent variable is ordinal from 1-4. My independent variable consists of categories. I haven't created dummies for the categories, but it would be easy to do so. From what I have gathered, an ...
0
votes
0answers
10 views

Calculate Sample Size for a Price Change for Specific Group

Say I operate a tour bus. For every tour I run, I offer guests the option to buy a camera. The camera currently costs 5. I know average purchase rate is 40 percent and the standard deviation per ...
0
votes
1answer
31 views

Difference between multi-factor ANOVA and Cox proportional hazard model?

I am wondering what is the difference between multi-factor ANOVA and Cox proportional hazard model? As I understood both of them test the significance of some independent factors on a dependent ...
0
votes
1answer
16 views

Statistical significance between retailer vs peers monthly satisfaction scores

I have a chart shown below of 2 retailers where consumer satisfaction scores have been captured for the same product. These scores were tracked monthly for both retailers through the month of June ...
0
votes
1answer
17 views

What is the meaning of Confidence Intervals regarding Hold Out samples?

Let's say you have a regression model that estimates GDP. Your model has a Standard Error of 1%. So, you can readily build Confidence Intervals around your regressed estimates. Your 95% CI will be ...
1
vote
0answers
22 views

A/B Testing (Pearson Ch-Square Test) with Unequal Sample Sizes

We have run A/B tests in the past using a 50/50 split, and have used the power prop function for calculating the sample size using the power.prop.test in R. My question is how do I calculate what my ...
0
votes
0answers
4 views

Statistical Test to Use with Nearest Neighbor Matching with Replacement?

I'm planning on doing a project to measure the effect of a treatment, using nearest neighbor matching with replacement of control units (i.e. more than one test unit can be matched to the same control ...
1
vote
3answers
83 views

Averaging ROC curves over folds in cross-validation

I have data from 10-fold cross-validation experiment: for each fold I have a predictor and a response variable so I can generate ROC curve and compute area under the ROC curve. I have a series of ...
1
vote
0answers
35 views

Sample size when using GPS-trackers to analyze movement of reindeers

If one were to try and analyze the movement of ~2 500 reindeers before and after a (man-made) change in landscape have occured. Specifically how spread out the herd is, both as a whole and if there ...
0
votes
0answers
21 views

To work out 7 point Likert scale sample size, can I increase the MoE to 7%?

I have a survey that consists of 7 point Likert scale data. Each sample is quite expensive to collect, so I'm trying to keep the sample size as small as possible without compromising the results. My ...
2
votes
1answer
14 views

Comparing performance of a single classifier on two datasets

I'm using a single classifier on two different datasets, consisting of non-overlapping set of observations. The performance on one dataset is higher than the other, and I want to show that the ...
1
vote
1answer
15 views

Comparing retention rates with differing sample sizes

I'm looking to compare the retention rate of people who attended an event and those who did not. The list of those who did not (70) is significantly larger than those who have (28). Here is an ...
0
votes
0answers
13 views

Statistical significance of bilayer biological network edges

I have a biological network with two types of nodes: genes and approved drugs. The edges can only exist between two different types of nodes. The mapping can be many-to-many (eg. a gene can be ...
0
votes
0answers
15 views

How to compare two independent samples using a permutation test? [duplicate]

Suppose I have two independent samples. How can I test if the two samples have been drawn from the same distribution using a permutation test? The sample sizes are equal. The population distributions ...
0
votes
0answers
17 views

AMLE Regression Statistics [closed]

The following is the output of the model (LOADEST-USGS) I used for nutrient load estimation. Is “Significance Level of PPCC Test” the same as “α-level”. If so, then what would be the exact critical ...
0
votes
0answers
17 views

Sampling non-normal (near power-law) distribution

Context: I have a population that shows three distinct behaviors (slope) on a cumulative distribution function (survival proportion over time) and I want to find factors that may explain these ...
75
votes
10answers
11k views

Is this really how p-values work? Can a million research papers per year be based on pure randomness?

I'm very new to statistics, and I'm just learning to understand the basics, including p-values. But there is a huge question mark in my mind right now, and I kind of hope my understanding is wrong. ...
0
votes
0answers
7 views

Input, analyse and graph Semantic Differential Data with R?

Has anyone tried to use Semantic Differential data in R? This is a very useful survey scale and I was surprised that information on it's use in R is very hard to find. The Likert package states ...
0
votes
0answers
28 views

What statistical test can be used to determine if a list of attributes fits a known list?

My questions relates to microarray signature matching, but I will generalize the concept for anybody who isn't familiar with this technology. Say there is a list of attributes for a given sample. ...
0
votes
0answers
33 views

repeated measures design of Analysis of variance (ANOVA) with mean not statistically significant and Standard deviation significant

The study area was sampled using a tessellation-based stratified sampling scheme. The area was divided into 50-m × 50-m squares, a point was randomly selected in each square. For each point, a ...
0
votes
0answers
11 views

Which test-statistics do I use if I just want to test one particular regression coefficient

Hello my Question is this. I want to test a model which has the form: $Y_{i,j}= \beta_0+\beta_1 I_{treatment}+ \beta_2j+\beta_3j*I_{treatment},$ (1) where $i$ is the Identifikation Number and $j$ is ...
0
votes
0answers
7 views

Interpreting Tukey test for significant differences

Hi I have 38 categories and I have their means. I want to see if the differences in means are significantly different. I ran an anova and the p-value is <2e-16. Then I wanted to investigate ...
0
votes
0answers
10 views

Comparing two empirical distributions using mean and variance

I want to compare two classifiers (random variables). Each classifier is binary variable. I have evaluated the two classifiers on different set {S_1,S_2,..S_m} each with N samples, and calculated the ...
0
votes
1answer
8 views

which statistical test to use when comparing the expectation and perception values in servqual?

which statistical test to use when comparing the expectation and perception values in servqual? i collected data among students before classes began as expectation score and then after the classes ...
1
vote
1answer
28 views

Kuiper's Test P Value in R

I am trying to implement a kuiper's test on a set of data and looking for an R package (or other program) that can give a p-value without monte carlo simulation or only a range of p-values. All of the ...
6
votes
3answers
971 views

What does a p-value of exactly 1.0000 mean?

I have a table of pairwise comparisons using paired t-tests, which I produced using the following code in R: ...
1
vote
1answer
71 views

Tests for statistical significance of Kalman Filter estimate

I am working on regression model with time-varying coefficient, which looks like For simplicity, I assume that $\alpha_t$, $\beta_t$, and $\gamma_t$ follow random walk. I have already learned how ...
1
vote
1answer
20 views

How to test for significant differences between 3 groups of 5 dimensional polytomous data

I have asked participants in a study to categorize the meaning of messages across 5-dimensions pertaining to Happiness, Sadness, Sarcasm, Honesty, and Anger. For each dimension there were 5 potential ...
0
votes
0answers
48 views

Why does my p-value = 0?

I created a vector that is 329 rows long, with each row containing a z-score. I derived my degrees of freedom by counting up the number of rows (329) and subtracting 1. Then, I computed a test ...
2
votes
0answers
34 views

At what point are options not “significant”?

For example, say there's a survey question with 500 possible answers (this isn't actually the case - it's not a survey, and there are far more than 500 options), and I want to display people's ...
0
votes
0answers
16 views

Degree of difference

Data (from four places) on area(sampled), production and productivity of a fruit is available from two sources . The data is available for the years 2012-13 and 2013-14. I have been asked to find the ...
0
votes
0answers
43 views

What significance test to use for count data?

I have TRANSFAC data of number of transcrption factor binding sites(TFBS) for each gene. I have ~1100 Transcription factors (TFs) and 2 sets of genes: 18 genes belonging to Pigmentation AND 5 genes ...
2
votes
2answers
98 views

Ratio of standard deviation to mean

Let's say that we have a confidence interval like this: µ ± σ, where σ is the standard deviation and µ is the mean. I want to know if the ratio σ/µ has any interpretation? Does it mean anything if we ...
0
votes
0answers
13 views

Testing Cointegration at level or with first differences?

I have a question regarding testing for cointegration. The situation is a follows: I have two time series that are both stationary at level when I consider "constant and trend". They are otherwise ...
0
votes
0answers
9 views

Results of modeltests are different - Rasch model

I'm a bit confused while scaling a test via Rasch modeling. Is it normal, that the results of Andersens LRtest in combination with graphical model test differs partially from the results of the ...
0
votes
0answers
13 views

Finding keywords in small pieces of text.

I'm (attempting to) construct a bayesian neural network to parse through tweets to then decide if someone bought/sold something and for how much. I'm having trouble figuring out what the input nodes ...
1
vote
1answer
28 views

Dependence, independence, multiple testing, and alpha error correction

although I have studied psychology and feel I should know a lot more about issues such as this one, I have a quite basic question (or maybe not so basic question, considering all the fuss about this) ...
1
vote
2answers
37 views

Why does bigger $n$ mean higher chance to reject the null hypothesis?

I'm new to statistics and to statistical tests, so please bare with me. The attached sketches are rough estimations, and are meant merely for visualization and clarity. Suppose the following is the ...
3
votes
1answer
130 views

Statistical significance test for averages of correlation coefficients

My setting looks like that I have several subjects $i$ (let's say 5) and for each subject I measure the lists/vectors $A_i$, $B_i$, $C_i$ and $D_i$. Then, for each subject $i$, I calculate the ...
0
votes
0answers
15 views

Relative increase between unpaired samples - statistical test to use?

Let's say we have data from which we calculate relative changes between unpaired samples, and want to see if these are significantly different from each other. For example: $a = \{0.112, 0.115, ...