Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

8
votes
1answer
115 views

Is the exact value of a 'p-value' meaningless?

I had a discussion with a statistician back in 2009 where he stated that the exact value of a p-value is irrelevant: the only thing that is important is whether it is significant or not. I.e. one ...
0
votes
0answers
14 views

Facing Single-Class Training Set when Using Random Sampling

In a highly imbalanced binary classification (rare class < 10% of whole data), when I perform random sample selection (less than 15% of whole data to be selected for training) in a trial of 1000 ...
0
votes
0answers
11 views

Differences between groups using proportions

I am doing two analyses. First, I want to compare performance of two groups (p<.05), with the independent variable being proportional data (total number of categorical responses/total number of ...
0
votes
0answers
12 views

Statistical tests to check quality of data before learning model

I have collected some gene expression data and I want to learn a Bayesian network out of it. Before that, I want to do some statistical analysis to test the quality of my data. Now I want to know ...
0
votes
0answers
12 views

Empty significance in SPSS linear regression

I got a weird result with linear regression: there are empty significances. I use 'Stepwise' method, so it should drop the inadequate variables. (Of course the "000" Betas are confusing too.) What ...
0
votes
0answers
21 views

Information gain from a data set

I have a doubt regarding the use of information gain for data classification rather than conventional term frequency matrix.Lets assume I have 500 documents and I created a feature set of 12500*500. ...
3
votes
1answer
32 views

p-value for weighted Pearson correlation coefficient

I'm computing a weighted correlation coefficient, using the method described here. I'd like to compute a p-value for the resulting r coefficient. How can I do this correctly, given that my r was ...
0
votes
1answer
33 views

Measuring Statistical Significance of Binary Classification using Matthews Correlation Coefficient

Based on the following relationship between Matthew's Correlation Coefficient (MCC) and Chi Square: (MCC is the Pearson product-moment Correlation Coefficient) Is it possible to conclude that: By ...
1
vote
0answers
22 views

Should I be using a Welch or standard T-test?

I'm really confused here. I am trying to compare two sets of data which differ in sample size. One has 257 entries and the other has 598. They are measures of scores on two different assessment types ...
0
votes
0answers
9 views

which test to use here to test the significance of overlap with respect to a control?

i'm not sure what statistical test to use to compare a scenario like this: I have a list of candidate regions A (560k regions) and I want to know if my candidate regions enrich for a particular set ...
1
vote
1answer
34 views

Adjusting significance level with for different kinds of tests (chisq, mann whitney etc.) on the same variable

I have conducted a series of tests on a single, dichotomic variable (presence/absence of a personality disorder in a neurological illness) in order to compare several things (i.e. presence of anxiety ...
2
votes
1answer
24 views

Is it ok to adjust significance level according to number of comparisons in simple correlations?

a referee asked me to "split" the significance level for the number of comparison, although i am not doing repeated comparisons but just spearman correlations between variables. I have set the ...
0
votes
1answer
17 views

Statistical significance of different models [duplicate]

I ran a specification with an outcome in $ terms, and a coefficient on a dummy variable was statistically significant. However, when I ran the exact same model with the outcome in log-dollar terms, ...
0
votes
0answers
13 views

Evaluating productivity metrics in an A/B test system

I'm beginning work on an A/B testing system for UX changes to a tool my team builds. Our users are doing simple, repeatable tasks assigned by another team's system. My team handles the UI/UX part. ...
4
votes
2answers
71 views

Help for planning a neuroscience experiment

I'm new here. I am planning a neuroscience experiment. I will be measuring brain signals from about twenty subjects. I will present the subjects four different kinds of stimuli. After all the data ...
2
votes
1answer
62 views

How to compare whether the coefficients of two independent variables statistically different from each other?

If I have two independent variables and they are dummy variable along with other independent variables and I run a linear probability model, I want to compare whether the coefficients of two dummy ...
1
vote
2answers
62 views

How can I determine statistical significance in an A/B test in which the KPI is dependent upon two variables - one bernouli and one continuous?

In my work in online marketing, we frequently run A/B or multivariate web page tests. The Key Performance Indicator for these tests is overall Revenue. The treatment that nets the most revenue, either ...
0
votes
1answer
66 views

Tree structured proportion of success : statistical significance?

I am exploring a tree structured as follows: A/ parent node = parent population, size N, with a proportion of success for an experiment (eg. p = nb success / N) B/ child nodes = sub populations for ...
2
votes
1answer
46 views

Statistical significance in time series and sub-series

So, when I did first year stats in undergrad, we did an experiment where we tampered with a bunch of coins, to see if it would cause a statistical difference in the results. This is a graph of the ...
0
votes
0answers
20 views

Comparing two means and need to know if they are statistically different

I have the net revenue from 81 companies in one year and I'm comparing to the net revenue of the same 81 companies 3 years later. How do I compare (simple subtraction of the means or t-test?), and how ...
0
votes
0answers
12 views

1/F distribution equality significance level

A question about equality $$F_{r_{1},r_{2},\alpha}=\frac{1}{F_{r_{2},r_{1},1-\alpha}}$$ We know if $X\sim\chi^{2}\left( r_{1}\right) ,~Y\sim\chi^{2}\left( r_{2}\right) $ and $X,~Y$ independent ...
1
vote
0answers
7 views

Calculating significance for unequal sample sizes in A/B Testing

I'm running some A/B tests on my site and am using varying visit distributions to minimize risk. Here is an example of one of the test that I have running: Control: 10% of traffic; Variation 1: 45% ...
1
vote
1answer
21 views

When using higher order terms in multivariable regression what is the effect on the P-value

I have a large dataset, lets say 500 lines of data. As a quick workaround to higher order regression I've made educated guesses about what higher order terms I would expect in my regression equation. ...
0
votes
0answers
6 views

significance Compairing two independent groups with single categorical variable

I want to see the significance between two group (One with Albandazole and without Albandazole) inrespect to their CT scane Resolution with three catories(Compltele,partially resolved, Calcified)
2
votes
0answers
22 views

Importance of multivariate normality assumption for BIC-like sparse model selection inference with PCA

I am reading a paper for robust, sparse PCA in which they propose a BIC-like criterion for selecting the appropriate value of the sparsity parameter $\lambda$. They define this as: ...
1
vote
1answer
30 views

Pearson correlation coefficient test: low r and low p

I am running a correlation coefficient test, and the results were: $r = 0.382, p = 2.76 \times 10^{-13}$. So the $r$ value is not that impressive (usually we see $r>.5$), but the $p$-value is ...
0
votes
1answer
19 views

Finding the probability of normality via Anderson-Darling, Shapiro-Wilk , and Kolomogrov-Smirov

I have quite a few distributions that were generated by some system, and I am trying to find the probability of accordance to normality for these distributions. So, the probability of accordance to ...
1
vote
0answers
22 views

Can I estimate Variance of Gamma from Negative Binomial distribution distributed data, given NB is Gamma-Poisson compound

I believe the data I have follows Negative Binomial distribution (over-dispersed Poisson). We know Negative Binomial is a Gamma-Poisson compound distribution. The variance of this Gamma distribution ...
2
votes
1answer
76 views

Hypothesis testing on Poisson Binomial distribution

Imagine $N$ baskets each filled with a different number of products chosen from a set $P$. It is possible to have repeated products in the same basket. For instance: ...
1
vote
0answers
39 views

Can a Linear-Log model be used instead of Robust Standard Errors?

If your regression model has heteroskedastic residuals, one should calculate White Standard Errors that correct for the mentioned heteroskedasticity. If the residuals are also autocorrelated one ...
1
vote
1answer
54 views

Interpretation of Maximum likelihood estimation

I have some problem to interpret the result of MLE estimation : Is it possible to get some advise about how to interpret it? the log likelihood function : $\sum^{n}_{i=1}\log\left( \phi\left( ...
1
vote
0answers
8 views

Fligner Wolfe test and 2-sample Wilcoxon test

What is the significant difference between the Fligner Wolfe test and 2-sample Wilcoxon test? How can I compare the treatment groups with the control groups of different studies after pooling?
0
votes
0answers
12 views

sequential testing and hierarchical tesing

With respect to the hypothesis testing, I once heard sth such as sequential testing, hierarchical testing, and multi-level testing. But I could not find a good reference discussing these topics. Any ...
3
votes
3answers
73 views

Which test for this data set of chemical concentrations

I have a very simple data set: a parameter (concentration of a chemical) was measured at day 0, day 1 and day 2 in three subjects (there is a control group as well, but here all the values are always ...
1
vote
0answers
43 views

Using different letters to show statistical significance

Table 3 of this paper uses the letters A, B and C to show statistically significant differences between variables. I cannot understand what the letters A, B and C actually mean however. Can anyone ...
1
vote
1answer
32 views

Could standardizing an independent variable cause the t-statistic of the OLS estimate to change?

Lets say we are looking to estimate the following standard OLS regression: $y_{i} = \beta_{0} + \beta_{1}*X_{i} + \beta_{2}*Z_{i} + \epsilon_{i}$ and that we choose to standardize $X$ as: ...
0
votes
0answers
19 views

Significant box's test equality covariance matrices, How bad it is?

Having equal number of cases across two groups (60 total cases), if one wants to perform a split-plot Anova and finds out that the Box's Test of Equality of Covariance Metrices is less than 0.001 then ...
2
votes
0answers
38 views

Package ‘fitdistrplus’

I am trying to use the package ‘fitdistrplus’ in R to fit one non standard distribution to my data set. I am trying to copy the methods the package creators used in their tutorial for specifying a ...
0
votes
0answers
23 views

Fisher's Exact Test to assess the significance of a difference between false positive rates

I have trained two binary classification models on the same data and evaluated them using the same test set. For each model I have calculated a false positive rate (and a count of false positives) at ...
1
vote
0answers
32 views

Best statistical test to show significance of two data coming from the same population?

Hope i'm posting in the right thread. I have a sample of genes where I calculate a number of features in the start of each gene, and a number of features in the end of each gene. I plotted the ...
0
votes
0answers
15 views

Mann-Whitney U and KS tests in R are not producing significant results [duplicate]

I wish to compare two test data sets using the Mann-Whitney U and KS tests in R. As a start, I thought to compare two similar sets (an A/A test) expecting to see a highly significant indication of ...
1
vote
0answers
16 views

Intuition For Overlapping Confidence Intervals and Statistical Significance [duplicate]

Two questions: I have a really hard time with the intuition for why the second part of this statement is true: If two statistics have non-overlapping confidence intervals, they are ...
7
votes
2answers
195 views

Can a narrow confidence interval around a non-significant effect provide evidence for the null?

It is obviously fallacious to assume that failure to reject the null implies that the null is true. But in a case where the null is not rejected and the corresponding confidence interval (CI) is ...
5
votes
2answers
164 views

What could be a statistical test for comparing funnel data before and after important changes?

I have some funnel data from a website I manage. After some time I did some improvement on a multi-step purchase process. I wonder if it's possible to formulate a test that with statistical ...
0
votes
0answers
10 views

NPS confidence intervals with bootstrap

I am looking for a method to find confidence intervals for NPS (net promoter score). One method was already suggested here. But I am thinking of using bootstrap. Do you think the following procedure ...
0
votes
0answers
28 views

Non-Stationarity, first differences and Panel Data

I have build a sentiment index and am now validating its statistical properties and significance. I stumbled upon two problems. (1) dfglsindicates that my ...
1
vote
1answer
60 views

Does heteroskedasticity matter if you have a large enough sample?

Let's say you run a regression with over 200 observations. Would this reasonably large sample mitigate the impact of residuals heteroskedasticity as an offshoot of the Central Limit Theorem, or ...
2
votes
0answers
52 views

Test for statistical difference between two curves on same plot?

I've got data plotted for A/Ci curves (plant physiology, A=photosynthesis rate, Ci=internal CO2 concentration--essentially it's a dose-response relationship). Generally, when this type of data is ...
0
votes
0answers
30 views

statistical significance test on cluster analysis result

I've done a hierarchical clustering on a data-set composed by 33 subjects and 2 continuous variables (called V1 and V2), which produces 3 clusters. Now I'm wodering if it make any sense to perform a ...
-2
votes
3answers
72 views

What is relationship between “significant correlation” and “significant difference”

What is relationship between "significant correlation" and "significant difference" between two variables?