Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

1
vote
0answers
14 views

Should I be using a Welch or standard T-test?

I'm really confused here. I am trying to compare two sets of data which differ in sample size. One has 257 entries and the other has 598. They are measures of scores on two different assessment types ...
0
votes
0answers
8 views

which test to use here to test the significance of overlap with respect to a control?

i'm not sure what statistical test to use to compare a scenario like this: I have a list of candidate regions A (560k regions) and I want to know if my candidate regions enrich for a particular set ...
0
votes
1answer
18 views

Adjusting significance level with for different kinds of tests (chisq, mann whitney etc.) on the same variable

I have conducted a series of tests on a single, dichotomic variable (presence/absence of a personality disorder in a neurological illness) in order to compare several things (i.e. presence of anxiety ...
2
votes
1answer
19 views

Is it ok to adjust significance level according to number of comparisons in simple correlations?

a referee asked me to "split" the significance level for the number of comparison, although i am not doing repeated comparisons but just spearman correlations between variables. I have set the ...
0
votes
1answer
16 views

Statistical significance of different models [duplicate]

I ran a specification with an outcome in $ terms, and a coefficient on a dummy variable was statistically significant. However, when I ran the exact same model with the outcome in log-dollar terms, ...
0
votes
0answers
6 views

Evaluating productivity metrics in an A/B test system

I'm beginning work on an A/B testing system for UX changes to a tool my team builds. Our users are doing simple, repeatable tasks assigned by another team's system. My team handles the UI/UX part. ...
4
votes
2answers
52 views

Help for planning a neuroscience experiment

I'm new here. I am planning a neuroscience experiment. I will be measuring brain signals from about twenty subjects. I will present the subjects four different kinds of stimuli. After all the data ...
2
votes
1answer
58 views

How to compare whether the coefficients of two independent variables statistically different from each other?

If I have two independent variables and they are dummy variable along with other independent variables and I run a linear probability model, I want to compare whether the coefficients of two dummy ...
1
vote
2answers
52 views

How can I determine statistical significance in an A/B test in which the KPI is dependent upon two variables - one bernouli and one continuous?

In my work in online marketing, we frequently run A/B or multivariate web page tests. The Key Performance Indicator for these tests is overall Revenue. The treatment that nets the most revenue, either ...
0
votes
1answer
61 views

Tree structured proportion of success : statistical significance?

I am exploring a tree structured as follows: A/ parent node = parent population, size N, with a proportion of success for an experiment (eg. p = nb success / N) B/ child nodes = sub populations for ...
2
votes
1answer
43 views

Statistical significance in time series and sub-series

So, when I did first year stats in undergrad, we did an experiment where we tampered with a bunch of coins, to see if it would cause a statistical difference in the results. This is a graph of the ...
0
votes
0answers
20 views

Comparing two means and need to know if they are statistically different

I have the net revenue from 81 companies in one year and I'm comparing to the net revenue of the same 81 companies 3 years later. How do I compare (simple subtraction of the means or t-test?), and how ...
0
votes
0answers
12 views

1/F distribution equality significance level

A question about equality $$F_{r_{1},r_{2},\alpha}=\frac{1}{F_{r_{2},r_{1},1-\alpha}}$$ We know if $X\sim\chi^{2}\left( r_{1}\right) ,~Y\sim\chi^{2}\left( r_{2}\right) $ and $X,~Y$ independent ...
1
vote
0answers
7 views

Calculating significance for unequal sample sizes in A/B Testing

I'm running some A/B tests on my site and am using varying visit distributions to minimize risk. Here is an example of one of the test that I have running: Control: 10% of traffic; Variation 1: 45% ...
1
vote
1answer
21 views

When using higher order terms in multivariable regression what is the effect on the P-value

I have a large dataset, lets say 500 lines of data. As a quick workaround to higher order regression I've made educated guesses about what higher order terms I would expect in my regression equation. ...
0
votes
0answers
6 views

significance Compairing two independent groups with single categorical variable

I want to see the significance between two group (One with Albandazole and without Albandazole) inrespect to their CT scane Resolution with three catories(Compltele,partially resolved, Calcified)
2
votes
0answers
20 views

Importance of multivariate normality assumption for BIC-like sparse model selection inference with PCA

I am reading a paper for robust, sparse PCA in which they propose a BIC-like criterion for selecting the appropriate value of the sparsity parameter $\lambda$. They define this as: ...
1
vote
1answer
28 views

Pearson correlation coefficient test: low r and low p

I am running a correlation coefficient test, and the results were: $r = 0.382, p = 2.76 \times 10^{-13}$. So the $r$ value is not that impressive (usually we see $r>.5$), but the $p$-value is ...
0
votes
1answer
19 views

Finding the probability of normality via Anderson-Darling, Shapiro-Wilk , and Kolomogrov-Smirov

I have quite a few distributions that were generated by some system, and I am trying to find the probability of accordance to normality for these distributions. So, the probability of accordance to ...
1
vote
0answers
18 views

Can I estimate Variance of Gamma from Negative Binomial distribution distributed data, given NB is Gamma-Poisson compound

I believe the data I have follows Negative Binomial distribution (over-dispersed Poisson). We know Negative Binomial is a Gamma-Poisson compound distribution. The variance of this Gamma distribution ...
2
votes
1answer
73 views

Hypothesis testing on Poisson Binomial distribution

Imagine $N$ baskets each filled with a different number of products chosen from a set $P$. It is possible to have repeated products in the same basket. For instance: ...
1
vote
0answers
39 views

Can a Linear-Log model be used instead of Robust Standard Errors?

If your regression model has heteroskedastic residuals, one should calculate White Standard Errors that correct for the mentioned heteroskedasticity. If the residuals are also autocorrelated one ...
1
vote
1answer
53 views

Interpretation of Maximum likelihood estimation

I have some problem to interpret the result of MLE estimation : Is it possible to get some advise about how to interpret it? the log likelihood function : $\sum^{n}_{i=1}\log\left( \phi\left( ...
1
vote
0answers
7 views

Fligner Wolfe test and 2-sample Wilcoxon test

What is the significant difference between the Fligner Wolfe test and 2-sample Wilcoxon test? How can I compare the treatment groups with the control groups of different studies after pooling?
0
votes
0answers
10 views

sequential testing and hierarchical tesing

With respect to the hypothesis testing, I once heard sth such as sequential testing, hierarchical testing, and multi-level testing. But I could not find a good reference discussing these topics. Any ...
3
votes
3answers
71 views

Which test for this data set of chemical concentrations

I have a very simple data set: a parameter (concentration of a chemical) was measured at day 0, day 1 and day 2 in three subjects (there is a control group as well, but here all the values are always ...
1
vote
0answers
37 views

Using different letters to show statistical significance

Table 3 of this paper uses the letters A, B and C to show statistically significant differences between variables. I cannot understand what the letters A, B and C actually mean however. Can anyone ...
1
vote
1answer
32 views

Could standardizing an independent variable cause the t-statistic of the OLS estimate to change?

Lets say we are looking to estimate the following standard OLS regression: $y_{i} = \beta_{0} + \beta_{1}*X_{i} + \beta_{2}*Z_{i} + \epsilon_{i}$ and that we choose to standardize $X$ as: ...
0
votes
0answers
18 views

Significant box's test equality covariance matrices, How bad it is?

Having equal number of cases across two groups (60 total cases), if one wants to perform a split-plot Anova and finds out that the Box's Test of Equality of Covariance Metrices is less than 0.001 then ...
2
votes
0answers
38 views

Package ‘fitdistrplus’

I am trying to use the package ‘fitdistrplus’ in R to fit one non standard distribution to my data set. I am trying to copy the methods the package creators used in their tutorial for specifying a ...
0
votes
0answers
20 views

Fisher's Exact Test to assess the significance of a difference between false positive rates

I have trained two binary classification models on the same data and evaluated them using the same test set. For each model I have calculated a false positive rate (and a count of false positives) at ...
1
vote
0answers
31 views

Best statistical test to show significance of two data coming from the same population?

Hope i'm posting in the right thread. I have a sample of genes where I calculate a number of features in the start of each gene, and a number of features in the end of each gene. I plotted the ...
0
votes
0answers
15 views

Mann-Whitney U and KS tests in R are not producing significant results [duplicate]

I wish to compare two test data sets using the Mann-Whitney U and KS tests in R. As a start, I thought to compare two similar sets (an A/A test) expecting to see a highly significant indication of ...
1
vote
0answers
15 views

Intuition For Overlapping Confidence Intervals and Statistical Significance [duplicate]

Two questions: I have a really hard time with the intuition for why the second part of this statement is true: If two statistics have non-overlapping confidence intervals, they are ...
7
votes
2answers
187 views

Can a narrow confidence interval around a non-significant effect provide evidence for the null?

It is obviously fallacious to assume that failure to reject the null implies that the null is true. But in a case where the null is not rejected and the corresponding confidence interval (CI) is ...
5
votes
2answers
153 views

What could be a statistical test for comparing funnel data before and after important changes?

I have some funnel data from a website I manage. After some time I did some improvement on a multi-step purchase process. I wonder if it's possible to formulate a test that with statistical ...
0
votes
0answers
9 views

NPS confidence intervals with bootstrap

I am looking for a method to find confidence intervals for NPS (net promoter score). One method was already suggested here. But I am thinking of using bootstrap. Do you think the following procedure ...
0
votes
0answers
27 views

Non-Stationarity, first differences and Panel Data

I have build a sentiment index and am now validating its statistical properties and significance. I stumbled upon two problems. (1) dfglsindicates that my ...
1
vote
1answer
59 views

Does heteroskedasticity matter if you have a large enough sample?

Let's say you run a regression with over 200 observations. Would this reasonably large sample mitigate the impact of residuals heteroskedasticity as an offshoot of the Central Limit Theorem, or ...
2
votes
0answers
46 views

Test for statistical difference between two curves on same plot?

I've got data plotted for A/Ci curves (plant physiology, A=photosynthesis rate, Ci=internal CO2 concentration--essentially it's a dose-response relationship). Generally, when this type of data is ...
0
votes
0answers
28 views

statistical significance test on cluster analysis result

I've done a hierarchical clustering on a data-set composed by 33 subjects and 2 continuous variables (called V1 and V2), which produces 3 clusters. Now I'm wodering if it make any sense to perform a ...
-2
votes
4answers
71 views

What is relationship between “significant correlation” and “significant difference”

What is relationship between "significant correlation" and "significant difference" between two variables?
1
vote
0answers
11 views

Test which method is better if data is gathered on overlapping time windows (not independent)

I have two methods which are being used to estimate a specific signal. I have a ground truth measurement of this signal and these two methods are using noisy data to estimate this signal. This signal ...
2
votes
1answer
31 views

Standard deviations and significance values

I was wondering if data 2 SD from the mean is deleted as outliers, is it possible there after to report 0.01 significance values? Thanks!
0
votes
0answers
12 views

Classifier accuracy significance assessment in unbalanced train sets

I work with highly unbalanced training sets, and I would like to measure the probability of having certain classifier accuracy by chance in order to measure statistical significances of the accuracies ...
1
vote
0answers
9 views

Long Run VAR alpha and beta significance levels

I am using a VAR with 2 variables and 4 lags. I am combining the coefficients of these variables to get an overall alpha and beta value for in the form $Y = \alpha + \beta X$. In order to get the long ...
0
votes
0answers
27 views

How to model to improve the room usage efficiency based on motion sensor history

To reduce the confusion, I changed my application from traffic to meeting room, so this application is about modeling a meeting room efficiency , the data collection is built by placing a kind of ...
2
votes
1answer
31 views

How do I prove the significance of a non-linear model?

For example, for linear models, you take the p-value of the regression, and you deduce if the regression is or not significative. But with non linear models, in R, there isn't shown a p-value ...
0
votes
0answers
13 views

Determining Significance of Proportions

I've done a test where from a cohort of 112 people I've determined whether they respond or do not respond to the treatment. Now, the definition of them responding or not responding is whether ...
1
vote
1answer
44 views

How can I find a level $\alpha$ most powerful test?

Let $X_1,X_2,\dots ,X_n$ be a random sample from a distribution with pdf $f(x,\theta)$. Find a level $\alpha$ most powerful test of $H:\theta=\theta_0$ against $K:\theta = \theta_1$ when ...