Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

learn more… | top users | synonyms (2)

1
vote
0answers
10 views

How does imputation affect significance test power?

There are many different imputation methods that make different assumptions about the missingness of the data (MCAR, MNAR, MAR). Since this assumption may not be met, I'm interested in knowing how the ...
-3
votes
0answers
23 views

The proof of lasso regression solution? In which it shrinks some of coefficents to zero? [duplicate]

I would like to know how the lasso method shrinks some of coefficients exactly to zero, "for example could show me how that works mathematically if there are two parameter". For example, with ridge ...
0
votes
0answers
13 views

How to compare means of 3+ variable within a sample group

Grip strength for 128 athletes was measured at 3 various elbow angle (90, 180, 45) with varying position and arm posture. Now I want to compare means of dependent variables within each elbow ...
0
votes
0answers
8 views

Resources for Determining Value-Added by Parameter Sweeps

I am working on a project that aims to use Design of Experiments to improve our testing process. We are performing tests using a parameter sweeps of numerous input variables (40+). Each input variable ...
0
votes
0answers
26 views

Generate $r_i$ following condition $\sum_{i=1}^n |\frac{r_i}{\sigma_i}|^2\leq\chi^2_{n,\alpha}$

How can I generate $r_i$ for $1 \leq i \leq n$, such that $\sum_{i=1}^n |\frac{r_i}{\sigma_i}|^2\leq\chi^2_{n,\alpha}$, where $\sigma_i^2$ is the variance of $r_i$ and, $\chi^2_{n,\alpha}$ is a chi-...
2
votes
1answer
57 views

K-S test is too sensitive

I'm trying to compare two samples to see if they come from same distribution. I'm not just interested in the measures of central tendency but in the overall shape and nature of the distributions. ...
0
votes
0answers
20 views

How does the symmetric matrix A= X' X is found to be QDQ'? [duplicate]

I am confused of how does the symmetric matrix A= X'X is found to be QDQ' ? where D is the diagonal matrix of A, and Q is orthogonal matrix i.e Q'Q=I identity matrix.
0
votes
0answers
14 views

Binary interactions and significance levels

I am a bit confused by the results from a logistic regression using two-way and three-way interactions (binary variables only). My first regression output was looking at binary predictors of TB and ...
0
votes
0answers
25 views

How to prove significantly high or low values?

I have the counts of genes in particular pathways from three organisms. How to statistically prove that few counts are significantly high or low as compared to another organism. My data looks like ...
0
votes
1answer
27 views

what test to use for comparing count data across multiple groups

I have five groups of students and 40 in each. And we counted the number of references spelling mistakes in their homework. We would like to compare to see if there exists any groups differences in ...
15
votes
2answers
486 views

High variance of the distribution of p-values (an argument in Taleb 2016)

I'm trying to understand the big picture claim made in Taleb, 2016, The Meta-Distribution of Standard P-Values. In it, Taleb makes the following argument for the unreliability of the p-value (as I ...
0
votes
1answer
23 views

Poisson normal approximation for comparing means of count data

I'm trying to compare the values of two counts from an A/B test, $c_1$ and $c_2$. If I assume the data is Poisson distributed then the mean and variance are $\lambda$ and the standard error is $\...
-1
votes
0answers
33 views

Post hoc test in two way ANOVA analysis

I have nine cultivars in which i measured the protein content at each day of high temperature treatment. I have collected the data from 5 days in all cultivars. Then i perform the two way anova, i got ...
1
vote
0answers
9 views

Poisson Regression vs. T-test for Balance

I have many subjects that were cluster-randomized to a treatment group and to a control group. I want to check the age balance of the groups following the randomization - e.g. whether there is a ...
9
votes
6answers
1k views

Brexit: was “leave” statistically significant? [closed]

In this post we ask a question about a natural phenomenon called humans attempt to find decision by counting votes. The specific incident of such natural phenomenon that this question is about is the ...
0
votes
0answers
24 views

Identifying statistically significant categorical variables with rma.mv

I'm running a meta-regression/multi-level analysis that contains only categorical variables. The printout of the data is as follows: res.fe ...
0
votes
2answers
32 views

Comparing huge data sets that are non-normal and different lengths in R

I'm comparing two data sets. Each set is extremely large, about 25,000 quantitative pieces of data in length. I want to find if these sets are significantly different, but the problem is that they are ...
1
vote
1answer
19 views

Grouped monthly sample sizes

This will be really simple, but I'm having a hard time figuring out how to Google this. Let's say we have 12 months of trailing data on something like "Calls per month," and that data is on the low ...
2
votes
2answers
298 views

The value of an Effect Size

I calculated a Cohen's d value of d= -2.1. I understand that there are small, medium, and large effect sizes. But in my case the d value is negative? Would it still be considered large since abs(-2.1)...
2
votes
0answers
53 views

How can I get p-value by using ecdf and bootstrapping?

First of all, forgive my for my ignorance about this concept. I might ask a basic question but what I have read from Exploratory Data Analysis: 2 Ways of Plotting Empirical Cumulative Distribution ...
3
votes
2answers
69 views

How to interpret Fisher Test?

How do I interpret the odds ratio for this fisher test output in R? ...below is my matrix that I input into fisher.test in R: ...and below this is the output of the test in R:
0
votes
0answers
17 views

BH procedure for FDR [closed]

If we will use the BH procedure at level q we know that the FDR will be lower then $\frac{m_0q}{m}$ when $m$ is the number of the overall comparisons and $m_0$ is the number of true null hypothesis ...
0
votes
0answers
16 views

Compare predictive power of 1 model on different data

I would like to test if the odds of bets on football matches with higher betting volume are more efficient (i.e. predict the result better) than bets with lower betting volume. I use a probit model ...
2
votes
2answers
125 views

How to determine if proportions are significantly different

I have a database of purchase events. Each event has a field green (a boolean field that determines whether the item was green) and a field ...
1
vote
1answer
22 views

Statistical significance changing in hierarchical regression?

What does it mean if, performing hierarchical linear regression, significance of a variable changes? So, in one step, statistical significance of a variable is 0.006, but in the next one (after adding ...
1
vote
0answers
19 views

Determine effect of two variables together vs seperate

I have a data analysis problem. I am trying to identify the effect that a number of binary variables (v1, v2, v3... v25) have on a numeric outcome measurement, let's call it the 'W-value'. I have ...
0
votes
1answer
20 views

How to draw a conditional distribution graph for B given A

Below is a frequency table which has data for testing a theory that students who speak a foreign language are also strong mathematics students. The question says to draw a graph showing conditional ...
4
votes
4answers
394 views

A p-value greater than 0.05 means that my results are meaningless?

I just came across a correlation factor of 0.70 but p-value 0.054, does this mean that my correlation factor is completely meaningless and I should try to provide more samples?
0
votes
0answers
29 views

Why do most natural language processing papers fail to give any significance test when reporting results?

Most natural language processing papers (> 90% from my experience) fail to give any significance test when reporting results. Is there any reason for that aside from the fact that it takes a bit of ...
2
votes
0answers
16 views

Hazard model for three-tiered-response dependent variable

My time-series dataset is composed of measurements for a number of independent variables and one dependent variable. The dependent variable accepts 3 responses: weak reaction, strong reaction, or no ...
0
votes
2answers
25 views

Comparing data from single samples

I measured mutations after drug treatment in two different lines (genotype A and B) at 3 different timepoints (1,2,3). Is there a test to see whether A1 is different from A2, A2 is different from A3, ...
4
votes
3answers
135 views

Real-world example on significance testing with large samples

The issues that arise when one uses statistical hypothesis testing framework with large samples are dissussed on CV (see e.g. this thread). The main problem discussed there is that in the real world ...
0
votes
1answer
41 views

What statistical test to use - comparing models?

Say, I have created 3 models that evaluates the amount of harmful pollution. It uses a variety of factors (location, time of year, etc.) to determine this. I have a bunch of data for 10+ Cities from ...
0
votes
0answers
6 views

How do I find if two categories of ratio data are different in a manner that is statistically significant?

I have 4 categories of data, A, B, C, and D, that are ratios (market share). Each category has different numbers of records (prescriber-level data). Each record is a market share percentage that is ...
0
votes
0answers
11 views

P-value calculation on difference in correlation

I have a slightly basic statistics question. I have two large correlation matrices (PCC) (one in wildtype condition, other in treatment condition) of related features. They are about 10000x10000. What ...
0
votes
0answers
11 views

How to measure the impact of a product on the global revenue?

We have products daily revenue and when each product started being sold by the company. We also have different break-down of product revenue by location, store, sales person, etc. We need to measure ...
0
votes
0answers
20 views

What statistic test should I run?

I am analyzing metagenome results from NGS sequencing. Right now I have two group of samples LR and HR which are divided in the case of high or low risk of hipercholesterolemia. I've performed box ...
1
vote
1answer
19 views

Statistical Test for Embedding?

I have a bunch of labeled objects. Let's say there are K classes altogether. Now suppose every object is mapped to a data point in R^n, or in other words, is embedded in R^n. Ideally a good embedding ...
0
votes
0answers
18 views

Check if my selection is better than a random selection from the population

I have a list of variables with scores associated with each of them (17k variables, scores are not normally distributed). I independently select a small number of these variables (10-30) and get the ...
0
votes
0answers
31 views

Can I flip the null and alternative hypotheses for logit regression in R?

I would like to accept the hypothesis that a variable is insignificant in determining the dependent variable of a logistic regression, but in R the glm summary which contains the p-values for each ...
2
votes
0answers
32 views

Null Hypothesis vs alternative hypothesis

I have a question that I can't seem to get my head around. The company repeatedly claims that 80 % of their clients are satisfied with the product. Use an hypothesis test at the 5% level of ...
0
votes
0answers
13 views

Repeated measures MANOVA for testing difference among multivariate time series?

I measured behavioural multivariate time series (3 variables) from 7 dyads performing an experimental task. For each dyad, I have a multivariate time series (that is, I don't have data from each ...
2
votes
0answers
35 views

How can I compute statistical significance with cross-validation? [duplicate]

I've done nested cross-validation with 10 outer runs and 10 inner runs to evaluate different classifiers. In the end I average the accuracies of the 10 outer cross-validation runs to get a final ...
2
votes
0answers
24 views

How can I compute the confidence level in the approximate randomization test?

(1) indicates that one can compute a confidence level from the approximate randomization test (step 4): However, (1) simply refers to (2) to view the actual calculation or table lookup. ...
1
vote
1answer
16 views

2X2 ANOVA: low power, statistical significance for only 1 IV, medium-low effect size

BACKGROUND: I run a 2X2 Between-Subjects ANOVA. I obtained: a low power (< 0.30) for both Independent Variables a not statistically significant result for one of the Independent Variables. ...
0
votes
0answers
30 views

estimate NND in the cluster with missing data

Would very appreciate any comments on the following problem: Imagine one has N sets of elements (points on the plane), coordinates (x,y) of 30% of the points in each set are known. So if we have 30 ...
0
votes
1answer
29 views

How can I determine significance of the presence of one variable on a binary dependent variable?

I want to know if the presence of one variable significantly determines the dependent variable (which is binary ie. can only be 0 or 1). Is there any statistical analysis, other than computing a ...
2
votes
0answers
24 views

probability of a single value belonging to a population

what is the appropriate approach to estimate the probability of a single sample belonging to a population? Specifically, I am calculating a particular value from the comparison of two time series. ...
1
vote
0answers
21 views

Specifying a convergence criterion for the approximate randomization test

All documents (e.g., (1,2)) that I have read on approximate randomization test indicate that one needs to choose the number of shuffles prior to running the approximate randomization test. Example ...
2
votes
1answer
10 views

Fixed effects with duplicate observations

I'm trying to estimate a Poisson regression model. My dataset spans 78 neighborhoods in some city, across 11 years, and has been drawn from multiple sources: the dependent variable (a crime count for ...