Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [statistical-significance]

Statistical significance refers to the probability that, if, in the population from which this sample were drawn the true effect were 0 (or some hypothesized value) a test statistic as extreme or more extreme than the one gotten in the sample could have occurred.

0
votes
1answer
18 views

Comparing two discrete independent variables

I would like to perform something like two-sample Wilcoxon tests/T-test (paired = FALSE) to compare two method's expected scores. Here is the table of counts: ...
0
votes
0answers
14 views

Population proportion/Significance level: Why does p-hat change SE?

Couldn't find this specific question neither here nor on google in general, so hoping someone could help me out: Lets take a classic example of polling 100 people on which of two candidates they will ...
0
votes
1answer
17 views

Use Pooled-Variance T-test or Paired Difference Test?

I am not asking for the answers! I am just confused why the solution is using the pooled-variance t-test, instead of the paired difference test. The samples are undoubtedly from the same population ...
0
votes
0answers
17 views

Multiple Regression Effect Size, Significance, and Cohens f^2

I have a multiple regression with a continuous dependent variable, 1 continuous independent variable, and a handful of binary independent variables. The R summary is pasted below: ...
0
votes
0answers
10 views

comparing two sets of coefficient of variation values

I have two sets of coefficient of variations (cv): ste1 = c(cv1, cv2, ..., cv15) set2 = c(cv1, cv2, ..., cv15) how can I compare set1 with set2? which test should I apply? I know that if I want to ...
0
votes
1answer
23 views

Testing if two countries have different GDPs

So I have a data set showing a GDP for 50+ countries. I want to see if the difference between two countries are significant. I am thinking I would use a two sided t test because the data is parametric ...
0
votes
0answers
14 views

Can a permutation test be used to evaluate a model?

Let's assume that we have used a given predictive model to generate predictions for an evaluation data set. Now I would need to use these predictions to decide how good is the model. Or, more ...
4
votes
1answer
159 views

u-shape for logistic regression?

I'm stuck and have real problems what to interpret from my current results. Maybe you can help me out? Thanks! Lets say... I'm investigating on the influence of health factors on dying. Dependent ...
0
votes
0answers
15 views

Measure statistically significant relationship

I've a some data related to servers, there are mainly 2 tables I'm interested in analyzing. server_id | service_type | server_temp | cpu_utilization the other ...
0
votes
1answer
40 views

How to conclude, based on samples, whether the mean annual rate of return is higher for NYSE than for NASDAQ [on hold]

The following data on annual rates of return were collected from eleven randomly selected stocks listed on the New York Stock Exchange (“the big board”) and twelve randomly selected stocks listed on ...
0
votes
1answer
20 views

Correcting for multiple post hoc tests in factorial ANOVA

I have done a two-way repeated-measures ANOVA with factor A having 3 levels and factor B having 5 levels. I am doing post hoc comparisons between all 3 levels of factor A within each of the 5 levels ...
0
votes
0answers
17 views

T-Test to test mean of subsample [on hold]

I have a data set of CDS spreads for 22 nations and would like to find out how the CDS spreads react to positive and negative credit rating announcements for the timeframes [-1,0] and [0,1]. I am ...
0
votes
1answer
43 views

How to determine if one predictive model is statistically significantly better than another one?

I have a data set, two competing predictive models (regressions) and I need to decide which predictive model is better. Let us also assume that I have a measure of accuracy (for example mean squared ...
0
votes
0answers
20 views

Testing for Significant Difference between series of Likert Scales

sorry if this or a similar question has been asked before I'm new to stats and a lot of it is currently over my head. I have a series of Likert scale questions (similar to the below) which I'd like to ...
0
votes
0answers
14 views

Statistical test for independence of a sequence of behaviours

I collected data from a social experiment. The data consists of a set of interaction behaviours of the participants. Each participant can perform 2 behaviours, either A and B, and they can perform ...
0
votes
1answer
26 views

How to interpret results on different t-tests for the same samples?

I am performing a t-test on the same couple of vectors in R. The first vector is the list of daily returns on a stock. The second vector is the list of daily log - returns on the same stock. Now, I ...
0
votes
1answer
27 views

Statistical test for the difference in mean for two normal distributions

I have measured $x$ and $y$ and estimated their uncertainty. They are both distributed normally. I found $x = 5 \pm 0.25$ and $y = 7 \pm 0.5$. I want to know if these estimates are statistically ...
0
votes
0answers
12 views

When determining is a sample mean is statistically significant why do we use the proportion outside the value as its probability? [closed]

So in stats we are learning how to test if a sample mean is statistically significant, that is, it's probability is less than .05, using the normal sampling distribution. However, I can't wrap my head ...
4
votes
0answers
50 views

What is Deborah Mayo's “severity”?

Can anyone give a detailed (and clear) explanation of what her "severity" means (isn't it just the power function assessed at different discrepancies taken as null hypothesis?) and how it fits in the ...
0
votes
0answers
16 views

stats for psychology degrees of freedom question [closed]

A dependent sample t-test is run with 50 participants taking a pre- and a post-test. What are the degrees of freedom for this test?
2
votes
1answer
249 views

Does the 2-sample KS test work? If so, why is it so unintuitive?

Here is an example. I have two data sets as shown below as histograms: I want to examine whether these data sets are drawn from the same underlying distribution. To do this, I am using the 2-sample ...
0
votes
2answers
45 views

What type of test I shoud use between two samples to compare mean equal.? [closed]

I would do Wilcoxon rank sum test to compare mean equal. In R >wilcox.test(a, b) We can assume non-normality of the samples.
0
votes
0answers
23 views

Statistical Test According to Independent Variables

I wish to test a statistical significance between means of data groups. The groups are determined by a continuous variable, however this continuous variable is applied to different test subjects (...
0
votes
0answers
38 views

Advice on choosing the most appropriate significance test (for some simple data)? [closed]

I'm asking for advice on how to pick right tests and mainly how to justify them. For the sake of data, I made a plot which shows a continuous approximation of the number of suicide-thought :| per ...
0
votes
0answers
13 views

Comparing statistics over win/loss

I have data from a league over a season. The season was conducted as a round robin so everyone played each other. Each game was a best of 5 so at the least a game produced 3 matches and at the most ...
0
votes
0answers
22 views

Statistically verify technique for outlier detection? [closed]

I have a question about the outlier detection technique, I have bunch of data for outlier detection, and I used the z-score technique to get the outlier values, and i will say I choose the 1 standard ...
3
votes
1answer
31 views

Looking for a statistical test which will test whether the “spread” from a line is statistically significant

Below are two examples of two x-y data pairs: The top example clearly has more "spread" from any sort of linear trend between x and y. The data points are clustered together but there are some ...
0
votes
0answers
23 views

Binomial distribution, hypotheses testing

Suppose the following instance of the hypotheses testing: suppose that the true state of the coin is that it is either exactly fair, or that it shows heads with probability $0.51$, and there are no ...
3
votes
3answers
409 views

Is there a constraint on the sum of the type-I & type II error probabilities?

Is it true that if $H_0$ and $H_a$ are complementary hypotheses of the Binomial trial, i.e., the negation of $H_0$ is $H_a$ then the type-I error $\alpha$ plus type-II error $\beta$ equals 1? Or is ...
0
votes
1answer
20 views

which statistical test should I use: 5 schools

I have conducted a study in 5 schools to investigate whether use of a game could improve math performance. In each school there were two classes, one experimental (used the game) and one control (did ...
0
votes
0answers
10 views

Are differences in proportions meaningful in a sample's segmentation

I am setting up an A/B test of a webpage where users will be randomly distributed between either A or B. I'll be using the N-1 Proportion Test to determine which page converts better than the other. ...
0
votes
1answer
24 views

Unsure which data to use for ANOVA test

I have determined that a one-way ANOVA test of variances will work best for my data. However, the nature of my data makes me very concerned about making any claims based on significance or not. My ...
1
vote
1answer
19 views

Compare a real-world process with model prediction

I have a real-world time series process with an dependent variable Y responds to multiple independent variables (not necessary a linear response but can assume so if necessary). A non-statistical ...
23
votes
3answers
3k views

Why does the Kolmogorov-Smirnov test work?

In reading about the 2-sample KS test, I understand exactly what it is doing but I don't understand why it works. In other words, I can follow all the steps to compute the empirical distribution ...
2
votes
1answer
42 views

What are the problems if in a AB test the control group is not big enough?

I've been asked to analyze some data but I'm quite new in statistic. I want to compare the means of a continuous parameter in two independent population A and B: A can be seen as the treatment group ...
0
votes
0answers
7 views

Significance test in trimodal distribution [duplicate]

I have a trimodal null distribution that looks like this: In order to calculate the significance we just check how extreme the values are respect to our null distribution. We do it this way: ...
0
votes
0answers
11 views

Misleading two-way ANOVA table interpretation

How can the hypothetical cases below (if possible) be explained? Assuming randomness, normality, homoscedasticity, etc 1-Overall effect (Fbetween) is significant Individual factors are NOT ...
0
votes
1answer
35 views

Type 1 and Type 2 errors trade-off

Reducing Type 1 error will always result in increasing the Type 2 error This statement is false. I understand the definitions of Type 1 and Type 2 errors. What I understand is that there is, in fact, ...
0
votes
1answer
32 views

Is there a relationship between the response and predictors?

I have been reading Introduction to statistical learning and I was going through Multiple Linear Regression. This is the topic that Im reading As i was further reading I encountered an equation that ...
0
votes
0answers
34 views

Significance test of procentage number with unknown variance?

Can anyone help me clarify if its possible to do significance test given the data below? I want to calculate if the procentage of people who remember having seen a video is significantly higher for ...
0
votes
0answers
11 views

How to test for significance of bias in time series forecasts

Let's say I have some data $y(t)$ for a given series of $t$ values and also some forecasts from a black-box model $\hat y(t)$ over the same values of $t$. The mean of $y(t)$ is greater than the mean ...
1
vote
0answers
23 views

Does Probability and P-value same thing? What is actually P-value means in hypothesis testing (t/ ANOVA / Tukey)? [duplicate]

We have been taught that P-values is "calculated probability". In this past 10 years, I have been using statistical test (t/ ANOVA / Tukey) more than thousand times. I have published scientific papers ...
0
votes
0answers
9 views

Most suitable comparison between groups

I have a neural network based algorithm which it was trained with 5 different datasets (A, B, C, D and E). For each one, 10 training runs was made {(A1, A2, ..., A10), (B1, ..., B10), ...}. Then, it ...
1
vote
0answers
30 views

Test for occurrences of pairs from samples with different distributions?

I have two collections of data, A and B, each containing a large number of 10-digit binary strings. For instance, an element of A could be a=1011001111, and an element of B could be b=1001010000. I ...
1
vote
0answers
16 views

Find significantly differences between multi-variant groups

I have 11 groups, each consist of about 200 observations, and each observation has 2 variables (which are actually the first two PLS components if it makes any difference). Each group is a substance ...
0
votes
0answers
25 views

Statistical inference of a variable based on categorical respone levels.

Suppose that we have a variable $X$ with three levels $1$ ,$2$ and $3$ accordingly.Also we have a catergorical respone variable with two levels $Y=\left\{Healed,Not Healed\right\}$ which tells as if ...
0
votes
0answers
10 views

Shadow significance

I've faced specific issue recently and kindly ask you to help. Imagine standard linear supervised learning framing for binary classification problem (X,y, OLS, p-vals, etc.). One can develop common ...
0
votes
0answers
10 views

Is there a relationship between the “significance level” of a KS test and the size of the error bars in a given data set?

Suppose I have a data set with error, $\mathbf{d}+\mathbf{\Delta d}$ where the error is some percentage of the data (e.g. $\mathbf{\Delta d} = \epsilon \mathbf{d}$, $0 < \epsilon < 1$). I also ...
0
votes
1answer
21 views

How to test two or more than two incidence rate (new event/person-time) in r

The Definition of incidence rate is defined as : new event/person-time Now there's my data: ...
0
votes
0answers
61 views

Does this correlation make sense?

I'm trying to work on a ML project and I have a dataset and I'm trying to see if there is a correlation between some of the features in my dataset. The dataset contains inspection notes for car parts ...