All Questions
223 questions
2
votes
2
answers
34
views
Paired sample (data) with unequal sample size
I am reading a publication in which the author mentioned that they used the paired t-test. In their study, a questionnaire was provided to a group of people before and after surgery to check the ...
- 155
1
vote
1
answer
37
views
Is a binomial of Chi-Square or t test the correct test to compare dichotomous variables?
You have an ML model that outputs 100,000 out of a million observations of a dichotomous variable 0 or 1. You want to see whether the distribution of the outputted 100,000 observations of the model is ...
- 21
0
votes
1
answer
39
views
P-value of 0 when comparing two treatments using bootstrap method
I am comparing two treatments and doing a t-test or a Wilcoxon test I get p-value of 2.2e-16.
I wanted to try bootstrapping because the samples are large (over 1000 ...
3
votes
2
answers
184
views
multiple t test analysis or indidivual t tests
I have to analyse samples with t tests. I am not sure if I should treat my samples as individual t tests or multiple t tests. here the details:
I have an experiment where I analysed the concentration ...
- 51
5
votes
3
answers
524
views
Welch t-test p-values are poorly calibrated for $N=2$ samples
I am performing a large number of Welch's t-tests (t-test with unequal variance) on very small sample sizes, often with only two samples per condition. I am finding the p-values are poorly calibrated: ...
- 153
9
votes
4
answers
554
views
Variable is significant in multiple linear regression but not in t-test of the subgroups
I am getting different p-values for a variable in t-test and its coefficient in multiple linear regression so I am unsure which one to believe.
As an example, my hypothetical dataset consists of ...
- 93
1
vote
1
answer
47
views
How do I do the p-adjustement in statistical tests?
context: a study about which type of bread and/or filling attracts ants the most
I have a table with 3 columns : the number of ants attracted, type of bread, type of filling. The question is : do ...
- 43
0
votes
0
answers
34
views
Difference between a comparison of independent variables t.test and linear regression model of one category?
The thing is I am trying to obtain confidence intervals of several variables (food, blood chemical analysis...) of patients who are divided in two groups according the intervention. Until now I was ...
- 437
0
votes
2
answers
48
views
Help with T test assuming equal variance results (p value and t critical value not leading to same answer)
Using the p value and negative t value with t critical value my results are said to be significantly different but using the positive t value with the t critical positive value the results are not ...
- 1
2
votes
0
answers
46
views
T-tests for differences of slopes of linear regression when intercept is forced through 0
I am looking to compare the differences in linear regression slopes between two datasets and at first I used a two sample t-test. However, when I force the intercept through $0$ for both regressions (...
- 21
5
votes
2
answers
618
views
How does reducing my alpha-threshold to 0.005 affect my required sample size?
Among statisticians there is an ongoing debate whether the long-standing "standard" threshold for declaring the outcome of a statistical test as "significant" (a.k.a. alpha-level) ...
- 356
5
votes
4
answers
902
views
Sample Size impact on Effect size
I have a small sample size ($n = 23$) and I ran some t-tests to see if there were any group differences (group one had $11$ participants and group two had $12$ participants). I can wrap my head around ...
- 51
2
votes
0
answers
55
views
TOST procedure + power analysis: how to interpret t-value inside bounds
I conducted a TOST procedure according to the guidelines by Lakens [1] and received the results from the table below. For "discontinuity", "fragmentation" and "...
1
vote
1
answer
84
views
If you change the sign of a two tailed test does the p value remain the same?
If the results of a t test are reversed are the p values the same?
I am running an experiment:
Treatment: sample size - 5k and metric value - 0.57 Control: sample size - 5k and metric value - 0.62
The ...
- 65
0
votes
1
answer
40
views
If two interventions, A and B are both effective i.e p-value is significant, is there a way to know whether A is MORE effective than B or vice versa?
Im new here and also consider me as a total beginner in statistics.
So in my scenario, lets say there's two learning interventions, Intervention A and B. The participants are divided into two groups, ...
2
votes
2
answers
72
views
Which type of t-test is best to use for my data
I'm looking for some guidance on which type of t-test is most suitable to use for my data.
I want to compare the means of the variable 'Rate' between two groups in my data. I have 6 years of data and ...
- 21
1
vote
1
answer
79
views
Asymptotics of $\mathbb E[-\log(p)]$ in a one-sample t-test as $n\to\infty.$
Consider a one-sample two-sided t-test, i.e. $X_1, \ldots, X_n$ are iid. $N(\mu, \sigma)$ random variables and we want to test $H_0\colon \mu=0$ versus $H_A\colon \mu\neq0$.
The $t$-statistic is ...
- 151
2
votes
1
answer
281
views
Performing a one sample t-test with `scipy.stats.ttest_1samp` gives unexpected p_value
I have a model metric (a single value) and a list of sample metrics from benchmark quasi-random models that I would like to do a t-test on to figure out if the model metric result is significantly ...
- 123
3
votes
3
answers
517
views
Countering t-test "any feature is significant" results for large sample size datasets
I'm doing some analysis over natural language data, which basically entails:
Computing some feature over all samples.
Evaluating if this feature statistically significantly discriminates between ...
- 31
1
vote
2
answers
241
views
Is there a statistical measure that quantifies the differences between samples? [duplicate]
Given some samples that each containing thousands of data points, I want to compare which sample is drastically different from other samples. Since the sample size is big, any statistical tests will ...
- 111
0
votes
1
answer
35
views
How to perform a proportion test comparing noise proportions in different groups?
I have a dataset containing observations of wave lengths in milliseconds and the corresponding durations of noise in milliseconds. Each observation is labeled with a group (A or B) and a subject.
I ...
6
votes
3
answers
1k
views
P-value adjustment for a single test with low sample size
I have a dataset of 2 conditions. Each condition has 15 measurements. I tested them using paired t-test to find if the difference is statistically significant. Should I use a p-value correction method ...
1
vote
0
answers
44
views
Which Non-Parametric Test should I use?
I want to make sure I am choosing the right statistical test.
I am working with 7 semesters of exams scores for a particular course.
Each semester the course ran with many sections. However, the ...
- 11
0
votes
0
answers
58
views
Could you recommend a textbook for t-test and p-value?
I am a graduate student and now I have a trouble to understand p-value. I watched youtube videos, but they didn't really help. I want to know it a bit in depth. Could you please recommend a book for ...
1
vote
0
answers
44
views
Reporting or combining multiple t-test results
I am analyzing an anomaly event of daily stock returns. For many different stocks (>1000), I do some analysis to identify a day where I observe this anomaly.
To evaluate the effect of this anomaly, ...
- 121
2
votes
0
answers
38
views
How to manually calculate p-values for lmers?
Hopefully this is not a duplicate. Provided the t-statistic, SE and 95% CI, how can I manually calculate p-values for lmers in R? More details in the quoted link below.
I've used ...
- 843
1
vote
1
answer
365
views
T-test p-value from lm() function does not equal t.test() function in R
I thought the the following code in R should run equivalent t-tests. But the p-value obtained from each method differs. The p-value using lm() is 3.415937e-05 but using t.test() it is 0.0001098368.
...
- 14.6k
0
votes
1
answer
236
views
emmeans returns similar p value for different confidence levels
I am computing multiple comparisons of the mean of an unbalanced design using estimated marginal means. I would like to see the significance level of the differences between treatments for each ...
- 23
3
votes
1
answer
246
views
How to mathematically express a p-value in terms of the t-statistic in a one-sample t-test
I am writing out a t-test by hand and I am confused about how to mathematically express the step of calculating / looking up the $p$-value of the t-statistic. This is what I have so far from the ...
- 33
0
votes
1
answer
130
views
R computes strange p value [closed]
I have two different t statistics which I want to use to compute corresponding p values. I don't understand why the second p value is what it is, when I compute it manually with ...
- 103
0
votes
1
answer
63
views
What is the best method to test the null hypothesis on volatile data?
I am trying to find a good method to test the null hypothesis(H0) on two unpaired samples. Those samples come from two different HTTP Servers and the unit I'm using is req/30s (requests concluded in ...
- 65
6
votes
3
answers
929
views
Student's t-test on "high" magnitude numbers
I am trying to calculate whether the difference between the two benchmarks is statically different or not.
The input is req/sec of a HTTP Server and I'm using ...
- 65
1
vote
1
answer
356
views
Why are my p-values for lognormal distribution mean not uniform?
I was wondering why my p-values for lognormal distribution are not uniform while using t-test for mean. Here is my code:
...
- 125
0
votes
1
answer
384
views
What does a low P-Value actually suggest? [duplicate]
Post Edit: (I believe the question I ask here has not been answered in other questions regarding p-value on stats.stackexchange, mine in synthesis asks "does a low P-value mean this (read ...
2
votes
1
answer
5k
views
How can I choose between homoscedastic and heteroscedastic?
I want to calculate the p-value between subgroups of my samples. For that, I am using the T.TEST function of Excel. But I do not understand the last parameter, type:
Paired
Two-sample equal variance (...
- 21
1
vote
0
answers
210
views
Is there a way of comparing/ evaluating a machine learning model with a recurrent model with a statistical significance test?
Comparing two different machine learning models (to assess if the difference between the mean performance is real or not using P-value and t-Statistic) is possible and strait forward in Python. ...
- 21
1
vote
0
answers
186
views
T-testing on sparse data
Problem: I have two stochastic processes, $S_1$ and $S_2$, that frequently are zero, but occasionally have positive values with unknown probabilities $q_1$ and $q_2$. e.g.
$$
S_1 = \{0,0,0,0,0,21,0,0,...
- 11
0
votes
0
answers
64
views
Understanding the relationship between a 'sampling distribution of a statistic' and a 'population distribution' in the context of a 'hypothesis test'
I would like to confirm that I am understanding the relationship between a sampling distribution of a statistic (an example of a 'statistic' would be a sample mean $\bar{x}$) and a population ...
- 71
1
vote
1
answer
95
views
0
votes
1
answer
83
views
Limit of t-score as r->0 (for Pearson's correlation)
I have a very basic question. The t-score used in the significance test for the Pearson's correlation is given by:
$$
t = r \frac{\sqrt{(n-2)}}{\sqrt{(1-r^2)}},
$$
where $n$ is the sample size and $r$ ...
1
vote
0
answers
95
views
Can I use both a Kolmogorov–Smirnov test and a T-test to compare two distributions?
I have two distributions (N ~ 300, μ = 1.89 and σ = 0.97) and (N ~ 40, μ = 1.74 and σ = 1.18).
A ks_2samp returns statistic = 0.2 and p-value = 0.057. Therefore, I cannot reject the null hypothesis ...
- 51
3
votes
2
answers
278
views
How to describe in words the logical conclusion of a one-tail t-test?
Lets say we have the following null and alternative hypothesis:
$$H_0: \mu = \mu_0, \; H_A:\mu > \mu_0 $$
Thus, the p-value in this situation represents the probability of obtaining test results ...
- 719
1
vote
2
answers
817
views
Why does null hypothesis in simple linear regression (i.e. slope = 0) have distribution?
I have been reading about simple linear regression ($y=wx+b$), and I started a section when it talks about null hypothesis $w=0$. Then, when it wants to calculate p-value, they use $t-score = \frac{\...
- 113
1
vote
0
answers
80
views
What role does t-statistic play in hypothesis testing?
I'm wondering what role t-statistic plays in hypothesis testing and the interpretation of the result.
In a two-sample two-tailed t-test where we have $H_0$: $\mu_A=\mu_B$ and $H_1$: $\mu_A \neq \mu_B$,...
- 71
0
votes
1
answer
109
views
Extremely low p-value, even though null-hypothesis should NOT be rejected
I'm doing hypothesis testing to check whether one variable affects another, by looking at the slope of the trendlines, on 268 datasets. From the analysis of each dataset (which contains many samples), ...
- 153
0
votes
0
answers
69
views
Combining One-way ANOVA followed by Fishers LSD and unpaired ttests?
I am analysing data on levels of proteins in the brain, there are 4 groups, I did a one way anova followed by Fishers LSD. I also carried out individual unpaired ttests comparing each group to the ...
- 1
3
votes
1
answer
599
views
Approximate real significance level and power from simulated p-values
I run $N = 1000$ simulations during which I generate $X\sim N(0,1)$ and $Y\sim N(1,1)$. During each simulation I conduct a t-test on the equality of means ($H_A: \mu_X \neq \mu_Y$) and find a p-value ...
- 261
2
votes
1
answer
737
views
T-Test (independent samples) alternative hypothesis 1[unequal means] is not significant, but alternative hypothesis 2 [mean 1 < mean 2] is significant
I conducted a T-test to see if two independent groups (English (n=324) vs. German-speaking participants (n=346)) differ in their motivation to do activities for fun (Likert scale instrument).
I use ...
- 21
0
votes
1
answer
166
views
Help understanding Anova Three Factors
I am looking to understand the following table from this article to better understand some statistical stuff.
The article is exploring the outcomes of meat via 3 factors(marinade, temperature and ...
- 135
2
votes
1
answer
746
views
Why is the $p$-value significant but the confidence interval includes zero in a $t$-test?
I am interested in learning whether the two groups below are statistically significant:
...
- 217