Tagged Questions

2
votes
0answers
35 views

Algorithm for determining performance speedup/slowdown in a code change vs. historical data?

I work on a large programming team, and I run a suite of performance tests on every change that is made in our program, which basically measure time it takes to run the test. For every code change, we ...
1
vote
2answers
54 views

Multiple t-tests vs. one-way ANOVA

I'm working on a classification problem and I have a very high F1 baseline of 85%. I have trained three classification models and I want to know which one is the best. How can I do so? I tried two ...
1
vote
1answer
56 views

How to perform T-Test in R including hypothesis

I would like to do a t-test in R. Unfortunately, statistics has never been my best subject, so I'm wondering what the output of t.test in R exactly mean. For ...
2
votes
0answers
32 views

Comparison of dependent but unpaired data sets

I'm trying to find a statistical test that can help me compare the means of two dependent groups. My data: parameter values from white matter region in an in vivo rat brain parameter values ...
2
votes
0answers
20 views

How to test significance of pre-post difference scores on 8 measures (1 group)

I have a set of pre and post treatment scores on 8 measures. I'd like to test which of these show significant improvement after treatment. If I use within-subjects t tests it will mean carrying out 8 ...
0
votes
0answers
46 views

Repeated 100x 10-fold cross validation, what is the sample size when doing an significance test?

I iterated my 10-fold cross validation 100 times for several methods. Now I want to use a t-test to test if the results are significant. However, I'm not sure what the sample size is. Is the sample ...
0
votes
0answers
47 views

Evaluate Quantile Regression results and t-stats at different levels.

I have a few questions regarding quantile regression and how to interpret the results. I have several independent variables and I want to figure out which combination of variables have the highest ...
0
votes
1answer
121 views

How are the p-values of the GLM in R calculated?

I have been running some binomial logistic regressions in R on a data set and I realised that the p-values of the estimated coefficients are not computed based upon a Normal distribution. For e.g. I ...
1
vote
0answers
41 views

What is a meaningful signifcance level when considering a pre-test and post-test model to assess learning in a college course?

When using a pre-test/post-test assessment to measure college learning, what is a meaningful significance level? I realize that p<.05 is the minimum criterion, yet with a correlated-groups t-test, ...
0
votes
2answers
195 views

Statistical Difference from Zero

I have a set of data that represents periodic readings. The data shows an upward trend but I need to test for a statistical difference from zero. I believe I should use the t test, two tailed but what ...
0
votes
1answer
98 views

Determining the statistical significance of differences in a small dataset

What would be the best way of determining the statistical significance of the differences in concentration between trials of a dataset such as this one: ...
1
vote
0answers
47 views

Analysis of infection data between two years using parametric or nonparametric tests?

I have three years of infection data from 6 different hospitals. ...
0
votes
0answers
71 views

Test for significance for multiple non-normally distributed samples

I was wondering if anyone could point me in the right direction for what exactly I'm looking for. I have binned multiple Boolean observations (1 or 0). The reason I made them Boolean is because it's ...
0
votes
0answers
289 views

How to find statistical significance between two years of data

I was asked to determine if the number of HAI infections in 2011 in statistically significant from the number of HAI infection in 2012. I performed a paired T-test using the actually numbers, but am ...
2
votes
0answers
86 views

How to test if two coins have different biases?

probably a noob question because I'm a noob. I have two coins, A and B. For each coin I have a sample of the results I get by tossing it. My null hypothesis is that A has equal or higher probability ...
2
votes
2answers
1k views

Manually Calculating P value from t-value in t-test

I have a sample dataset with 31 values. I ran a two-tailed t-test using R to test if the true mean is equal to 10: t.test(x=data, mu=10, conf.level=0.95) Output: ...
0
votes
0answers
132 views

fisher's test vs pairwise fisher's test

I have a fold enrichment table for different days as given below. I need to find the significance of every fold enrichment. ...
-1
votes
1answer
145 views

How to provide statistical evidence that an experimental method is providing higher mean percentages?

I performed an experiment using two different methods. The data are percentages and higher percentages indicate a better method. The results were as follows: ...
0
votes
1answer
219 views

P values and t statistics?

I performed an experiment using two different methods on same object, the results of the experiments were Method 1 70,72,70,75 and 77% Method 2 80,81,67,75 and 82% On basis on average I claimed ...
0
votes
0answers
90 views

How to identify significant changes in metrics

We gather various metrics for a system many times per hour. Then for each hour, compute and store the mean, variance, and sample size of each metric. Later to evaluate load and system changes, on an ...
2
votes
0answers
186 views

Statistical test for two datasets with different variance and mean

I'm doing a research based on cross-ISP traffic of P2P applications. I have collected a significant amount of real world test data with the optimization applied and without it. However the dataset ...
2
votes
1answer
108 views

Testing hierarchical data

It is easy to compare paired data, using paired t-test. But suppose this pairing is hierarchical. Below is an example: The stem cells of two mouse strains S1 and S2 are cultured in two different ...
5
votes
3answers
164 views

Contradiction between significant effect in multiple regression, but non-significant t-test on its own

I ran a multiple regression using 10 independent variables and the single dependent variable (consumer complaining behaviour). One of those independent variables was gender. The $R^2$ for the model ...
0
votes
0answers
103 views

t-stats in OLS regression

Very basic question: suppose I run an OLS regression $Y$ ~ $X_1 + X_2$ and obtain t-stats of $t_1$, $t_2$ respectively. If $t_1$ >> $t_2$ (such as an order of magnitude larger), is it safe to conclude ...
2
votes
2answers
87 views

Which test to use for comparing changes in computer source code?

I am not sure which test I have to use to analyze my data. The situation is as follows: I have two computer programs which are doing exactly the same. Both share a large amount of source code, but ...
0
votes
1answer
45 views

Appropriate test for difference in trials with varying calibration

Let's say I collect data from an experiment where for each measurement device, 5 trials are conducted at different temperatures. Each device is calibrated separately, and the 5 trial temperatures are ...
0
votes
2answers
188 views

Significant Statistical difference One-sample t-test

I am trying to test if one algorithm is statistically significantly faster than another using the One-sample t-test These are the results I have and I am trying to prove Algo2 is significantly faster ...
2
votes
2answers
1k views

Comparing two classifier accuracy results for statistical significance with t-test

I want to compare the accuracy of two classifiers for statistical significance. Both classifiers are run on the same data set. This leads me to believe I should be using a one sample t-test from what ...
6
votes
1answer
366 views

Tracking down the assumptions made by SciPy's ttest_ind() function

I'm trying to write my own Python code to compute t-statistics and p-values for one and two tailed independent t tests. I can use the normal approximation, but for the moment I am trying to just use ...
3
votes
4answers
325 views

Comparing smoke and hormones level in two groups of people. Which test?

I'm totally not a pro in statistics so i hope the question is not too simple! I have two groups of patients. One group are smokers, the second are non-smokers. I want to compare the level of some ...
8
votes
2answers
2k views

What sense does it make to compare p-values to each other?

I have two populations, each containing 1000 samples. For each sample I have two properties A & B. I have used a t-test separately for A & B: both were found significantly different within the ...
1
vote
1answer
261 views

How can I compare whether classifiers outcome is significantly different?

How can I compare whether classifiers outcome is significantly different? I have a relatively small data set which I'm trying to classify. My training set consists of 24 items come from two different ...
4
votes
2answers
290 views

Is significance more important than absolute value?

I have three groups and I'm comparing two graphs in each group using the t-test: the absolute difference between two graphs in the first group is 34 with a p=0.024. (*) The absolute difference in the ...
4
votes
4answers
9k views

How to calculate number of participants required to compare mean scores on a questionnaire between two groups?

I have to create a questionnaire that compares the sportiness of Facebook users vs. non-Facebook users. I need to calculate the number of participants that are required so that the result is ...
2
votes
1answer
289 views

Significance test for differences in baseline and remeasurement rates between two groups

I need help in knowing which test to perform with this analysis. I have two groups (lets say group A and B). A study was performed with initial rates of heart failures in group A and B (i.e, ...
3
votes
1answer
247 views

Can I split a series of observations of a variable over time into two groups instead of working with time series?

First of all, I’m new to statistics and this is the first time I am trying to apply it to a real world problem. I am doing analysis of a series of observations of a variable over all weeks of a year. ...