# Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

9,424 questions
Filter by
Sorted by
Tagged with
14 views

### Would the Mantel-Haenszel test for linear trend be appropriate for this data/hypothesis?

I am testing whether or not there is an association between the parents education level and the pupils grades in the oral exam in social studies in the last year of danish primary school from the ...
1 vote
23 views

### Significance test for comparing different 10-fold cross-validated Machine Learning Regressions

Is there a recommended significance test for comparing different 10-fold cross validated regressions? For instance, I want to compare the performance of LASSO against Random Forest for my dataset. ...
1 vote
13 views

• 177
15 views

### Fisher's exact test when resampling from the population

I am using different ML classifiers for making predictions related to a binary classification. I would like to compare two groups. Let's call them A and B. The already trained and tested classifier ...
32 views

### Problem formulating a hypothesis test for the mean

Suppose that I would like to support the following hypothesis: h = The average weight of people in New York is 70kg. If I take h to be the null hypothesis then I can only reject it (I assume that ...
• 883
15 views

### How to conduct t-test for comparing the accuracy of two binary classifiers? [closed]

I am using two binary classifiers that predict the accuracy of samples over a dataset. I need to check if the difference in the mean accuracy between the two models is statistically significant. ...
8 views

### How do I perform a content analysis in SPSS? [closed]

I am struggling to find the right statistical test for a content analysis. I have categorized and coded the different variables (e.g. text length) and generated the Excel file for the analysis. The ...
274 views

### How to report a P-value?

I am writing the results of my thesis, and I have a few questions on how to report P-values. When is it appropriate to give the exact P-value, instead of writing e.g. P<0.05 (also in case of non-...
• 91
1 vote
42 views
+50

### Why does the fact that I estimated my parameters change the way I should use chi−sqaured test?

Suppose I have a data sample and I want to test whether normal distribution with mean $0$ and variance $1$ fits this data sample. If I understand the chi-squared test correctly, I think that I should ...
26 views

### How to test the statistical significance of several clustering algorithms? [closed]

I have computed ARI, NMI and Silhouette scores of several clustering algorithms on single cell RNA sequence data. These metrics are computed three times for each algorithm and for each dataset. Now, I ...
32 views

### Comparing scientific studies (p-values, q-values)

I am comparing two studies. Each study consists of an affected (cancer) and control groups of patients, for which a number of biomarkers (gut microbial species) were studied. For each biomarker ...
• 1,578
23 views

### how to measure the effect of a recurring event?

I have multiple cities with data on theatre visitors and an event as the mentioning of the theatre in the local news. I want to estimate whether the event of a mentioning lead to more visitors for the ...
24 views

### Testing sequence of ones and zeros for randomness

I am given a sequence of $40$ ones and zeros and I have to test the null hypothesis that ${40 \choose n_1}$ sequences are all equally probable ($n_1$ being the number of ones). To do so, I have to use ...
• 133
1 vote
21 views

### How do I test that two samples don’t differ significantly by their demographics?

I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics ...
• 125
25 views

### Statistics And Probability [closed]

The Audit Team of ABC University claims that the mean monthly salary of the teachers is Php18,000.00 with a standard deviation of Php4,500.00. A researcher takes a random sample of 20 teachers and ...
1 vote
8 views

### Do differences between ordinations prove a relationship between variables?

I have a large dataset that contains a variety of environmental characteristics (air temperature, tree cover, insect counts) as well as bird abundance data for each point. I was talking with my friend ...
11 views

### How to correctly estimate the ratio different of lower grain unit metric in Cluster randomized experiment?

I work on Education tech products that teachers/students would use in their learning journey. When we run experiment to test hypothesis of a feature, we need to do cluster randomization (cluster = ...
21 views

### Using the same subjects in different measures

I'm working on an experiment where I collect 3 categories of reaction time data form subjects who are video game players, distinguished by the game that they play (eg. RTS, FPS, MOBA). I wish to ...
10 views

### Simplified DID: How to run significance test

If we know that Variable C = Variable A - Variable B, and Variable A is the difference in two groups (binary data) during time period X. It is statistically significant. Variable B is the difference ...
35 views

### How to calculate the statistical significance of the difference in a categorical value of two populations?

There has recently been a statistical report regarding whether Cypriots are ready for a female president. There was a difference in the opinions of men and women and I'm wondering if it is ...
• 101
7 views

### How can I efficiently test whether a subsets of nodes in a DAG are "lined up" more often than expected by random chance?

I have a directed acyclic graph with N nodes, each of which is assigned to one of K groups, with K < N. My hypothesis is that nodes in the same group tend to "line up" along a linear path....
11 views

### How do I interpret t-ratio of interaction variable?

I need to answer this question for my Econometrics HW: Does the gender gap differ significantly in urban versus non-urban areas? Test at the 5% level. If I am not wrong I need to use the t-test for ...
26 views

### Testing the collection of Bernoulli variables

Suppose that I am throwing a triple dice (i.e, three dices simultaneously) $n$ times. Then number of times that the sum of these three dices $k=3,\ldots,18$ appears is my sample $X_1,\ldots,X_n$ is ...
• 133
16 views

### Testing equality of coefficients from two different samples

I have the regression statistics for the same regression run on two different samples, and am asked to explain whether it is possible to test for equality of the coefficents, $\beta_1$and $\beta_2$ ...
35 views

### Delta Method to calculate the standard error of ratio in an AB testing context

In AB testing context, if we have a control group and test group (2 groups), and I'd like to calculate the relative difference (Mean test/ Mean control -1) and the confidence interval of this ratio ...
46 views

### Does the likelihood ratio test violate the likelihood principle?

I've been going over Berger's famous example of negative binomial vs binomial sampling leading to two different p-values conditional on the same observed data. To summarize, suppose we observe 9 tails ...
• 51
21 views

### Model that tests if categorical variables give higher results than others [closed]

Consider a set of categorical variables (values are 0 or 1) which are used to predict revenue range values. For instance the observation $(x_1, x_2, x_3, x_4, ..., x_{20}) = (0,1,0,1, ..., 0)$ might ...
• 113
27 views

### Validating random variable generation from inverse transform sampling

I'm building a simulator and I have to implement some probability distributions. What is the best (formal) way of validating this implementation? I took a look at KS-tests but it seems to me they are ...
34 views

### two sample t-test for non-normal population

I assume this question has been beaten to death and thus I am just looking for a reference which goes through the details. Assuming all populations we deal with have finite means and variances (even ...
• 133
19 views

### T-testing on sparse data

Problem: I have two stochastic processes, $S_1$ and $S_2$, that frequently are zero, but occasionally have positive values with unknown probabilities $q_1$ and $q_2$. e.g.  S_1 = \{0,0,0,0,0,21,0,0,...
2k views

### If p-values are random, why not decide using the test statistic instead?

I am currently reading the paper Duncan J Murdoch, Yu-Ling Tsai & James Adcock (2008) P-Values are Random Variables, The American Statistician, 62:3, 242-245, DOI: 10.1198/000313008X332421 In ...
• 2,916
1 vote
22 views

### Uniformly most powerful test

Suppose we have Xi~Exp(λ), and we want to construct a most powerful test for H0 : λ = λ0, H1 : λ = λ1 I then proceed to use the Neyman Pearson lemma : reject H0 when the likelihood ratio L(λ1;X)/L(...
30 views

### With Stationarity How can ARMA Modelling have any Validity?

I have recently been thrown into the deep end with time-series econometrics. The first thing I have learned is that in order to avoid the spurious correlation trap, I need to ensure that all the ...
1 vote
33 views

### Is this interpretation of the relationship between p-values and type I error rates correct?

I am trying to check my understanding of the relationship between p-values and type I error rates. I am going to use the following example to do so, so I would appreciate any feedback on it. Suppose ...
• 2,916
28 views

### Z - Score between average scores

I have a test that gives students a score between 0 and 10. Students can be black, white, asian or hispanic. I want to compare the average score of each race against the overall average, and I want to ...
• 21
8 views

### Hypothesis testing on a moving-average model on unevenly-spaced time series

I have some irregular time series $X$, where $X_t$ are identically distributed, and bounded between 0 and 1. I perform a moving average $Y_t = \sum_{t'} \frac{w(t-t')}{\sum_k w(t-k)}X_{t'}$. The ...
1 vote
26 views

### What is the advantage of a scoring scale with many levels?

In my research I have different ordinal variables with 3 levels of scores. Each score represents the severity of a condition. In other studies I see that the same variables are scored on a scale with ...
• 91