Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Would the Mantel-Haenszel test for linear trend be appropriate for this data/hypothesis?

I am testing whether or not there is an association between the parents education level and the pupils grades in the oral exam in social studies in the last year of danish primary school from the ...
user avatar
  • 1
1 vote
0 answers
23 views

Significance test for comparing different 10-fold cross-validated Machine Learning Regressions

Is there a recommended significance test for comparing different 10-fold cross validated regressions? For instance, I want to compare the performance of LASSO against Random Forest for my dataset. ...
user avatar
1 vote
0 answers
13 views

Can the F test be generalized for a null hyp where variances are unequal?

Sorry for the question, maybe this appears in a book a little more advanced than the ones I have seen. Suppose we have two samples from normal distributions with sample variances $S_{n,1}^2$ and $S_{m,...
user avatar
  • 179
0 votes
0 answers
10 views

Suggestions for an independence test in a complex design

In an experiment I surveyed the effect of two treatments (pre & post) in different species. After every experimental run I tested whether the measured average effect was greater, smaller or not ...
user avatar
  • 123
0 votes
0 answers
5 views

Calculate sample size for multiple treatments vs control, given the proportion mix of treatments

I want to know what my sample sizes should be for each of my treatments and control, given a proportion mix between the groups. For example, let’s say I have 3 treatments (A, B, C) and 1 control (D). ...
user avatar
2 votes
0 answers
24 views

Confidence Intervals and Hyptothesis testing

Iam currently reading up on basic statistics and Iam somewhat confused about the computation/inference of confidence intervals and hypothesis tests. As far as I have understood there are several ...
user avatar
  • 21
1 vote
2 answers
26 views

Proof of statistical assumptions behind the methods or tests

When reading statistics literature, they provide the assumptions of when specific tests/methodologies can be used. Take factor analysis as an example; you may find this inside a paper: "Generally ...
user avatar
  • 135
0 votes
0 answers
6 views

Testing Dependant Contingency Tables

Suppose we have two treatent groups $A$ and $B$ (of $n$ individuals each), we wish to determine if $A$ is statistically more likely to respond to treatment compared to $B$. We run the simplest ...
user avatar
0 votes
0 answers
23 views

How to interpret high P-value and low Mcnemar's Test P-value in Caret R?

I am using the caret R package to evaluate machine learning models. This gives me a lot of information about different metrics. I am confused about my result why ...
user avatar
  • 101
0 votes
0 answers
17 views

Modified Structural Break Test

Consider a simple AR(1) model $$y_t=c+\lambda y_{t-1}+\epsilon_t$$ For example using the so called Andrews supremum statistic, I can test for a single structural break in $\lambda$ at an unspecified ...
user avatar
  • 218
0 votes
1 answer
15 views

How to compare variables with data from multiple experiments?

I have an experiment in which i compare the amount of attached bacteria in three different conditions. I have repeated this experimented three times, so now i have three datasets with the same data (...
user avatar
1 vote
1 answer
64 views

Consecutive coin flips, what is the appropriate statistical test for this word problem? [closed]

I was listening to a podcast by NDGT (Neil deGrasse Tyson, a prominent scientist) and he posed a simple thought experiment to illustrate the susceptibilities to cognitive bias. What I've come here to ...
user avatar
1 vote
0 answers
23 views

How to find the distribution that minimize the KL distance(divergence) to a known distribution?

Everyone! I'm reading the paper On test marginal versus conditional and I'm a little confused one place in page 9, when the author use "maximum likelihood projection" to get the covariance ...
user avatar
1 vote
0 answers
19 views

How to test a product of a OLS estimators?

I estimated a vector $\beta $ using OLS $\hat \beta =(\hat e_1 \hat e_2 \hat e_3)$ and I have the covariance matrice $\hat v(\hat \beta )$. How can I test this null hypothesis $H_0 : \hat e_1 \hat ...
user avatar
  • 177
0 votes
0 answers
15 views

Fisher's exact test when resampling from the population

I am using different ML classifiers for making predictions related to a binary classification. I would like to compare two groups. Let's call them A and B. The already trained and tested classifier ...
user avatar
0 votes
1 answer
32 views

Problem formulating a hypothesis test for the mean

Suppose that I would like to support the following hypothesis: h = The average weight of people in New York is 70kg. If I take h to be the null hypothesis then I can only reject it (I assume that ...
user avatar
  • 883
0 votes
0 answers
15 views

How to conduct t-test for comparing the accuracy of two binary classifiers? [closed]

I am using two binary classifiers that predict the accuracy of samples over a dataset. I need to check if the difference in the mean accuracy between the two models is statistically significant. ...
user avatar
0 votes
0 answers
8 views

How do I perform a content analysis in SPSS? [closed]

I am struggling to find the right statistical test for a content analysis. I have categorized and coded the different variables (e.g. text length) and generated the Excel file for the analysis. The ...
user avatar
  • 1
6 votes
2 answers
274 views

How to report a P-value?

I am writing the results of my thesis, and I have a few questions on how to report P-values. When is it appropriate to give the exact P-value, instead of writing e.g. P<0.05 (also in case of non-...
user avatar
  • 91
1 vote
0 answers
42 views
+50

Why does the fact that I estimated my parameters change the way I should use chi−sqaured test?

Suppose I have a data sample and I want to test whether normal distribution with mean $0$ and variance $1$ fits this data sample. If I understand the chi-squared test correctly, I think that I should ...
user avatar
0 votes
0 answers
26 views

How to test the statistical significance of several clustering algorithms? [closed]

I have computed ARI, NMI and Silhouette scores of several clustering algorithms on single cell RNA sequence data. These metrics are computed three times for each algorithm and for each dataset. Now, I ...
user avatar
0 votes
1 answer
32 views

Comparing scientific studies (p-values, q-values)

I am comparing two studies. Each study consists of an affected (cancer) and control groups of patients, for which a number of biomarkers (gut microbial species) were studied. For each biomarker ...
user avatar
  • 1,578
0 votes
0 answers
23 views

how to measure the effect of a recurring event?

I have multiple cities with data on theatre visitors and an event as the mentioning of the theatre in the local news. I want to estimate whether the event of a mentioning lead to more visitors for the ...
user avatar
2 votes
0 answers
24 views

Testing sequence of ones and zeros for randomness

I am given a sequence of $40$ ones and zeros and I have to test the null hypothesis that ${40 \choose n_1}$ sequences are all equally probable ($n_1$ being the number of ones). To do so, I have to use ...
user avatar
1 vote
1 answer
21 views

How do I test that two samples don’t differ significantly by their demographics?

I am conducting a study where I am randomly assigning people to different conditions. Because I am recruiting from a representative sample in the US, there will be some variability in the demographics ...
user avatar
  • 125
0 votes
0 answers
25 views

Statistics And Probability [closed]

The Audit Team of ABC University claims that the mean monthly salary of the teachers is Php18,000.00 with a standard deviation of Php4,500.00. A researcher takes a random sample of 20 teachers and ...
user avatar
1 vote
1 answer
8 views

Do differences between ordinations prove a relationship between variables?

I have a large dataset that contains a variety of environmental characteristics (air temperature, tree cover, insect counts) as well as bird abundance data for each point. I was talking with my friend ...
user avatar
0 votes
0 answers
11 views

How to correctly estimate the ratio different of lower grain unit metric in Cluster randomized experiment?

I work on Education tech products that teachers/students would use in their learning journey. When we run experiment to test hypothesis of a feature, we need to do cluster randomization (cluster = ...
user avatar
0 votes
0 answers
21 views

Using the same subjects in different measures

I'm working on an experiment where I collect 3 categories of reaction time data form subjects who are video game players, distinguished by the game that they play (eg. RTS, FPS, MOBA). I wish to ...
user avatar
0 votes
0 answers
10 views

Simplified DID: How to run significance test

If we know that Variable C = Variable A - Variable B, and Variable A is the difference in two groups (binary data) during time period X. It is statistically significant. Variable B is the difference ...
user avatar
0 votes
1 answer
35 views

How to calculate the statistical significance of the difference in a categorical value of two populations?

There has recently been a statistical report regarding whether Cypriots are ready for a female president. There was a difference in the opinions of men and women and I'm wondering if it is ...
user avatar
  • 101
0 votes
0 answers
7 views

How can I efficiently test whether a subsets of nodes in a DAG are "lined up" more often than expected by random chance?

I have a directed acyclic graph with N nodes, each of which is assigned to one of K groups, with K < N. My hypothesis is that nodes in the same group tend to "line up" along a linear path....
user avatar
0 votes
0 answers
11 views

How do I interpret t-ratio of interaction variable?

I need to answer this question for my Econometrics HW: Does the gender gap differ significantly in urban versus non-urban areas? Test at the 5% level. If I am not wrong I need to use the t-test for ...
user avatar
0 votes
1 answer
26 views

Testing the collection of Bernoulli variables

Suppose that I am throwing a triple dice (i.e, three dices simultaneously) $n$ times. Then number of times that the sum of these three dices $k=3,\ldots,18$ appears is my sample $X_1,\ldots,X_n$ is ...
user avatar
0 votes
0 answers
16 views

Testing equality of coefficients from two different samples

I have the regression statistics for the same regression run on two different samples, and am asked to explain whether it is possible to test for equality of the coefficents, $\beta_1$and $\beta_2$ ...
user avatar
0 votes
2 answers
35 views

Delta Method to calculate the standard error of ratio in an AB testing context

In AB testing context, if we have a control group and test group (2 groups), and I'd like to calculate the relative difference (Mean test/ Mean control -1) and the confidence interval of this ratio ...
user avatar
3 votes
0 answers
46 views

Does the likelihood ratio test violate the likelihood principle?

I've been going over Berger's famous example of negative binomial vs binomial sampling leading to two different p-values conditional on the same observed data. To summarize, suppose we observe 9 tails ...
user avatar
  • 51
0 votes
0 answers
21 views

Model that tests if categorical variables give higher results than others [closed]

Consider a set of categorical variables (values are 0 or 1) which are used to predict revenue range values. For instance the observation $(x_1, x_2, x_3, x_4, ..., x_{20}) = (0,1,0,1, ..., 0)$ might ...
user avatar
  • 113
2 votes
1 answer
27 views

Validating random variable generation from inverse transform sampling

I'm building a simulator and I have to implement some probability distributions. What is the best (formal) way of validating this implementation? I took a look at KS-tests but it seems to me they are ...
user avatar
3 votes
1 answer
34 views

two sample t-test for non-normal population

I assume this question has been beaten to death and thus I am just looking for a reference which goes through the details. Assuming all populations we deal with have finite means and variances (even ...
user avatar
0 votes
0 answers
19 views

T-testing on sparse data

Problem: I have two stochastic processes, $S_1$ and $S_2$, that frequently are zero, but occasionally have positive values with unknown probabilities $q_1$ and $q_2$. e.g. $$ S_1 = \{0,0,0,0,0,21,0,0,...
user avatar
  • 1
13 votes
7 answers
2k views

If p-values are random, why not decide using the test statistic instead?

I am currently reading the paper Duncan J Murdoch, Yu-Ling Tsai & James Adcock (2008) P-Values are Random Variables, The American Statistician, 62:3, 242-245, DOI: 10.1198/000313008X332421 In ...
user avatar
  • 2,916
1 vote
0 answers
22 views

Uniformly most powerful test

Suppose we have Xi~Exp(λ), and we want to construct a most powerful test for H0 : λ = λ0, H1 : λ = λ1 I then proceed to use the Neyman Pearson lemma : reject H0 when the likelihood ratio L(λ1;X)/L(...
user avatar
0 votes
0 answers
30 views

With Stationarity How can ARMA Modelling have any Validity?

I have recently been thrown into the deep end with time-series econometrics. The first thing I have learned is that in order to avoid the spurious correlation trap, I need to ensure that all the ...
user avatar
1 vote
1 answer
33 views

Is this interpretation of the relationship between p-values and type I error rates correct?

I am trying to check my understanding of the relationship between p-values and type I error rates. I am going to use the following example to do so, so I would appreciate any feedback on it. Suppose ...
user avatar
  • 2,916
0 votes
1 answer
28 views

Z - Score between average scores

I have a test that gives students a score between 0 and 10. Students can be black, white, asian or hispanic. I want to compare the average score of each race against the overall average, and I want to ...
user avatar
  • 21
0 votes
0 answers
8 views

Hypothesis testing on a moving-average model on unevenly-spaced time series

I have some irregular time series $X$, where $X_t$ are identically distributed, and bounded between 0 and 1. I perform a moving average $Y_t = \sum_{t'} \frac{w(t-t')}{\sum_k w(t-k)}X_{t'}$. The ...
user avatar
1 vote
2 answers
26 views

What is the advantage of a scoring scale with many levels?

In my research I have different ordinal variables with 3 levels of scores. Each score represents the severity of a condition. In other studies I see that the same variables are scored on a scale with ...
user avatar
  • 91
2 votes
1 answer
22 views

Logistic regression - One model trained on different groups

I have a logistic regression model that trains a set of binary independent variables (X) on a binary response variable (Y). The data was gathered from different individuals for who also e.g. socio-...
user avatar
  • 121
0 votes
1 answer
33 views

References: t-test and Chi-squared test can be conducted with unequal sample sizes

I have two independent groups: sample size of group 1 is 24000, and sample size of group 2 is 246. With the drastically different sample sizes in the two group, I need to use the Student’s t-test or ...
user avatar
  • 1

1
2 3 4 5
189