Questions tagged [hypothesis-testing]

Hypothesis testing assesses whether data are inconsistent with a given hypothesis rather than being an effect of random fluctuations.

Filter by
Sorted by
Tagged with
0
votes
0answers
6 views

K-sample test for equality of distribution in time series

I have a set of time series, with potentially very weird behavior (i.e. I'd prefer to make as few observations about them as possible, apart from stationarity -- not a full ARIMA model). Luckily, they'...
1
vote
0answers
37 views

What role does t-statistic play in hypothesis testing?

I'm wondering what role t-statistic plays in hypothesis testing and the interpretation of the result. In a two-sample two-tailed t-test where we have $H_0$: $\mu_A=\mu_B$ and $H_1$: $\mu_A \neq \mu_B$,...
0
votes
0answers
10 views

Generalized likelihood ratio test for known means and unknown variance

Consider a sequence, $X_1, X_2, \dots, X_n$, of independent random variables. I have two hypotheses: $H_0 : X_k \sim \mathcal{N}(\mu_0, \sigma^2), k=1,2,\dots,n$ $H_1 : X_k \sim \mathcal{N}(\mu_1, \...
1
vote
0answers
17 views

Hypothesis testing with Gaussian process regression?

Say I have two sets of observations (data points), $\mathbf U_1$ and $\mathbf U_2$. Each of these datasets consists of a collection of (not necessarily the same number of) points, e.g. $\mathbf U_1=\{\...
0
votes
0answers
11 views

How to find $\gamma_i$ and $c_i$ for an UMPU Test for exponential family, hypothesis over an interval

From my textbook: For a one-parametric exponential family $$p_\vartheta(x)=c(\vartheta)h(x)\exp(Q(\vartheta)T(x)),$$ where $Q$ is increasing, an UMPU test of the hypothesis $$H: \vartheta \in [\...
0
votes
1answer
37 views

Reasons for deploying poor models - Appropriate to do so? [closed]

Am working on a simple logistic regression with 1000 records and 28 features. My business users suggest that they want to first see what the AI can do by itself based on our data as it is. Meaning, ...
0
votes
0answers
23 views

Cant reject null hypotheses

I am running a ttest exercise, in which I have to compare the mean of two populations. Specifically,exercise is: ttest age, by(labforce) ttest age, by(labforce) unequal In both cases, I get p-values ...
1
vote
1answer
23 views

Choose statistical test method: comparing the frequency of a disease (only interested the disease post-procedure) in 2 groups of patients

I am working on 2 groups of patients, group A and group B. The patients in both groups have received a same procedure. We have observed the patients before and after the procedure. Suppose there is a ...
0
votes
0answers
35 views

Is Bayesian approach best for this problem and if so, how do I calculate likelihood?

I have 5 years of solar irradiance (power) data measured at a particular location. I would like to understand whether the measured data, which very often is not accurate and has integrity issues, is ...
0
votes
0answers
13 views

can the chi-squared test tell me if my data can form meaningful clusters?

I have some data that consists of equal length vectors and a label of 0 or 1. The vectors contain counts of certain events. (For example, $x=[2,3,1...]$ means event 1 happened twice, event 2 happened ...
0
votes
1answer
49 views

Extremely low p-value, even though null-hypothesis should NOT be rejected

I'm doing hypothesis testing to check whether one variable affects another, by looking at the slope of the trendlines, on 268 datasets. From the analysis of each dataset (which contains many samples), ...
0
votes
0answers
10 views

Can the probabilities be trusted from xgboost on the test data set if used for early stopping round?

I am using xgboost to predict on a binary data set and using the dtest as the eval for early stopping. The python code extract below: ...
1
vote
1answer
23 views

Testing a linear restriction on coefficients of a logistic regression (in SPSS)

I have a logistic regression model $$ P(Y=1|X_1,X_2,X_3)=\frac{1}{1+\exp[-(\beta_0+\beta_1 X_1+\beta_2 X_2+\beta_3 X_3)]}. $$ I would like to test a null hypothesis $H_0\colon \beta_2=c\beta_1$. ...
0
votes
0answers
15 views

Pre- and post significance analysis with random samples (independant / unpaired) for discrete (binary) metric

I am conducting an analysis where I want to check if there was a significant difference in CTR (discrete metric) when an element is moved higher in the page. The samples will be different but I have ...
2
votes
0answers
11 views

Multi-step hypothesis testing

I have an estimator $\hat{R}$ that I am computing for $N$ groups. For each group I am performing a hypothesis test with significance ($\alpha$) : $H_0$: $\mathbb{E}[\hat{R}]=r$, $H_1$: $\mathbb{E}[\...
0
votes
0answers
19 views

How to simulate type I error for random-effects model?

Building upon this post how to simulate type I error and type II error. I would like to simulate type I error for a random-effects model I generated. The statistic of interest is standard deviations ...
0
votes
0answers
9 views

How do you work out the optical time window search for a particle decaying, when there is a constant level of background radiation? [closed]

Let's suppose we are monitoring a particle that decays in exponential fashion, with a mean lifetime of $\tau$. This particle, when it decays produces by-products. But, in our experiment, we also have ...
2
votes
1answer
226 views

Inconsistent result in proportion test in R

I have some problems understanding the results of this online experiment. I sent my website users 2 types of messages that can result in a subscription or not. The users can belong to two different ...
0
votes
0answers
16 views

Ur.df interpretation

I have been working on and reading about the ur.df test in R, so I know what the tests mean, what the hypotheses are and how to read the critical values. (For ...
0
votes
0answers
27 views

P-value vs. CDF of t statistics

All, P-value could be defined simply as the p-value is the probability of obtaining test results at least as extreme as the results actually observed, under the assumption that the null hypothesis is ...
0
votes
0answers
13 views

How well does OOB estimate the generalization error?

I frequently see people evaluate the models using the OOB, but well does OOB estimates the generalisation error? Is it an unbiased estimator for the generalisation error ? How large is the margin of ...
0
votes
0answers
23 views

Designing an experiment to test before vs. after unsuccessful payment percentage

I'm working at an online payment processor, which is used by multiple vendors (kind of like paypal). The experiment that I'm working on involves just one vendor. I have 100k data points from that ...
2
votes
1answer
16 views

Using outcome label to create feature qualifies as data leakage?

I have a small dataset of 1000 rows and 10 features, working on binary classification to predict whether the supplier's likelihood of meeting the target (coded as ...
0
votes
2answers
96 views

Why Student's t-test doesn't require normality of population?

this PSU master's degree course says t-test can be used if either of the conditions is true: population is normal. sample size n>30. However, I read that t test is applicable when $X_{i}\stackrel{...
0
votes
0answers
63 views

Does this Python function correctly implement a beta-binomial test?

Note: I, originally, asked this question on codereview.stackexchange.com (see here), but I was advised to ask the question here to find out whether the statistics/math used in the function is correct (...
2
votes
2answers
95 views

How to fix hypothesis testing: MBF versus "directional correctness"

A common misinterpretation of a p-value is that it represents the probability of a false positive in the context of hypothesis testing. Here a "positive" means rejecting the null. There are ...
0
votes
0answers
93 views
+50

Can you do hypothesis testing when instead of a 'sample' size you have 'actual' size? Alternatively, how would you use statistics to compare means?

From chess se: What is white's increased advantage in chess90 as compared to chess870? (Chess960 can be split into 2 subsets, chess90 and chess870) Intro: It is known in standard chess that white has ...
0
votes
1answer
11 views

Usefulness of identifier variables in ML model building

I have a dataset with 1000 rows and I have an identifier variable called person_id. May I check why is it told that we should remove identifier variables from ML ...
0
votes
0answers
15 views

How plausible are RCTs/AB tests?

Suppose that I wish to quantify the efficacy of a drug compared to a control. I proceed to construct my RCT (using sample sizes which guarantees a particular certainty and significance level). I also ...
0
votes
1answer
37 views

simple vs simple hypothesis, finding sample size given type 1 and 2 error probabilities

The problem is: It is decided to investigate the hypotheses H0 : μ = €678 and H1 : μ = €720. Assuming that the prices are normally distributed with standard deviation €100, what sample size must be ...
0
votes
1answer
20 views

Hypothesis testing of multiple genes for multiple samples

If I have an x number of genes for n number of samples with values representing gene expression values for cancer. And I have the same data for a healthy individual and I want to check if the change ...
3
votes
3answers
71 views

Does Kruskal-Wallis test have a non-parametric analogue that isn't sensitive to groups having different variability?

I need a non-parametric test for my project (because I can't be sure that the distribution would always be normal), so I decided to use Kruskal-Wallis test. But the problem is that Kruskal-Wallis test ...
1
vote
1answer
15 views

How to interpret low but statistically significant correlation

I am doing an analysis on whether older planes would lead to higher delays. However, when I run correlation tests between delays and age of the planes, it shows that the sample correlation is 0.005 ...
1
vote
0answers
11 views

Is HEGY or CH test better in a small sample (40 monthly observations)?

I have a time series of monthly data that is 40 observations ($3\frac{1}{3}$ seasonal cycles) long. HEGY test and CH test give contradicting results w.r.t. presence of a seasonal unit root. HEGY test ...
0
votes
0answers
18 views

How to answer the question "Does promotion affect sales?" for this time-series dataset?

I'm doing a project on this time-series analysis that can be obtained from Kaggle. The train.csv dataset has information about sales across different stores, products (or families) from 2013 to 2017. ...
0
votes
0answers
36 views

Statistical Test for pairwise data

I have 3 disjoint groups $A,B \text{ and }C$. Each group contains $N_g,g\in\{A,B,C\}$ models and each model is associated with data (training and testing) which is denoted by $(M_i^g,D_i^g),i\in\{1,\...
0
votes
0answers
33 views

question on hypothesis testing

Independent random samples of size $N_1 $ = 16 and $N_2$ = 25 from the populations with standard deviation $\sigma_1$ = 4.8 has the means $\bar{X_1}$ = 18.2. if the population is not normal and ...
2
votes
2answers
32 views

What's the drawbacks of choosing a high statistical power and low significance level?

I want to perform an AB test for two products (a control and a variant). I want to detect a 20% uplift with a very high certainty i.e., statistical power and a very low probability of a type I error (...
0
votes
1answer
34 views

Why type I error rate is rejection area in hypothesis testing?

In hypothesis testing, we set up a rejection area for rejecting $H_0$ in favor of $H_1$ with $\alpha$. I don't understand why type I error (rejecting $H_0$, when $H_0$ is actually true) is the area ...
0
votes
0answers
17 views

Best way to create a supersaturated sesign

I have full data for $2^{12}$(All the 4096 runs) full factorial design. I know the exact model, I created all the data. I am trying to create a D-Optimal Supersaturated design for an application. I ...
1
vote
1answer
37 views

Statistical test: which party do most of the voters belong to?

I am looking for a statistical test able to explain the results I have found. I have the following fields (political categories): Right Center Left I would like to ...
-1
votes
0answers
55 views

Help with exercise on PARAMETRIC TESTS

I'm stuck on an exercise and I really don't know where to start from. I don't have a strong background in statistics and the professor didn't provide any examples on such exercises so I'm completely ...
0
votes
0answers
32 views

Paired Data Confidence Intervals For Accuracy

I had an individual take a test with 500 binary questions before and after an intervention. Let's say they scored an 80% initially and a 90% on the second pass through. I can measure the absolute ...
1
vote
0answers
57 views

Non-parametric test for two sample quantiles

I would like to know if there is a non-parametric test on quantiles. EDIT: My focus is on continuous distributions (e.g. Weibull distribution). In a general setting we define quantiles as $Q_X(p) = ...
0
votes
0answers
18 views

Ordering drug effectiveness from best to worst

Problem The (mean) effectiveness of a drug $d$ on trial $i$ is given by \begin{align} X_{di}:=\widehat{\mu}_{di} \sim \mathcal{N}(\mu_{di}, \sigma_{di}^2), \end{align} with known variances. Hence, for ...
1
vote
1answer
37 views

Chess opening move selection [duplicate]

So I’m playing a chess game and using 365chess.com to help with my moves (this is totally legal btw as long as engine analysis is turned off) it says move A leads to a win with white* 36% of the time ...
0
votes
0answers
21 views

How to Compare Data Distribution of 2 large datasets?

I want to compare two super large datasets (petabytes). I have tried to use scipy ks_2samp kolmogoroc smirnov 2 sample test, but I get all the pvalues to be 0. I tried to downsample by random sampling ...
1
vote
0answers
30 views

Using same control variables as in propensity score matching

I matched my treatment and control group in STATA, using propensity score matching. I also made sure that each treatment/control pair share the same industry and year ( it's a panel data). If I'm now ...
1
vote
0answers
15 views

Robust way to compute significance between counts without standard deviation

I'm a little rusty, but without information on standard deviation, and simply using the counts themselves, one can compute some basic significance which I know as a rule of thumb as simply the ...
0
votes
0answers
5 views

Test for effects specific to single groups in multiple group testing

I measured about 20.000 variables (let us say they are all normally distributed for simplicity) for one control group and 4 treatment groups. Now, I would like to answer the following question: Which ...

1
2 3 4 5
183