Skip to main content
Search type Search syntax
Tags [tag]
Exact "words here"
Author user:1234
user:me (yours)
Score score:3 (3+)
score:0 (none)
Answers answers:3 (3+)
answers:0 (none)
isaccepted:yes
hasaccepted:no
inquestion:1234
Views views:250
Code code:"if (foo != bar)"
Sections title:apples
body:"apples oranges"
URL url:"*.example.com"
Saves in:saves
Status closed:yes
duplicate:no
migrated:no
wiki:no
Types is:question
is:answer
Exclude -[tag]
-apples
For more details on advanced search visit our help page
Results tagged with
Search options not deleted user 111259

Hypothesis testing assesses whether data are inconsistent with a given hypothesis (usually a null hypothesis of no effect).

4 votes
Accepted

Small effect size with large sample yields significant result, how to understand

I can't understand how a small effect can produce a significant result with a large sample? In short, this is because large samples offer us precise enough estimates to state that the parameter bein …
Demetri Pananos's user avatar
1 vote

Low performance on test set but high on cross-val

How long ago was the training set sampled? Whatever your studying may have a temporal component which left unmodelled will certainly lead to your model failing to internally validate. a) Z standard …
cottontail's user avatar
  • 1,099
0 votes
Accepted

Reliable method for hypothesis testing on zero-inflated continuous data

In what follows, please remember that I am showing an approach, not the approach. The approach depends on assumptions, much of which I can not validate (nor do I have the wish to validate thoroughly …
Demetri Pananos's user avatar
4 votes

Misunderstanding p-value

Your characterization of the p value is incorrect. For a two sided test, the correct interpretation of the p value is as follows: The p value represents the probability of observing a test statist …
Demetri Pananos's user avatar
1 vote

Evaluating a classifier's performance on different groups of subjects

Since precision and recall are basically binary outcomes, you could compare the precisions and recalls directly. However, with the given sample sizes I think you're probably going to have a poor time …
Demetri Pananos's user avatar
6 votes

Questions on Basic Data Cleansing For Linear Regression

I'm not sure what resources you are following, but I fear they may not be very good ones. First, it appears that you're plotting the marginal distribution of the outcome. You may be doing this to det …
Demetri Pananos's user avatar
3 votes
Accepted

Hypothesis testing: Difference of means (pre-treatment)

Your conclusion is/was correct. Randomization is not about balancing groups. Indeed, there is a good chance that unobserved confounders are unbalanced and yet randomization still works. In fact, t …
Demetri Pananos's user avatar
1 vote

Define metric to use after ab test is finished

What you are doing is essentially shooting the side of a barn and then later drawing targets around your bullet holes so as to make it appear that you are a good marksman. Setting aside the fact that …
Demetri Pananos's user avatar
4 votes

Hypothesis testing for detecting a (damped) sinusoidal signal in noise

It may be possible to use Bayesian statistics here. Let's set testing aside for a moment and instead focus on estimation. If we can construct a model and assess the estimates of $A$, maybe we can say …
Demetri Pananos's user avatar
2 votes

How can power and effect size help me interpret my A/B test results?

Surely, they can help me interpret the results of my test, one way or another? Power is something computed conditional on some assumption about the true effect size. If the effect size was at least …
Demetri Pananos's user avatar
3 votes
Accepted

How do data/observations themselves contribute to our ability to make determinations of whet...

This is a big topic, so I will provide highlights. It is true that statistics can not confirm a theory. However, we can say if data are consistent with some assumption about the world conditional on …
Demetri Pananos's user avatar
6 votes

Are post-hoc adjusted analyses more valid than initial analyses?

Does the statistically insignificant value from the post-hoc adjusted analysis supercede the initial analysis of the primary outcome which isn't adjusted for confounders? What I assume you're trying …
Demetri Pananos's user avatar
3 votes
Accepted

Anomaly detection via hypothesis test in Poisson distribution

I probably wouldn't do this with a hypothesis test, but we can use some probabilistic reasoning here. Poisson regression is a way to sort of "fit" a distribution on various covariates (like day of wee …
Demetri Pananos's user avatar
1 vote

Stat Sig Testing in Bayesian A/B/C+ Context

My question is this: in a bayesian test of proportion with more than 2 variants, is there a pretty standard approach for evaluating the statistical significance of the difference between their posts? …
Demetri Pananos's user avatar
1 vote

How do I compare the success rates of two groups of two different sample sizes?

OK, first thing's first. Let's just do a test of proportions and see what we get N_A <- 1000 cure_A <- 200 N_B <- 300000 cure_B <- 51000 prop.test(c(cure_A, cure_B), c(N_A, N_B)) #> #> 2-sample te …
Demetri Pananos's user avatar

1
2 3 4 5
10
15 30 50 per page