Skip to main content

New answers tagged

1 vote

Statistical Inference on Samples vs Populations

You write "I want to know if this change (2%) can be attributable to randomness or non-randomness (i.e. statistically significant)." But "statistical significance" is not merely a ...
Graham Wright's user avatar
1 vote

Statistical Inference on Samples vs Populations

I already wrote in comments that chances are applying any standard test to the example situation would be misleading/invalid even in case that the "2nd interpretation" is taken. However in ...
Christian Hennig's user avatar
1 vote

Why is my pvalue for chisquared test so high?

It's not clear where your doubt arises. The standard error of your proportion (either one) under $H_0$ is $\sqrt{\frac{1\times 3}{4\times 4 \times 100000}}\approx 0.00137$. Meanwhile the "error&...
Glen_b's user avatar
  • 286k
2 votes
Accepted

Why is my pvalue for chisquared test so high?

Whether you meant to do this or not, your null hypothesis is that there is a 3:1 ratio. Indeed, there is such a 3:1 ratio at the population level, and your empirical data show roughly a 3:1 ratio. ...
Dave's user avatar
  • 65k
0 votes

How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?

Based on your description 0.4 +ve derivative from zero to 0.4 and around zero or slightly negative derivative up to 1. You could fit $y = a_0 + a_1 x + a_2 (x-0.4)^+$, which is a piecewise linear ...
seanv507's user avatar
  • 7,268
0 votes
Accepted

How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?

A Q-Q plot approach would not be appropriate for your objective. Not only that, it would give you misleading results. This is because a Q-Q plot changes the order of the data, arranging it in ...
jginestet's user avatar
  • 1,849
1 vote

The F-test and the t-test reject the null hypothesis while the KS test and the Chi-Squared test do not

The lack of rejection by those tests is not a confirmation of their null hypotheses. What the lack of rejection means is that there is insufficient evidence for that particular test to refute the null ...
Dave's user avatar
  • 65k
0 votes

Statistical Inference on Samples vs Populations

You wrote, "I want to know if this change (2%) can be attributable to randomness or non-randomness [....]". That may sound, on the surface, to be equivalent to asking whether the two groups ...
rolando2's user avatar
  • 12.8k
1 vote

Non-parametric one-sample mean test for a bounded variable (based on Chebyshev's inequality?)

Let $\mathcal B$ be the ensemble of distributions of random variables bounded to $[0, 1]$. We have $n$ variables $x_1 ... x_n$ IID from some distribution in $\mathcal B$. We want to test the null $E(X)...
Guillaume Dehaene's user avatar
0 votes

Statistical Inference on Samples vs Populations

My view is that, if you have the whole population, you have no inference to do. Inference is about going from a sample to a population. Some people posit some sort of "super population" and ...
Peter Flom's user avatar
  • 125k
4 votes

How to show that many functions (a hundred, a thousand) have the same shape an distribution of values over an interval?

To show that several functions are more or less the same you could just superimpose them graphically. I don't think quantile plots of any flavour are directly relevant or likely to be helpful. The ...
Nick Cox's user avatar
  • 58.6k
-1 votes

Can hypothesis testing be done on two points?

There is absolutely no issue with what you are trying to do. It is indeed a test of 2 proportions, aka a very basic, simple 2x2 contingency matrix. A fisher-exact, or $\chi^2$ test will give you the ...
jginestet's user avatar
  • 1,849
2 votes

Can hypothesis testing be done on two points?

I see it as a test on equality of proportions. This, however, only would hold if you had the exact numbers. You can take a look here for performing such test in R: http://www.sthda.com/english/wiki/...
Federico Tedeschi's user avatar
-3 votes

Can hypothesis testing be done on two points?

In frequentist statistics the idea is to compare the distributions of data, and the more data you have in each group, the more reliable result you will get. From this perspective comparing a single ...
Mikolaj Buchwald's user avatar
4 votes

How does one develop an intuitive understanding of statistical methods and the ability to "know" which test to use, why, and how to interpret results

I teach statistics to biology students. They're usually less than thrilled about statistics coming in. They also come to us for statistical consultancy during research projects Personally, the biggest ...
2 votes
Accepted

ANOVA finding a main effect prevents inflation of Type I error rate

...these multiple comparisons do not inflate the Type I error rate because they are only conducted if the ANOVA finds a main effect. This is an example of right for the wrong reasons. Even if the ...
Frans Rodenburg's user avatar
1 vote

Underpowered studies and minimum effect size

Wouldn't we always want to specify (our best estimate of) the true effect size? YES! Always specify the observed effect and if your best estimate differs from that then you must say how and why. And ...
Michael Lew's user avatar
  • 16.2k
-1 votes

Underpowered studies and minimum effect size

I have switched from the term Null hypothesis to the term Main hypothesis. This makes it clearer that you can pick any of the two alternatives as the Main hypothesis which you protect with the choice ...
W_vH's user avatar
  • 339
1 vote

Underpowered studies and minimum effect size

It depends on the context. For one, if you have good reasons to think the true effect size is much smaller than the smallest effect size of interest (SESOI), and if that means that the SESOI is ...
J-J-J's user avatar
  • 4,944
1 vote

What criterion to use to compare multiple correlations of binary variables?

I think the problem might be that your formulation of null-hypothesis is not actionable. 'Question formulated less clearly than the rest' ... how would one derive something computable from it? You ...
Cryo's user avatar
  • 783
1 vote
Accepted

Non-parametric one-sample mean test for a bounded variable (based on Chebyshev's inequality?)

I think for this there are several well known concentration inequalities that can be applied. In particular Hoeffding, Azuma, and McDiarmid. Not that there's any real difference between the bounds one ...
MotiNK's user avatar
  • 1,624
0 votes

Can p-values for Pearson's correlation test be computed just from correlation coefficient and sample size?

Yes, the p-value can be computed from the sample correlation and sample size There are various versions of the Pearson correlation test that use different test statistics based on different ...
Ben's user avatar
  • 129k
10 votes
Accepted

Related low p-values that do not meet statistically significant thresholds

Interesting question, let me break it down into a few distinct problems: Do multiple borderline significant results indicate overall significance? The answer to this one is simple. No. If anything, ...
Frans Rodenburg's user avatar
3 votes
Accepted

Power of One Sided t-test

It doesn't have a closed form. The wikipedia article gives the cdf in terms of infinite series and nonstandard ('not closed form') functions https://en.wikipedia.org/wiki/Noncentral_t-distribution#...
Glen_b's user avatar
  • 286k
0 votes

Looking for a statistical test for significance of association between a treatment and non-mutually exclusive categories

But don't you have 4 mutually exclusive categories? A, B, A&B, neither. Per your problem statement, a cell has to be in one, and only 1 of these 4 states (after all, this is the case of the ...
jginestet's user avatar
  • 1,849
1 vote
Accepted

Test whether my new method's type I error rate correctly matches chosen alpha

Empirical validation of null-rejection methods tldr: in null-rejection testing, the empirical rate is typically different than the nominal rate (due to approximations), but it only matters if it is ...
Guillaume Dehaene's user avatar
1 vote

Looking for a statistical test for significance of association between a treatment and non-mutually exclusive categories

Had the categorical variable been an independent variable, this is a somewhat standard problem of multiple regression. For a multiple choice variable with $k$ categories, instead of having $k-1$ ...
Frank Harrell's user avatar
0 votes

Test whether my new method's type I error rate correctly matches chosen alpha

This is not an answer about statistical methods, but only to point out that it is common practice in the literature to not actually test whether a rejection rate is significantly different than the ...
A Friendly Fish's user avatar
2 votes

Impact of sample size on metric lift

OP as you have stated - the purpose of your analysis is to see whether the results do not change depending on the number of people in the test. (and this is related to your previous question Impact of ...
seanv507's user avatar
  • 7,268
3 votes

Non-parametric one-sample mean test for a bounded variable (based on Chebyshev's inequality?)

Here is a maximum likelihood approach. Suppose you have $n$ observations of $X$ which total $T$, and you want to test the null hypothesis $E[X]=\mu$, with $T/n<\mu$. Then the maximum-likelihood ...
Matt F.'s user avatar
  • 5,342
1 vote

Impact of sample size on metric lift

I assume that model assumptions for the CIs are fulfilled, which in reality may not be the case. Still the answer is no. For starters, your CIs are 95% CIs and they may not cover the true metric lift (...
Christian Hennig's user avatar
0 votes

Interpretability for chi-squared test?

I'm responding to this post for future reference and to expand on a potentially interesting aspect that has not been fully covered in previous answers (i.e., the selection of an appropriate ...
NewAtGis's user avatar
  • 151
2 votes
Accepted

ML V REML for Hypothesis Testing

In the comments section of the first answer you can read that regarding the recommendation for using REML for model comparison, "your nested models need to have the same variables with fixed ...
Christian Hennig's user avatar
0 votes

How can I show statistically that one of my replicates is likely contaminated?

I personally do not think that there is such a thing as an "outlier"; there is either erroneous data (measurement error, transcription error, etc.), or then there is data. It may be ...
jginestet's user avatar
  • 1,849
2 votes

Is it possible for some p-values to be impossible? (because statistic generated by parametric bootstrap is mostly the same value.)

If $A_n$ differs substantially from the identity matrix - where "substantially" depends upon the sample size - the MLE of $\lambda$ will be driven to zero quite often, and your results will ...
jbowman's user avatar
  • 40.5k
4 votes

Is it possible for some p-values to be impossible? (because statistic generated by parametric bootstrap is mostly the same value.)

As others have mentioned, yes there are cases where some p-values are impossible. But I don't think that is the case here. It is also possible that there is an error in your calculations. But it is ...
Greg Snow's user avatar
  • 52.5k
4 votes

Are there alternative statistics to a p-value in NHST?

Your last statement talks about the probability of a hypothesis being true. That can be approached by Bayesian statistics. But that is a whole topic that I will not get into here (but is a good ...
Greg Snow's user avatar
  • 52.5k
4 votes

Are there alternative statistics to a p-value in NHST?

To put it briefly, this is now an operational question. Your observations are sound. In fact, nearly every statistician has (or should) consider the exact problem as you've described it. Closely tied ...
AdamO's user avatar
  • 63.7k
2 votes

How to perform a joint significance test for multiple coefficients across several regression models?

You can do this using a combination of weighted least squares (to account for likely heteroskedasticity), an F-test, and an appropriately structured linear regression. First, we construct our ...
jbowman's user avatar
  • 40.5k
1 vote

How to perform a joint significance test for multiple coefficients across several regression models?

Your goal is to show evidence that the each b-coefficients is zero. This is called "equivalence testing" and often performed by two one-sided tests, abbreviated as "TOST". This can ...
BenP's user avatar
  • 1,838
1 vote

How to perform a joint significance test for multiple coefficients across several regression models?

Testing the null hypothesis $\beta_{1x}=\ldots=\beta_{10x}=0$ assuming that the standard assumptions hold, two possibilties come immediately to mind. One is that you can test the null hypotheses $\...
Christian Hennig's user avatar
2 votes

t-test for partially paired and partially unpaired data

Building on Frank's comment on Apr 7, 2012 at 15:51, about "gls", below is a small example using gls from R package nlme. As correlation structure I used heterogeneous compound symmetry. ...
BenP's user avatar
  • 1,838
1 vote
Accepted

Why do we compute the standard deviation of the proportion using the value assumed under the null hypothesis?

The latest example in the excerpt is a large sample test concerning a single population proportion. Let's discuss briefly the framework involved in a generalized manner: The null hypothesis is $\...
User1865345's user avatar
  • 9,427
0 votes

Hypothesis testing with control and treatment group - which statistical analysis to use?

If your data is arranged as it seems, then you can simply subtract the t1 measurement for each participant from the t2 measurement and then use a Student's t-test to compare the mean differences. That ...
Michael Lew's user avatar
  • 16.2k
7 votes

confidence interval and rejection

Yes, you should consider providing both the p-value for your chosen null and a confidence interval. The mathematical link between hypothesis testing and confidence intervals should not drive your ...
Graham Bornholt's user avatar
6 votes

confidence interval and rejection

YES I would consider this a best-practice. While you are correct to recognize the relationship between hypothesis testing and confidence intervals, the confidence interval gives a range of "...
Dave's user avatar
  • 65k
0 votes

What is correct test to investigate difference between two conditions with repeated measures per subject?

I apoligize if I do not provide an answer to your question, but I see many issues with the overall experiment. I will try to list them below. Maybe you can clarify some of them, and I can then procide ...
jginestet's user avatar
  • 1,849
1 vote
Accepted

Testing dependence of two categorical variables with data separated by test subject

So, if we can assume that the datapoints are indeed independent, your first inclination of using a 2x2 contingency table (with $\chi^2$ or Fisher-exact) would work, and give you a valid result; it ...
jginestet's user avatar
  • 1,849
0 votes

What is correct test to investigate difference between two conditions with repeated measures per subject?

RM ANOVA makes assumptions that seem unlikely to be met here, primarily sphericity. One part of that is that all the covariances are equal. Also, RM ANOVA doesn't deal well with missing data, if you ...
Peter Flom's user avatar
  • 125k
1 vote

Testing the effect of a continious IV on DV, in order to explain group differences

In my view you overinterpret the difference between "significant" and "not significant". Note that an insignificant result does not mean that the null hypothesis is true, i.e., ...
Christian Hennig's user avatar

Top 50 recent answers are included