New answers tagged

0 votes

Testing similarity of rank orders

The Friedman test seems appropriate for testing your null hypothesis here. It's equivalent to a significance test of Kendall's W, which is sometimes used as an index of agreement between rankings that ...
user avatar
  • 690
0 votes

Hypothesis Testing: Different p-values from different test statistics

Depending on what you know and one the assumptions you are willing to make about the population from which your sample was taken, there may be many different ways to test whether $H_0: \mu=\mu_0$ ...
user avatar
  • 50.8k
0 votes

What statistical test is suitable to summarise patient time in hospital with health condition

Your model seem to indicate that age and health condition are significantly associated with the number of days in the hospital. While interpretation of the age is easy (the older the patients, the ...
user avatar
11 votes

When the type of test does not fit the research purpose

First review your understanding of ANOVA & its relation to (ordinary least-squares) regression. "In addition, adding several variables in the logistic model changes the significance of the ...
user avatar
1 vote

What statistical test is suitable to summarise patient time in hospital with health condition

A regression analysis could be useful in this case. You can estimate a multivariate linear regression model. Patient time in the hospital will be your dependent variable. Age and health condition (...
user avatar
1 vote

Approximate distribution of test statistic for weighted sample mean

For moderate tail probabilities, the Satterthwaite approximation is better than it has any right to be. It's anticonservative for small tail probabilities. There are two approximations, due to Davies ...
user avatar
4 votes

How do I test for significant change when looking at a metric such as "average products added to cart in per session"?

This is like, THE QUESTION, in AB testing. There are a bunch of ways to answer this, each of which have their pros and cons. I'll give my thoughts here for posterity. The t-test Perhaps the most ...
user avatar
0 votes

Approximate distribution of test statistic for weighted sample mean

If we assume T is a, possibly scaled, student-t distribution, then what is the degrees of freedom and possibly scaling? Let $W\sim\mathcal{N}(\mu_W, \sigma_W^2)$ with $\sigma_W^2=\sum_i \gamma_i$, ...
user avatar
2 votes

Wilcoxon Test - non normality, non equal variances, sample size not the same

Ben Bolker's answer is great. I just wanted to add an answer to the "Would another test be better?" part. If you want to be able to conclude that prices for women are greater than those for ...
user avatar
  • 1,600
1 vote

Are methods of p-value adjustment only necessary when a universal/overall hypothesis is the target of inference?

I think you are essentially correct. As you can see, the common thread in the three scenarios you describe is that there are multiple comparisons to make some overall inference. When you perform ...
user avatar
  • 97.2k
1 vote

Are methods of p-value adjustment only necessary when a universal/overall hypothesis is the target of inference?

Think of it like this. In a one-way ANOVA setup, you are testing whether the population means of more than two independent groups are equal. Clearly, if there are only two independent groups, the ...
user avatar
0 votes

Which statistical test should be used to compare two groups with biological and technical replicates?

I'd use the Friedman test. It's used when the groups are dependent (e.g. repeated measures or randomized block design). The format is friedman.test(y ~ A | B, data) ...
user avatar
  • 523
0 votes

Which statistical test should be used to compare two groups with biological and technical replicates?

Your primary interest is differences between two groups defined by expression of the receptor. Each individual cell line represents the corresponding class of cell lines: those either expressing the ...
user avatar
  • 64k
1 vote
Accepted

hypothesis testing with data that are not equally independent

I would set this up as a mixed model with group as a random effect to account for the non-independence within groups, and ...
user avatar
  • 12k
0 votes

What is the correct test to use for a categorial dependent variable and an ordinal independent variable?

My ordinal independent variable is numbered from 1-9, with 9 being completely satisfied with the website, and 1 being completely dissatisfied. From your description this sounds more like your ...
user avatar
0 votes

Which statistical test should be used to compare two groups with biological and technical replicates?

A mixed-effects model (a.k.a. hierarchical model) with a random effect to account for the technical replicates would be the standard way to analyse your data. Random effects are not ideal when the ...
user avatar
  • 12k
4 votes
Accepted

Assumptions of Mann-Whitney test for at least ordinal data

I just stumbled across this, and since I am the author of Karch (2021) and do not fully agree with the answers so far, here are my two cents. I will skip the assumption of no ties as there is ...
user avatar
  • 1,600
3 votes
Accepted

Election result - how to say if we can determine a winner?

Statistical significance is a way of talking about the uncertainty associated with using the results of a random sample to make predictions about the characteristics of the larger population that ...
user avatar
3 votes

The meaning of Test Unit Roots: unanswerable and answerable questions

In my mind, these issues are not particularly wedded to unit roots. What Hamilton (and the papers cited there) seems to be saying is that parameters very close to each other are very hard to ...
user avatar
8 votes
Accepted

What are "poor finite sample properties"?

In the context of hypothesis tests, poor finite sample properties usually mean that the actual rejection rate of the test differs from the nominal one. Recall that the nominal one is the level at ...
user avatar
1 vote
Accepted

How to test a linear regression against a NULL expectation?

Since Y is calculated from X, I calculated the null by shuffling X from the original sample, calculate Y, the slope between them. Repeat 1000 times. Find the average slope. Use the equation of the ...
user avatar
  • 115k
1 vote

When not to look at p-value

Before you look at anything at all you should have a clear goal or research question. You need a reason why you want to look at the correlations. What are you gonna do with it? For instance, Do you ...
user avatar
1 vote

When not to look at p-value

If you are looking for correlations, your first look should not be at the p-value, but at the value of the correlation coefficient $r$. High correlation means high $|r|$, not high p-value. The p-value ...
user avatar
  • 3,111
5 votes
Accepted

Asymptotic normality of OLS estimators in practice

"I can trust the standard errors of my coefficients and their accompanying p-values" .... well, yes, at least in the limit as $n\to\infty$, as long as it's only the distributional assumption ...
user avatar
  • 261k
0 votes

When not to look at p-value

Documentation forcor.test in R says "By default, probabilities for method="spearman" and method="kendall" are found by normal theory. If ...
user avatar
  • 50.8k
1 vote

When not to look at p-value

It sounds like you might want to do exploratory data analysis instead of a deep dive into the meaning of p-values. Basically, make plots, lots of them, to understand the patterns in your data. It ...
user avatar
  • 2,742
1 vote

McNemar test or Two proportion z-test?

The data is not paired because a participant chooses either A or B at their original preference. Compare with testing whether a patient has a disease with two different procedures A and B. So McNemar'...
user avatar
  • 2,742
1 vote

What to do if you Double count in experimental design?

First define a metric that monitors the customer behavior you are interested and that is related to the business goal you want to optimize for. Then pick an appropriate statistical procedure. Take ...
user avatar
  • 2,742
2 votes
Accepted

What to do if you Double count in experimental design?

modify the test we use to compute p-values when we double count? when we double count the sample, the iid assumption doesn't hold, then the A/B test is more like a cluster-randomized experiment: we ...
user avatar
  • 96
2 votes
Accepted

Difference between calculating a t-statistic manually and with R

I'd put my money on this being a small rounding error. Using more precise values from the second table of values ...
user avatar
  • 2,385
4 votes
Accepted

Hypothesis testing for $\sigma^2$ in linear regression

As pointed out in the comments, the result you give for the pivotal quantity in this case is already enough to perform the hypothesis test. (Remember that when you do a classical hypothesis test you ...
user avatar
  • 97.2k
1 vote
Accepted

alternative to ANOVA or simplifying approximations for multi-factor,, batch design with binomial data (advanced beginner)

Retrieve the original count data. Then treat this as a binomial logistic regression. In R with the glm() function, you can provide a two-column outcome specifying ...
user avatar
  • 64k
0 votes

H0;b1=0, b2=1 testing null hypothesis

Yes, the approach and code that you have stated is correct, and the F-test in the output will test the null hypothesis that b1=0 and b2=1 vs the alternative that b1 and b2 are unspecified. You are ...
user avatar
  • 9,719
0 votes
Accepted

Goodness of fit test for a transition density function of a Markov process

Great question! Here is my answer: Define the events $$ A_n = \{X_n^{(0.025)} \leq X_n \leq X_n^{(0.975)}\ |\ X_{n-1} = x_{n-1}\}. $$ Under $\mathbb{H}_0$, $\{A_n\}_{n = 2}^N$ is a collection of ...
user avatar
  • 11
0 votes

TOST for equivalence test

Generally, the value of $\varepsilon$ should be selected with the meaning "the smallest magnitude of difference between $\mu_1$ and $\mu_2$ that I care about." In practice, there may be ...
user avatar
  • 26.6k
3 votes

Hypothesis Testing when the aim is to claim that the two population mean (or any other statistic) are same

If your aim is to claim evidence that two (or more) populations are similar, for example, have similar means, then you are unequivocally looking for some variation of equivalence testing. In a two-...
user avatar
  • 26.6k
0 votes

Likelihood ratio test vs. p-value under the null hypothesis

Comment: I think you are mixing up (at least) two foundational approaches to testing a null hypothesis. If you can answer a few of my questions below, maybe that will help you answer your own question....
user avatar
  • 50.8k
0 votes

Appropriate test for proportions data

Suppose there is a Type D to represent the $\dots$ at the bottom of your list. Then you might ask whether outcomes might be evenly apportioned among A, B, C, and D. If there are $n = 1000$ altogether. ...
user avatar
  • 50.8k
1 vote

How important is the arrangement of data in a 2x2 contingency table for fishers exact test?

The usual way of constructing contingency tables is to let one factor be in rows, and the other in columns. That corresponds to your table 2, which is what you should use for your analysis, and yes, ...
user avatar
1 vote

How to do normality test for high dimension data?

From a practical standpoint, high-dimensional data will never be multinormal (unless generated by simulation). So always reject will be a fairly good test! See the answer at Standard Gaussianity test ...
user avatar
0 votes

Inter-laboratory results comparison

Correlation is not an appropriate metric for checking measurement agreement. The measurements [1, 1.5, 2] and [2, 2.5, 3] are perfectly correlated yet in poor agreement. Instead you can use inter-...
user avatar
  • 2,742
0 votes

Comparing 2 independent counts of the same data

Your question is about agreement, which has its own tag on this site agreement-statistics. An earlier question similar to yours is Testing for similarity on count data, also What is the proper inter-...
user avatar
0 votes

How to test proportion of each factor level against mean proportion across all levels (binary outcome)

You have a 4x2-contingency table, and can use a chi-squared test. But you say you want a test for each individual proportion. So use that an extension of the chi-squared test is binary logistic ...
user avatar
4 votes
Accepted

Quantile-quantile plot from scratch and comparing to multiple standard distributions

It looks like you think that plotting quantiles of one lognormal against a different lognormal should produce a straight line, but that is not the case in general. That works for location-scale ...
user avatar
  • 261k
0 votes

How can one show the sum of observed variables is "zero"?

A good setup for this problem is as an errors-in-variables model. We have observations, $x_i$, of underlying variables, $\bar x_i$, with noise $\epsilon_i$: $$ x_i = \bar x_i + \epsilon_i. $$ We are ...
user avatar
  • 1
0 votes
Accepted

Proof of the Student t-test for independent samples drawn from the same normal distribution when $\mu \neq 0$

The simple answer is that the statistic does not depend at all on $\mu$, and this is much easier to see from the original, non-transformed formula: $$t = \frac{\overline x - \overline y}{\sqrt{\frac{...
user avatar
0 votes

Chi squared test on group counts from different sizes

First of all, I am not a statistics expert. My understanding is that there are different questions behind your two chi-square tests. The first tests whether the (absolute) distribution of frequencies ...
user avatar
  • 13
0 votes

Correct Interpretation for One-tailed Hypothesis with Inverse Result

If you have any interest in saying anything but "we fail to reject $H_0$" when the relationship is in the opposite direction to the one you predicted, you should not have been using a one-...
user avatar
  • 261k
1 vote
Accepted

Best way to determine that two sets of sample means are not different (with 95% confidence)?

You're looking at this wrong! Almost certainly, your data is not "equal" to the data given in a research paper and it would be silly to try to claim that they are equal when there is any ...
user avatar
  • 6,228
4 votes

Correct Interpretation for One-tailed Hypothesis with Inverse Result

There are several possibilities: If you have made a mistake using software and tested against an alternative in the wrong tail from what you intended, then it is best to do the correct test. If the ...
user avatar
  • 50.8k

Top 50 recent answers are included