1
$\begingroup$

I have been reading about simple linear regression ($y=wx+b$), and I started a section when it talks about null hypothesis $w=0$. Then, when it wants to calculate p-value, they use $t-score = \frac{\bar{w}-0}{SE}$, where SE is the standard error. What I do not understand is this fraction.

In z-score, we have $\frac{x-\mu}{\sigma}$, and $\mu$ and $\sigma$ are parameters of distribution. So, similarly, regarding the t-score, I should think that 0 and $SE$ are parameters of distribution (of null hypothesis. correct?). However, if the null hypothesis says the slope, $w$, is zero, why should I consider a distribution for it? And, if there is a distribution, why is its spread parameter equal to $SE$, which is an estimation of standard deviation of $w$ (calculated based on alternative hypothesis)?

I tried to look at different articles and a couple of books, but whenever I get to this part, I still feel like something is missing.

$\endgroup$

2 Answers 2

4
$\begingroup$

Why does null hypothesis in simple linear regression (i.e. slope = 0) have distribution?

A null hypothesis is not a random variable; it doesn't have a distribution.

A test statistic has a distribution. In particular we can compute what the distribution of some test statistic would be if the null hypothesis were true.

If the sample value of the test statistic is such that this value or one more extreme (further toward what you're expect if the alternative were true) would be particularly rarely observed if the null were true, then we have a choice between saying "the null is true but some very rare event happened" and "the null is not true and we needn't invoke an unusual event to explain it".

As the chance of observing something at least as unusual as our sample's test statistic becomes very small, the null becomes harder to maintain as an explanation. We choose to reject the null for the most extreme of these and not to reject the null for the test statistics that would not be surprising. The least extreme test statistic that we would still reject is the critical value, and that and all more extreme values form the rejection region (the set of values of the test statistic that lead us to reject $H_0$).

I should think that 0 and SE are parameters of distribution (of null hypothesis. correct?).

$0$ is the value of the mean parameter under a particular null hypothesis but the standard error in the denominator is not a parameter; it is a sample estimate of a parameter - it's an estimate of the standard deviation of the distribution of $\hat{w}$.

However, if the null hypothesis says the slope, w, is zero, why should I consider a distribution for it?

You're conflating the population slope with the sample slope. The population slope is hypothesized to be $0$ but if that were true, the sample slope would not be $0$. It would be some number more or less "near" $0$.

To see if the sample slope is "too far" from $0$ to be reasonably consistent with having come from a population slope of $0$, we need to know how much we should reasonably expect the sample slope to vary from $0$.

That is why we need to consider the distribution that the sample slope $\hat{w}$ would have if $H_0$ were true.

And, if there is a distribution, why is its spread parameter equal to SE, which is an estimation of standard deviation of w (calculated based on alternative hypothesis)?

In this particular situation (regression), the standard deviation of the distribution of $\hat{w}$ (not $w$) is a sensible estimator under both hypotheses, not just under the alternative. It's perfectly reasonable to use it in this case.

$\endgroup$
4
  • $\begingroup$ Thank you so much for the response. The two statements (1) population slope vs sample slope in $H_{0}$, and (2) "the standard deviation of the distribution of $\hat{w}$ (not w) is the same under both hypotheses" helped me a lot! Just to confirm my understanding: the t-score $\frac{\bar{w}-0}{SE(\bar{w})}$, is similar to z-score format: 0 and $SE(\bar{w})$ are parameters of t-distribution before standardization, and I want to calculate the area under the curve: $P(w\ge \bar{w})$, and for two-sided p-value, I have to double it. Correct? $\endgroup$
    – Nima S
    Commented Feb 6, 2022 at 16:55
  • $\begingroup$ 1. What area you want to calculate depends on the alternative (since that defines "more extreme". 2. Define your symbols with care. Note that the specific probability you want to calculate would compare a random variable (the estimated value of the slope from a regression when $H_0$ is true) with the specific value you observed. But beware that you don't do a calculation that could lead to a two-tailed probability greater than $1$. You need to consider what cases "more extreme" -- when in doubt, draw a diagram. $\endgroup$
    – Glen_b
    Commented Feb 6, 2022 at 22:54
  • $\begingroup$ I see. So, just to wrap up (case: linear regression)>>> $H_{0}: w=0$. If true, I expect my $\hat{w}$ (from samples) to be close to 0 (with std equal to $SE(\hat{w})$). How far is it from 0? Of course, I need to measure it in terms of std (that is why I have $SE$ in the denominator). In my opinion, $H_{a}$ means slope not zero, so I can consider extremely positive (+inf) OR negative (-inf). By the way, both +/- slopes make sense in the example I was working with. Could you please let me know if my train of thoughts is correct? $\endgroup$
    – Nima S
    Commented Feb 7, 2022 at 4:34
  • 1
    $\begingroup$ That all looks right. $\endgroup$
    – Glen_b
    Commented Feb 7, 2022 at 7:36
0
$\begingroup$

The null hypothesis is nothing more than a specified value of the parameter of interest in the statistical model. It is the point in parameter space where $\omega=0$. (More completely, it is the line in parameter space where $\omega=0$ over the range of all possible $\beta$ values.) It is probably not helpful to think of it 'having a distribution'.

The things where we worry about distributions are typically populations and statistics such as the test statistic $t$. Within the statistical model (the equations and assumptions underlying the method) the test statistic has a known distribution so that you can compare the observed value of the statistic to that distribution. You can therefore see how 'extreme' the observed value is (i.e. how unexpected it is, or how strange it is) relative to its distribution.

The statistical model also allows determination of what the distribution of the test statistic would be if the 'true' value of the parameter of interest was any value other than the null hypothesis.

The assumptions relating to population distribution are usually necessary in order that the test statistic will have a particular known distribution.

If statistical model shows that the observed value of the test statistic would rarely be found under the null hypothesis then the data cast doubt on that null hypothesis.

$\endgroup$
1
  • $\begingroup$ @ Michael Lew: Thanks for the answer! I am going through it and trying to understand it. Would you mind explain what the random variable(rv) is in t-distribution? (or provide me with a link that can explain it). What I am trying to understand here is what that rv is given that null hypothesis is correct. When you said "The assumptions relating to population distribution are usually necessary", may I ask population of what (rv)? (let us focus on simple univariate linear regression) $\endgroup$
    – Nima S
    Commented Feb 5, 2022 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.