6
$\begingroup$

F test and t test are performed in regression models.

In linear model output in R, we get fitted values and expected values of response variable. Suppose I have height as explanatory variable and body weight as response variable for 100 data points.

Each variable (explanatory or independent variable, if we have multiple regression model) coefficient in linear model is associated with a t-value (along with its p value)? How is this t-value computed?

Also there is one F test at the end; again I am curious to know about its computation?

Also in ANOVA after linear model, I have seen a F-test.

Although I am new statistics learner and not from statistical background, I have gone through with lots of tutorials on this. Please do not suggest for going me with basic tutorials as i have already done that. I am only curious to know about the T and F test computation using some basic example.

$\endgroup$
9
  • $\begingroup$ What's a 'predictory' variable? From your text, it actually sounds like you mean 'response variable' $\endgroup$
    – Glen_b
    Commented Jul 5, 2013 at 6:45
  • $\begingroup$ yes ! response variable or independent variable . I am editing it . thanks $\endgroup$ Commented Jul 5, 2013 at 6:52
  • $\begingroup$ Whoah. Response variable = dependent variable = y-variable. Independent variable = explanatory variable = predictor variable = x-variable. Which is it? $\endgroup$
    – Glen_b
    Commented Jul 5, 2013 at 6:54
  • 1
    $\begingroup$ Thanks Glen_b, I am delighted with the learning of types of variables in regression models andthe answer given below by Maaten buis made me clear the concept. $\endgroup$ Commented Jul 5, 2013 at 8:59
  • 1
    $\begingroup$ @bioinformatician Here are lists of terms that may help you. Let's start with synonyms for "dependent variable" = "explained variable", "predictand", "regressand", "response", "endogenous", "outcome", "controlled variable". Next are some synonyms for "explanatory variable" = "independent variable", "predictor", "regressor", "stimulus", "exogenous", "covariate", "control variable". Some of these terms are more popular than others across different disciplines. $\endgroup$ Commented Jul 5, 2013 at 10:23

2 Answers 2

11
$\begingroup$

The misunderstanding is your first premise "F test and $t$-test are performed between two populations", this is incorrect or at least incomplete. The $t$-test that is next to a coefficient tests the null hypothesis that that coefficient equals 0. If the corresponding variable is binary, for example 0 = male, 1 = female, then that describes the two populations but with the added complication that you also adjust for the other covariates in your model. If that variable is continuous, for example years of education, you can think of comparing someone with 0 years of education with someone with 1 years of education, and comparing someone with 1 years of education with someone with 2 years of education, etc, with the constraint that each step has the same effect on the expected outcome and again with the complication that you adjust for the other covariates in your model.

An F-test after linear regression tests the null hypothesis that all coefficients in your model except the constant are equal to 0. So the groups that you are comparing is even more complex.

$\endgroup$
1
  • $\begingroup$ Dear Maarten Buis ! Nice explanation. My written Upvote to you :) ..my current reputation score does not permit me to vote :( !! $\endgroup$ Commented Jul 5, 2013 at 11:09
0
$\begingroup$

Some notations in the very beginning, I'm using z~N(0,1), u~χ2(p), v~χ2(q) and z, u and v are mutually independent(important condition)

  1. t = z/sqrt(u/p). For each of the coefficient βj, if you test whether h0: βj =0. Then (βj-0)/1 is basically z, and sample variances (n-2)S^2~χ2(n-2), then you also have your bottom part. So when t is large, which means it deviates from the H0 (significant p-value) and we reject Ho.
  2. F = (u/p)/(v/q), where u could have non-central parameters λ. How do you get two independent χ2 in general linear regression? Estimated βhat (the whole vector) and estimated sample variance s^2 are always independent. So F-test in linear regression basically are (SSR/k)/(SSE/(n-k-1)). (SSR: sum of squares of regression SSE: sum of squares of error). Under H0: β=0, top will have central chi-square (and therefore non-central F), otherwise, it will follow non-central test statistics. So if you want to know relationship between t and F, then think about the simple linear regression. Y=Xb+a (b is a scalar), then t-test for b and overall F test are the same thing.
  3. For (one-way) ANOVA, there are lots of statistical stuff regarding the non-full rank X matrix and estimable functions stuff, I don't want to burden you with all that. But the basic idea is, for example we have 4 treatment in covid-19, and we want to compare whether there is difference between the 4 groups. Then overall F = \sum{n=1}^{4-1}(Fi)/(4-1) for total (4-1) linearly independent orthogonal contrasts. So if the overall F has a big value, we would reject H0: no difference between 4 groups.

Lol I just realized you asked this question so many years ago and probably not confused anymore. But if there's any chance you're still interested, you can check out 'Linear model in statistics' book for more rigorous explanations. I was reviewing the book for my qualifier and happened to bump into this :)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.