Return to Answer

Consider that they can all be written as a regression equation (perhaps with slightly differing interpretations than their traditional forms).

Regression: $$ Y=\beta_0 + \beta_1X_{\text{(continuous)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

t-test: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

ANOVA: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

The prototypical regression is conceptualized with $X$ as a continuous variable. However, the only assumption that is actually made about $X$ is that it is a vector of known constants. It could be a continuous variable, but it could also be a dummy code (i.e., a vector of $0$'s & $1$'s that indicates whether an observation is a member of an indicated group--e.g., a treatment group). Thus, in the second equation, $X$ could be such a dummy code, and the p-value would be the same as that from a t-test in its more traditional form.

The meaning of the betas would differ here, though. In this case, $\beta_0$ would be the mean of the control group (for which the entries in the dummy variable would be $0$'s), and $\beta_1$ would be the difference between the mean of the treatment group and the control group.

Remember that it is perfectly reasonable to have / run an ANOVA with only two groups (although a t-test would be more common), and you have all three connected. If you prefer seeing how it would work if you had an ANOVA with 3 groups; it would be: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code 1)}} + \beta_2X_{\text{(dummy code 2)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$ Note that when you have $g$ groups, you have $g-1$ dummy codes to represent them. The reference group (typically the control group) is indicated by having $0$'s for both dummy code 1 & dummy code 2. In this case, you would not want to interpret the p-values of the t-tests for these betas that come with standard statistical output--they only indicate whether the indicated group differs from the control group when assessed in isolation. That is, these tests are not independent. Instead, you would want to assess whether the group means vary by constructing an ANOVA table and conducting an F-test. For what it's worth, the betas are interpreted just as with the t-test version described above: $\beta_0$ is the mean of the control / reference group, $\beta_1$ indicates the difference between the means of group 1 and the reference group, and $\beta_2$ indicates the difference between group 2 and the reference group.

In light of @whuber's comments below, these can also be represented via matrix equations:
$$ \bf Y=\bf X\boldsymbol\beta + \boldsymbol\varepsilon $$ Represented this way, $\bf Y$ & $\boldsymbol\varepsilon$ are vectors of length $N$, and $\boldsymbol\beta$ is a vector of length $p+1$. $\bf X$ is now a matrix with $N$ rows and $(p+1)$ columns. In a prototypical regression you have $p$ continuous $X$ variables and the intercept. Thus, your $\bf X$ matrix is composed of a series of column vectors side by side, one for each $X$ variable, with a column of $1$'s on the far left for the intercept.

If you are representing an ANOVA with $g$ groups in this way, remember that you would have $g-1$ dummy variables indicating the groups, with the reference group indicated by an observation having $0$'s in each dummy variable. As above, you would still have an intercept. Thus, $p=g-1$.

Consider that they can all be written as a regression equation (perhaps with slightly differing interpretations than their traditional forms).

Regression: $$ Y=\beta_0 + \beta_1X_{\text{(continuous)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

t-test: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

ANOVA: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

The prototypical regression is conceptualized with $X$ as a continuous variable. However, the only assumption that is actually made about $X$ is that it is a vector of known constants. It could be a continuous variable, but it could also be a dummy code (i.e., a vector of $0$'s & $1$'s that indicates whether an observation is a member of an indicated group--e.g., a treatment group). Thus, in the second equation, $X$ could be such a dummy code, and the p-value would be the same as that from a t-test in its more traditional form.

The meaning of the betas would differ here, though. In this case, $\beta_0$ would be the mean of the control group (for which the entries in the dummy variable would be $0$'s), and $\beta_1$ would be the difference between the mean of the treatment group and the mean of the control group.

Now, remember that it is perfectly reasonable to have / run an ANOVA with only two groups (although a t-test would be more common), and you have all three connected. If you prefer seeing how it would work if you had an ANOVA with 3 groups; it would be: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code 1)}} + \beta_2X_{\text{(dummy code 2)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$ Note that when you have $g$ groups, you have $g-1$ dummy codes to represent them. The reference group (typically the control group) is indicated by having $0$'s for all dummy codes (in this case, both dummy code 1 & dummy code 2). In this case, you would not want to interpret the p-values of the t-tests for these betas that come with standard statistical output--they only indicate whether the indicated group differs from the control group when assessed in isolation. That is, these tests are not independent. Instead, you would want to assess whether the group means vary by constructing an ANOVA table and conducting an F-test. For what it's worth, the betas are interpreted just as with the t-test version described above: $\beta_0$ is the mean of the control / reference group, $\beta_1$ indicates the difference between the means of group 1 and the reference group, and $\beta_2$ indicates the difference between group 2 and the reference group.

In light of @whuber's comments below, these can also be represented via matrix equations:
$$ \bf Y=\bf X\boldsymbol\beta + \boldsymbol\varepsilon $$ Represented this way, $\bf Y$ & $\boldsymbol\varepsilon$ are vectors of length $N$, and $\boldsymbol\beta$ is a vector of length $p+1$. $\bf X$ is now a matrix with $N$ rows and $(p+1)$ columns. In a prototypical regression you have $p$ continuous $X$ variables and the intercept. Thus, your $\bf X$ matrix is composed of a series of column vectors side by side, one for each $X$ variable, with a column of $1$'s on the far left for the intercept.

If you are representing an ANOVA with $g$ groups in this way, remember that you would have $g-1$ dummy variables indicating the groups, with the reference group indicated by an observation having $0$'s in each dummy variable. As above, you would still have an intercept. Thus, $p=g-1$.

Consider that they can all be written as a regression equation (perhaps with slightly differing interpretations than their traditional forms).

Regression: $$ Y=\beta_0 + \beta_1X_{\text{(continuous)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

t-test: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

ANOVA: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

The prototypical regression is conceptualized with $X$ as a continuous variable. However, the only assumption that is actually made about $X$ is that it is a vector of known constants. It could be a continuous variable, but it could also be a dummy code (i.e., a vector of $0$'s & $1$'s that indicates whether an observation is a member of an indicated group--e.g., a treatment group). Thus, in the second equation, $X$ could be such a dummy code, and the p-value would be the same as that from a t-test in its more traditional form.

The meaning of the betas would differ here, though. In this case, $\beta_0$ would be the mean of the control group (for which the entries in the dummy variable would be $0$'s), and $\beta_1$ would be the difference between the mean of the treatment group and the control group.

Remember that it is perfectly reasonable to have / run an ANOVA with only two groups (although a t-test would be more common), and you have all three connected. If you prefer seeing how it would work if you had an ANOVA with 3 groups; it would be: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code 1)}} + \beta_2X_{\text{(dummy code 2)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$ Note that when you have $g$ groups, you have $g-1$ dummy codes to represent them. The reference group (typically the control group) is indicated by having $0$'s for both dummy code 1 & dummy code 2. In this case, you would not want to interpret the p-values of the t-tests for these betas that come with standard statistical output--they only indicate whether the indicated group differs from the control group when assessed in isolation. That is, these tests are not independent. Instead, you would want to assess whether the group means vary by constructing an ANOVA table and conducting an F-test. For what it's worth, the betas are interpreted just as with the t-test version described above: $\beta_0$ is the mean of the control / reference group, $\beta_1$ indicates the difference between the means of group 1 and the reference group, and $\beta_2$ indicates the difference between group 2 and the reference group.

Consider that they can all be written as a regression equation (perhaps with slightly differing interpretations than their traditional forms).

Regression: $$ Y=\beta_0 + \beta_1X_{\text{(continuous)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

t-test: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

ANOVA: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

The prototypical regression is conceptualized with $X$ as a continuous variable. However, the only assumption that is actually made about $X$ is that it is a vector of known constants. It could be a continuous variable, but it could also be a dummy code (i.e., a vector of $0$'s & $1$'s that indicates whether an observation is a member of an indicated group--e.g., a treatment group). Thus, in the second equation, $X$ could be such a dummy code, and the p-value would be the same as that from a t-test in its more traditional form.

The meaning of the betas would differ here, though. In this case, $\beta_0$ would be the mean of the control group (for which the entries in the dummy variable would be $0$'s), and $\beta_1$ would be the difference between the mean of the treatment group and the control group.

Remember that it is perfectly reasonable to have / run an ANOVA with only two groups (although a t-test would be more common), and you have all three connected. If you prefer seeing how it would work if you had an ANOVA with 3 groups; it would be: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code 1)}} + \beta_2X_{\text{(dummy code 2)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$ Note that when you have $g$ groups, you have $g-1$ dummy codes to represent them. The reference group (typically the control group) is indicated by having $0$'s for both dummy code 1 & dummy code 2. In this case, you would not want to interpret the p-values of the t-tests for these betas that come with standard statistical output--they only indicate whether the indicated group differs from the control group when assessed in isolation. That is, these tests are not independent. Instead, you would want to assess whether the group means vary by constructing an ANOVA table and conducting an F-test. For what it's worth, the betas are interpreted just as with the t-test version described above: $\beta_0$ is the mean of the control / reference group, $\beta_1$ indicates the difference between the means of group 1 and the reference group, and $\beta_2$ indicates the difference between group 2 and the reference group.

In light of @whuber's comments below, these can also be represented via matrix equations:
$$ \bf Y=\bf X\boldsymbol\beta + \boldsymbol\varepsilon $$ Represented this way, $\bf Y$ & $\boldsymbol\varepsilon$ are vectors of length $N$, and $\boldsymbol\beta$ is a vector of length $p+1$. $\bf X$ is now a matrix with $N$ rows and $(p+1)$ columns. In a prototypical regression you have $p$ continuous $X$ variables and the intercept. Thus, your $\bf X$ matrix is composed of a series of column vectors side by side, one for each $X$ variable, with a column of $1$'s on the far left for the intercept.

If you are representing an ANOVA with $g$ groups in this way, remember that you would have $g-1$ dummy variables indicating the groups, with the reference group indicated by an observation having $0$'s in each dummy variable. As above, you would still have an intercept. Thus, $p=g-1$.

Consider that they can all be written as a regression equation (perhaps with slightly differing interpretations than their traditional forms).

Regression:
  $$ Y=\beta_0 + \beta_1X_{\text{(continuous)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

t-test: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

ANOVA: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

The prototypical regression is conceptualized with $X$ as a continuous variable. However, the only assumption that is actually made about $X$ is that it is a vector of known constants. It could be a continuous variable, but it could also be a dummy code (i.e., a vector of $0$'s & $1$'s that indicates whether an observation is a member of an indicated group--e.g., a treatment group). Thus, in the second equation, $X$ could be such a dummy code, and the p-value would be the same as that from a t-test in its more traditional form.

The meaning of the betas would differ here, though. In this case, $\beta_0$ would be the mean of the control group (for which the entries in the dummy variable would be $0$'s), and $\beta_1$ would be the difference between the mean of the treatment group and the control group.

Remember that it is perfectly reasonable to have / run an ANOVA with only two groups (although a t-test would be more common), and you have all three connected. If you prefer, you could imagine an ANOVA with 3 groups; it would be: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code 1)}} + \beta_2X_{\text{(dummy code 2)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$ Note that when you have $g$ groups, you have $g-1$ dummy codes to represent them. The reference group (typically the control group) is indicated by having $0$'s for both dummy code 1 & dummy code 2. In this case, you would not want to interpret the p-values of the t-tests for these betas that come with standard statistical output--they only indicate whether the indicated group differs from the control group when assessed in isolation. That is, these tests are not independent. Instead, you would want to assess whether the group means vary by constructing an ANOVA table and conducting an F-test. For what it's worth, the betas are interpreted just as with the t-test version described above: $\beta_1$ indicates the difference between group 1 and the control / reference group, and $\beta_2$ indicates the difference between group 2 and the reference group.

Consider that they can all be written as a regression equation (perhaps with slightly differing interpretations than their traditional forms).

Regression: $$ Y=\beta_0 + \beta_1X_{\text{(continuous)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

t-test: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

ANOVA: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$

The prototypical regression is conceptualized with $X$ as a continuous variable. However, the only assumption that is actually made about $X$ is that it is a vector of known constants. It could be a continuous variable, but it could also be a dummy code (i.e., a vector of $0$'s & $1$'s that indicates whether an observation is a member of an indicated group--e.g., a treatment group). Thus, in the second equation, $X$ could be such a dummy code, and the p-value would be the same as that from a t-test in its more traditional form.

The meaning of the betas would differ here, though. In this case, $\beta_0$ would be the mean of the control group (for which the entries in the dummy variable would be $0$'s), and $\beta_1$ would be the difference between the mean of the treatment group and the control group.

Remember that it is perfectly reasonable to have / run an ANOVA with only two groups (although a t-test would be more common), and you have all three connected. If you prefer seeing how it would work if you had an ANOVA with 3 groups; it would be: $$ Y=\beta_0 + \beta_1X_{\text{(dummy code 1)}} + \beta_2X_{\text{(dummy code 2)}} + \varepsilon \\ \text{where }\varepsilon\sim\mathcal N(0, \sigma^2) $$ Note that when you have $g$ groups, you have $g-1$ dummy codes to represent them. The reference group (typically the control group) is indicated by having $0$'s for both dummy code 1 & dummy code 2. In this case, you would not want to interpret the p-values of the t-tests for these betas that come with standard statistical output--they only indicate whether the indicated group differs from the control group when assessed in isolation. That is, these tests are not independent. Instead, you would want to assess whether the group means vary by constructing an ANOVA table and conducting an F-test. For what it's worth, the betas are interpreted just as with the t-test version described above: $\beta_0$ is the mean of the control / reference group, $\beta_1$ indicates the difference between the means of group 1 and the reference group, and $\beta_2$ indicates the difference between group 2 and the reference group.