Different results for type I and type II ANOVA I am performing an ANOVA for a research question. I want to to test whether "Grade", "Nim" and "Tim" has a significant influence on "value". My question is regarding the different types for SS.
If I use type I, "Grade" and "Tim" are significant while "Nim" is not. This was to be expected if I evaluate the differences in means. However, if I use tpye 2 ANOVA, than also "Nim" is highly significant even if the means are almost equal. How can it be and how to interpret this?
The vector in "values" is actually dependent on another variable called (x) and I vary x in a certain range an test significant differnce in each obtained set of data. So this is just one example of a vector "value". I observed that even if the group means by Nim flipped in order, I dont obtain a p-value larger 0.05.
library(rstatix)

data <- data.frame(Grade=c(0, 0, 0, 1, 1, 1),
                   Nim =c(0, 1, 0, 0, 1, 0),
                   Tim = c(0, 0, 1, 0, 0, 1),
                   value=c(44.68, 59.53, 77.08, 39.17, 55.92, 70.27))
#Type I ANOVA
data %>% anova_test(value ~ Grade + Nim + Tim, type=1)

#Type II ANOVA
data %>% anova_test(value ~ Grade + Nim + Tim, type=2)

#means
data %>%
  group_by(Grade) %>%
  summarize(mean = mean(value))

data %>%
  group_by(Nim) %>%
  summarize(mean = mean(value))

data %>%
  group_by(Tim) %>%
  summarize(mean = mean(value))

 A: There are two issues with your analysis. (a) You are trying to interpret the statistical significance of predictors rather than understand their effect on the outcome. In other words, you are looking at p-values rather than at a scatter plot of your data. (b) You are comparing the unadjusted group means for each predictor separately while the ANOVA sums of squares are adjusted.
Since you know the difference between the type I and type II ANOVA, I won't explain it. If you need a review, read this great answer: How to interpret type I, type II, and type III ANOVA and MANOVA?
Now, the model is Y ~ Grade + Nim + Tim and you are investigating the difference between the type I and type II sums of squares for the predictor in the middle, Nim. With respect to Nim, the type I ANOVA means to adjust for Grade first and then compute the sum of squares of Nim. And the type II ANOVA means to adjust for both Grade and Tim and then compute the sum of squares for Nim.
Let's illustrate the difference between adjusting for Grade alone and adjusting for both Grade and Tim. Since there are no interactions in the model, "adjusting" is equivalent to fitting a regression and taking the residuals. The left panel shows the raw measurements; the middle panel corresponds to Y ~ Grade; the right panel corresponds to Y ~ Grade + Tim. The horizontal bars indicate the two Nim group means (after adjustment). In the first the two panels the bars are at the same height so Nim is not significant; on the right the bar at Nim = 1 is (significantly) higher than the bar at Nim = 0.

Now the interpretation is clearer: Tim has a stronger effect than either Grade or Nim. For example, Y is higher at Tim = 1 vs at Tim = 0, and Y is higher at Grade = 0 vs Grade 1. But the Grade effect is detected only if Tim is in the model: a strong effect explains a lot of the variability in Y, which reduces the residual error and makes it possible to detect smaller effects. Notice how the spread of the y-axis shrinks from left to right as we include more informative predictors.
To sum up, the type I ANOVA is not especially insightful; focus on understanding & reporting the type II AVOVA results. Plots always help with the interpretation. And finally, the design is not balanced: there are no measurements with Tim = 1 and Nim = 1. If feasible, you should "complete" the experiment; this will improve the analysis and make it possible to check for interactions between Nim and Tim.
