EACH GROUP
The classical assumption is about an independently and identically distributed error term. If you have residuals (which estimate the error terms) that, pooled together, look normal, yet the individual groups do not look normal, then you have evidence of a violation of this classical assumption that leads to the usual p-values and confidence intervals. That is, if the individual groups do not have residuals that look similar, then something is happening with the independence or identical distribution aspects of the assumption.
When there is a complicated regression model, especially with continuous predictor variables, we do not necessarily have the luxury of being able to drill down to the residuals at every level of feature combinations, as there is often only onceone residual for a given combination of feature values. However, we do get that luxury when there are groups of residuals with the same feature values (e.g., 84 residuals for the group of dogs, 71 residuals for the group of cats, 58 residuals for the group of koalas).