For your second question:
What I am doing wrong or not understanding?
The mean values that you directly calculate don't take into account any of the other predictors included in your models, such as gender, age_group, condition, valence, or any group other than affiliative_score_group. They don't involve the further correction that you attempt with your ...
This is documented. I suggest reading the vignette on "basics", where EMMs are described: EMMs defined
The reference grid consists of combinations of predictors. The predictions for the reference grid are each linear combinations of the regression coefficients. You can find out what these are by doing something like this:
rg <- ref_grid(model)
Because it is a balanced experiment, and you are using a model that presumes the error variance is homogeneous. Accordingly, the SE of each cell mean is $s/\sqrt n$ where $s$ is the estimated error SD and $n$ is the number of observations in each mean.
In this particular example, there are 6 cell means with equal counts, and you can get $s$ from the model: