Yes, in principle you should use Bout ID nested within Monkey ID as a random effect. If you don't, then you are effectively assuming that the bout-to-bout variation in call pitch within monkeys is entirely explained by variation in stress hormones. This is an optimistic/potentially anti-conservative assumption. The "masking" of the fixed effect of stress would occur because you can't really infer that stress is causing pitch variation; it could be explained by other, unmeasured/uncontrolled variation among bouts. On the other hand, I would expect that if there really is a strong stress effect, it wouldn't be entirely wiped out by the presence of the random effect. (You could do some simulations to test this.)
Just to add some complication, I think the maximal model for this observational design would be
pitch ~ 1 + hormone + (1 + hormone | MonkeyID) + (1 | MonkeyID:BoutID)
— because (as I understand it) it's plausible that the effect of hormone on pitch varies among monkeys, and because you have measured multiple hormone levels for each monkey, you can in principle quantify this variation. This gets you into the usual mess of whether in practice you have sufficient data to quantify it ... see contrasting views between Barr et al. 2013 vs Bates, Kliegl, Matuschek et al ... (or go Bayesian or ...) (see the relevant section of the GLMM FAQ for more discussion)
With some apologies for the brevity of explanation: here is code that simulates something like your situation, which you could use for informal exploration or power analysis. One thing I realized while setting it up is that you can simplify the analysis slightly by collapsing the data to the bout level (i.e., averaging pitches within bout). With weights proportional to the number of calls per bout (i.e. variance inversely proportional to number of calls), this analysis is equivalent (this is a variation of the argument made by Murtaugh ) to the full, three-level (monkey/bout/call) analysis - it also clarifies that your effective sample size is equivalent to the number of bouts, not the number of calls (although having many calls per bout will reduce the among-bout variation and improve power ...) In one example I tried (with a larger among-monkey variation in hormone-pitch relationship), using the maximal model greatly improved resolution/power ...
nb <- c(1, 1, 1, 1, 2, 2, 3, 4) ## bouts per monkey
b_tot <- sum(nb) ## total bouts
nc <- round(170/b_tot) ## 170 calls total (approx)
dd <- data.frame(MonkeyID=rep(1:length(nb),nb*nc),
## hormone varies by bout
## (could vary by monkey/bout)
dd$hormone <- rnorm(b_tot,mean=5)[dd$BoutID]
fullform <- pitch ~ 1 + hormone + (1 + hormone | MonkeyID) + (1|MonkeyID:BoutID)
subform <- pitch ~ 1 + hormone + (1 | MonkeyID/BoutID)
dd$pitch <- simulate(fullform[-2],
## theta: scaled var-cov
## pos 1: among bouts
## pos 2:4: among monkeys
sigma=1 ## residual std dev
fit0 <- lmer(fullform, data=dd) ## singular fit;
## helps figure out order of theta parameters
fit <- lmer(subform, data=dd)
dd0 <- dd %>% group_by(MonkeyID,BoutID) %>%
summary(lme(pitch~hormone, random = ~1|MonkeyID,
summary(lme(pitch~hormone, random = ~hormone|MonkeyID,
Barr, Dale J., Roger Levy, Christoph Scheepers, and Harry J. Tily. “Random Effects Structure for Confirmatory Hypothesis Testing: Keep It Maximal.” Journal of Memory and Language 68, no. 3 (April 2013): 255–78. https://doi.org/10.1016/j.jml.2012.11.001.
Bates, Douglas, Reinhold Kliegl, Shravan Vasishth, and Harald Baayen. “Parsimonious Mixed Models.” ArXiv:1506.04967 [Stat], June 16, 2015. http://arxiv.org/abs/1506.04967.
Matuschek, Hannes, Reinhold Kliegl, Shravan Vasishth, Harald Baayen, and Douglas Bates. “Balancing Type I Error and Power in Linear Mixed Models.” ArXiv:1511.01864 [Stat], November 5, 2015. http://arxiv.org/abs/1511.01864.
Murtaugh, Paul A. “Simplicity and Complexity in Ecological Data Analysis.” Ecology 88, no. 1 (2007): 56–62.