Expectation with an operator notation $E()$ (varying preferences on good fonts, roman or italic, plain or fancy, are found) does imply taking the mean of its argument, but in a mathematical or theoretical context. The term goes back to Christiaan Huygens in the 17th century. The idea is explicit in much of probability theory and mathematical statistics and, for example, Peter Whittle's book Probability via expectation makes clear how it could be made even more central.
It is basically just a matter of convention that means (averages) are also often expressed rather differently, notably by single symbols, and especially when those means are to be calculated from data. However, Whittle in the book just cited uses a notation $A()$ for averaging and angle brackets around variables or expressions to be averaged are common in physical science.
See e.g. https://en.wikipedia.org/wiki/Peter_Whittle_(mathematician) on Whittle (1927-2021). The book in question went through two titles, three publishers and four English editions between 1970 and 2000, although the second edition is really a reprint. Although not elementary, it includes many different examples and emphases and sufficient wry comments to make it interesting and useful even to readers without a strong mathematical background (such as myself).