I don't get how it follows that we should say "factor loads variable"
rather than vice-versa
Abstract explanation. If a point seen as object has a coordinate on an axis seen as feature then the coordinate is how much the feature loads the point, how much it charges, by itself, that point. If my height is 1.86 m then this is how I'm loaded by height (not how much height is loaded by me). Note that loading is variable's coordinate on factor-as-axis on the loading plot.
Latent-trait explanation. Factor is conceptualized as an entity which plays "in" the variables or "behind" them and which makes them correlate. Therefore "load" is intuitively a good word to express the degree how strongly the variable is dependent on, driven by, the latent factor. Factor analysis model is regressional model whereby factors explain or "influence" observed variables. Any regression coefficient (not only factor analytic) may be labeled a "loading": regressional coefficient = regressional weight = regressional loading. More reason to call a factor's coefficient "loading" comes from the fact that in the factor model, factors $F$s are set standardized, each unit-variance, while a variable $V$ isn't necessarily standardized. There comes therefore that the effect on $V$ is realized/expressed completely and only via the loading coefficients. Whenever in regressional model a standardized variable predicts a potentially unstandardized one - call the coefficient "loading".
Why we need the term "loading" at all, when we already had the term
We actually don't need. Word "loading" is simply a tradition stemming from psychologists' liking for figurative sense (FA started to develop a century ago among psychologists). Moreover, the term "loading" may have somewhat different statistical meaning in other related multivariate methods (such as discriminant analysis). In general, some people in some cases call "loadings" regression coefficients, while other or in other cases - correlation coefficients. So the term is confusing. It is not a statistical term, ultimately.
If you don't like the word, don't use it. You may also say "variable loads (on) factor" if you want; to me, it is simply a thoughtless speech, not a vice.
P.S. I've just looked in an English dictionary (English isn't my language) and observed that to load may have meanings as (1) "I loaded the cart" (by a bag, or by myself as embarked); (2) "the ship loads (up) many passengers (on it)". If to follow the second word usage, it would be quite OK to say "the variable loads the factor (on itself, the variable) well".