Why are regression coefficients in a factor analysis model called "loadings"? In this thread @ttnphns writes that

Because it is regression coefficients [...] I insist that it is better
  to say "factor loads variable" than "variable loads factor".

I learned from here that a factor analysis model is the system of equations 

$V_1 = a_{1I}F_I + a_{1II}F_{II} + E_1$
$V_2 = a_{2I}F_I + a_{2II}F_{II} + E_2$
$...$
$V_p = …$
where coefficient a is a loading, F is a factor [...], and
  variable E is regression residuals.

However, I don't get how it follows that we should say "factor loads variable" rather than vice-versa. What is it about the term "loading" that makes it follow?
I also don't know why we need the term "loading" at all, when we already had the term "regression coefficient". Is it because sometimes the regression coefficients are also correlation coefficients, and statisticians wanted a generic term to cover both cases?
I'm hoping that the answer to this question will make it easier for students to remember that factors load observed variables, rather than the other way around.
 A: 
I don't get how it follows that we should say "factor loads variable"
  rather than vice-versa

Abstract explanation. If a point seen as object has a coordinate on an axis seen as feature then the coordinate is how much the feature loads the point, how much it charges, by itself, that point. If my height is 1.86 m then this is how I'm loaded by height (not how much height is loaded by me). Note that loading is variable's coordinate on factor-as-axis on the loading plot.
Latent-trait explanation. Factor is conceptualized as an entity which plays "in" the variables or "behind" them and which makes them correlate. Therefore "load" is intuitively a good word to express the degree how strongly the variable is dependent on, driven by, the latent factor. Factor analysis model is regressional model whereby factors explain or "influence" observed variables. Any regression coefficient (not only factor analytic) may be labeled a "loading": regressional coefficient = regressional weight = regressional loading. More reason to call a factor's coefficient "loading" comes from the fact that in the factor model, factors $F$s are set standardized, each unit-variance, while a variable $V$ isn't necessarily standardized. There comes therefore that the effect on $V$ is realized/expressed completely and only via the loading coefficients. Whenever in regressional model a standardized variable predicts a potentially unstandardized one - call the coefficient "loading".

Why we need the term "loading" at all, when we already had the term
  "regression coefficient"

We actually don't need. Word "loading" is simply a tradition stemming from psychologists' liking for figurative sense (FA started to develop a century ago among psychologists). Moreover, the term "loading" may have somewhat different statistical meaning in other related multivariate methods (such as discriminant analysis). In general, some people in some cases call "loadings" regression coefficients, while other or in other cases - correlation coefficients. So the term is confusing. It is not a statistical term, ultimately.
If you don't like the word, don't use it. You may also say "variable loads (on) factor" if you want; to me, it is simply a thoughtless speech, not a vice.
P.S. I've just looked in an English dictionary (English isn't my language) and observed that to load may have meanings as (1) "I loaded the cart" (by a bag, or by myself as embarked); (2) "the ship loads (up) many passengers (on it)". If to follow the second word usage, it would be quite OK to say "the variable loads the factor (on itself, the variable) well".
