Can it be said, that in case of non-proportional hazards, Cox returns average hazard ratio over the entire period? I have read two opposite statements about the hazard ratios obtained from the Cox procedure in presence of non-proportional hazards. One is that the HR for a certain covariate, that fails this assumption, is averaged over the entire period.
So, if initially it's 0.8 and then it grows to 2, we can say it's 1.4 in average. Which hides the fact, that the hazards were opposite and swapped.
The others say, that it cannot be simply averaged and instead other measures should be used, like the restricted mean survival time. Can you name any source resolving or giving more insights on that? Or simply explain, how to interpret Cox in case of crossing curves (resulting in swapped hazards) or varying hazards in general?
 A: There is a very large statistics literature on this topic.  In the face of non-proportional hazards (non-PH), the hazard ratio is a kind of average, and we know the weights that are used as shown in some of the references and R software appearing here.  The weighting is not as simple as what you wrote, but you have the right idea.
When there is strong non-PH such as when curves cross, it is not very fruitful to interpret Cox regression coefficients.  And switching to mean survival time brings in other problems and leaves us with a hard-to-interpret result (e.g., the mean life length for subjects who died within 3 years was 2.4) and is too dependent on the choice of time horizon.  Instead I recommend flexibly modeling what you don't know.  See Section 20.7 of this especially the Breslow et al reference.  Fit non-PH directly and get time-specific hazard ratios.  You can also get cumulative incidence at a specific time estimated from this extended Cox model.
If the hazard ratios of all the predictors converge to 1.0 as $t \rightarrow \infty$ you should switch to an accelerated failure time model instead.  The above link to the RMS course notes goes into detail about that.
