Why is dimensionality reduction used if it almost always reduces the explained variation? Let's say I have $N$ covariates in my regression model, and they explain 95% of the variation of the target set, i.e. $r^2=0.95$. If there are multicollinearity between these covariates and PCA is performed to reduce the dimensionality. If the principal components explains, say 80% of the variation (as opposed to 95%), then I have incurred some loss in the accuracy of my model.
Effectively, if PCA solves the issue of multicollinearity at the cost of accuracy, is there any benefit to it, other than the fact it can speed up model training and can reduce collinear covariates into statistically independent and robust variables?
 A: Data reduction (unsupervised learning) is not always used because of any hope of wonderful performance, but rather out of necessity.  When one has the "too many variables too few observations" problem, the primarily alternatives are penalized maximum likelihood estimation (ridge regression, lasso, elastic net, etc.) or data reduction.  Data reduction, which as a side benefit deals well with collinearity, can be more interpretable, and works in any predictive context.  Data reduction is IMHO much preferred over variable selection, because in the majority of problems variable selection yields a result that is too random/unstable.  The spirit of data reduction is this: Estimate the model complexity that can be supported by your available sample size.  Reduce the dimensionality (in a way that is completely masked to Y) and fit a single model whose number of parameters (that are estimated against Y) is supported by the effective sample size.
When using variable clustering or sparse principal components, one represents groups of variables with scores.  Sometimes an entire group can be dropped.  This procedure is not distorted by collinearities.
A: Your question is implicitly assuming that reducing explained variation is necessarily a bad thing. Recall that $R^2$ is defined as:
$$
R^2 = 1 - \frac{SS_{res}}{SS_{tot}}
$$
where $SS_{res} = \sum_{i}{(y_i - \hat{y})^2}$ is a residual sum of squares and $SS_{tot} = \sum_{i}{(y_i - \bar{y})^2}$ is a total sum of squares. You can easily get an $R^2 = 1$ (i.e. $SS_{res} = 0$) by fitting a line that passes through all of the (training) points (though this, in general, requires more flexible model as opposed to a simple linear regression, as noted by Eric), which is a perfect example of overfitting. So reducing explained variation isn't necessarily bad as it could result in a better performance on unseen (test) data. PCA can be a good preprocessing technique if there are reasons to believe that the dataset has an intrinsic lower-dimensional structure.
A: Take a simple example of computing seasonal adjustment factor for months across a set of years for a company's sales. Assume there is no linear trend except if years are associated with an inflationary period. Note: In reality, one would work a log transform of the data which assumes a constant percent change relationship across time.
Collapsing the month data across years produces good results by month if inflationary periods are rare. If you happen to guess that the year seasonality is in a non-inflationary period, you have the best estimates with the best error estimates. So, the dimensionality reduction (ignoring years) is clearly best.
However, if it turns out that you are in an inflationary periods, not so good monthly seasonal adjustment. However, a year model may capture the inflation trend and produce better results.
So which model to use, collapsed or full?
One approach is to estimate the probability that you could be an inflationary period based on history,
Next, what is the operational cost associated with having an average error of X in a months' seasonality.
Knowing the difference in cost by month due to modeling error for collapsed vs full for inflationary versus non-inflationary, and the associated probability of each case, one can make a decision that produces the lowest expected cost.
This assumes that this exercise repeats over time and starting parameters estimate are good estimates.
So, the specific answer relates to the nature of the data, model specification/estimation precision and associated knowledge relating to error cost estimates.
A: Let me give a quick chime in. I'm a data analyst for DNA methylation data. I have an awesome dataset that is about 1,000 people x methylation measure in over 3 million locations each x 3 points in time. That's nearly 10 billion data points.
If I want to analyze this data set... well, let's say some processes can take several days to weeks to run.
Alternatively, I can do a data reduction with PCA/UMAP and work with a much much smaller dataset that will give me fairly accurate and generalizable results within minutes. So even if my explained variation is lower, it can really make sense.
Something else that is worth considering is what are your results being used for? If my output is being used to make an important life-or-death decision then I really want to minimize any and all errors. If my output is going to inform further future research then I have a larger margin of error to work with.
Just a quick example, let's say my analysis yields the top 20 drugs that could be effective to treat a certain cancer. If this is going to a patient, I want drugs 1-3 to represent the best, 2nd best, and 3rd best treatment options respectively. Here the result has to be super precise, with very little error.
If, however, the goal is to try these out in cell cultures, then I don't really care if the top result happens to be the 3rd best or the 1st one. As long as my top 25% contains drugs that are more likely to work I don't really care about the order. (A lot of assumptions in this statement but just to exemplify).
To summarize, dimensionality reduction can also help with computation time for big data analysis. This is highly dependant on what the output will be used for.
A: In your question there is an implicit assumption about the regressor being linear.
In case it is linear your assertion is correct.
But for the case of non linear regressor you may think on the dimensionality reduction step as a feature extraction.
In that case it has a very important role in order to get good results.
It might reduce the noise, it might assist with the learning, etc...
A: 
If the principal components explains, say 80% of the variation (as opposed to 95%), then I have incurred some loss in the accuracy of my model.

Performing PCA does not reduce the accuracy of the model. The principal components, when you use all of them, should also explain the 95%. It is the reduction of the dimensionality which reduces the explained variation.
So this is a matter of model selection and finding models with fewer parameters. The role of PCA is to do this model selection by redefining the parameter space in order to find a small number of components that explain a large amount of variation.
