Benefit in getting Orthogonal dimensions from a data set What are the benefits of getting a new data set in which all their dimensions are independent from an original data set (applying PCA)?
we know we can reduce the storage space needed, speed up computation or remove redundant features after doing PCA for example.
but What computations in a statistical or probabilistic point of view are expedited or benefited  when all dimensions are orthogonal?
 A: It's not just the orthogonality, but the ordering from large to small eigenvalues that's useful, because it aids data reduction.
For example, principal components regression is a technique for dealing with when the variances of regression coefficients become highly inflated owing to collinearity of the predictors. You do PCA on the predictors, regress the response on the principal components, then transform the estimates back to the original set of predictors. If you use all the PCs, you get the same results as you would by regressing directly on the original predictors. If you use a subset of the largest you get a different result - you've introduced some bias, but hopefully reduced the variance of the coefficient estimates.
Outlier detection is another use of PCA: observations that aren't extreme in the original variables can be extreme in the smaller principal components because they go against the correlation structure of the other variables.  Such observations won't necessarily stick out even in scatterplots of all pairs of the original variables.
