Is there any value in dimensionality reduction of a data set where all variables are approximately orthogonal? Suppose I have an $N$-dimensional data set where the $N$ dimensions are roughly orthogonal (have correlation zero).
Is there any utility in terms of:


*

*Visualization

*Representation (for classifier efficiency)

*Or other criteria


to perform dimensionality reduction on the data?
 A: If all N variables are roughly orthogonal then the dimension reduction will do relatively little reducing. E.g. in R
set.seed(123)
x1 <- rnorm(100)
x2 <- rnorm(100)
x3 <- rnorm(100)
x4 <- rnorm(100)
x5 <- rnorm(100)
x6 <- rnorm(100)
x7 <- rnorm(100)
x8 <- rnorm(100)
x9 <- rnorm(100)
x10 <- rnorm(100)

df1 <- cbind(x1,x2,x3,x4,x5,x6,x7,x8,x9,x10)

pcsol <- princomp(df1)
loadings(pcsol)

In essence, "orthogonal" implies "already at its smallest size". 
A: You might try a more general non linear dimensionality reduction manifold learning method like locally linear embedding, laplacian eigenmaps or t-SNE.
It is perfectly possible for there to be a lower dimensional subspace (manifold) in your data in a way that leaves 0 correlation between the N-basis dimensions. For example a circle of points about the origin or wave form as seen here. PCA won't pick this up but other methods will.
Looking at such methods is especially interesting and common for visualization and exploratory data analysis. For use within a classifier or other model you'll need to restrict yourself to the methods that can be fit on training and applied on test which excludes lots of these methods. If this is your main interest you should also look into methods for unsupervised pretraining and (supervised) feature engineering. 
