# Ways to reduce high dimensional data for visualization

I'm working on a 2D physical simulation and I am collecting data in time at several points. These discrete points are along vertical lines, with multiple lines in the axial direction. This makes the dataset effectively 4D.

For instance, let's assume I have collection points at (X,Y) coordinates of:

• (0,0), (1,0), (2,0)
• (0,1), (1,1), (2,1)
• (0,2), (1,2), (2,2)

and at each point I am collecting $\{P,T,U,V\}$ where $P$ is pressure, $T$ is temperature, $U,V$ are the X- and Y-components of velocity. At each iteration of the simulation, these variables are stored for all 9 collection points. So all my data is continuous in time at each discrete point in space.

For example, the data for a single point would look like:

I am interested in showing, say, Pressure at all points for all times to show vertical and axial waves. If I were to do this along a single line (either vertical or axial), I could use a waterfall plot with axes (Y, time, Pressure). But if I have 3 vertical lines and 3 axial lines, this would be 6 waterfall plots to get a complete picture of the wave motion in both directions. The spatial coordinates are discrete variables while the field (in this case Pressure) and time are continuous.

In the above figures for example, the large pressure peak at $t\approx0.000125$ could be traveling in the X or Y direction.

Is there a method to show them all at once? Usually color can be added to make a "fourth" dimension visible, but is there another possible approach? I plan on plotting it as many ways as possible to see if anything reveals information others don't, so please pitch any ideas.

What if the simulation were 3D and I had a 5D resulting dataset? Does that change the possible visualization methods?

-
Are all the dimensions discrete, or are some continuous? If so, which ones are which? – naught101 Oct 28 '12 at 2:20
The (X,Y) are discrete while (P, time) are continuous. – tpg2114 Oct 28 '12 at 13:57
I recommend considering faceting as an alternative (or complement) to 3-d – Michael Bishop Oct 28 '12 at 20:50
Does it need to be a static plot that can be printed? If not, you could show your data as a time-lapsed series of plots. If I remember correctly, JMP software does that sort of thing. – Emil Friedman Oct 30 '12 at 18:16
@naught101 Updated acoordingly. – tpg2114 Nov 9 '12 at 2:10

I had some seven-dimensional data myself. Although I finally settled for a small selection of 3-dimensional slice-throughs, one option is the Parallel Coordinates Plot. This works for an arbitrary number of dimensions! From Wikipedia:

Parallel coordinates is a common way of visualizing high-dimensional geometry and analyzing multivariate data.

To show a set of points in an n-dimensional space, a backdrop is drawn consisting of n parallel lines, typically vertical and equally spaced. A point in n-dimensional space is represented as a polyline with vertices on the parallel axes; the position of the vertex on the ith axis corresponds to the ith coordinate of the point.

-
 That's a great plot. Excellent use of colour. The legend would be better on the side, and re-ordered to match the colours on the last axis, but it's not vital. – naught101 Oct 28 '12 at 2:22 @naught101 It's from Wikipedia, feel free to send an improved one there ;-) – gerrit Oct 28 '12 at 9:27

Principal Component Analysis is generally a good choice for dimension reduction in most cases, I am not sure it will suit for your particular problem, but it will find the orthogonal dimensions along which most variation of data samples are captured. If you develop in R, you can use prcomp() to simply convert your original matrix of data points to the PCA form.

-

Here are a couple of ways of portraying 3-D data with ggplot2. You can combine approaches (facet grids, colors, shapes, etc.) to increase the dimensionality of your graphic.

doInstall <- TRUE  # Change to FALSE if you don't want packages installed.
toInstall <- c("ggplot2")
if(doInstall){install.packages(toInstall, repos = "http://cran.r-project.org")}
lapply(toInstall, library, character.only = TRUE)

# Air passenger data. ts converted to long matrix:
myData <- data.frame(Year = c(floor(time(AirPassengers) + .01)),
Month = c(cycle(AirPassengers)),
Value = c(AirPassengers))
# Easy conversion code from: http://stackoverflow.com/a/4973859/479554

# Convert month numbers to names, using a built-in constant:
myData$Month <- factor(myData$Month)
levels(myData$Month) <- month.abb # One possibility: zp1 <- ggplot(myData, aes(x = Year, y = Value, colour = Month)) zp1 <- zp1 + geom_line() print(zp1) # This is fine, if you can differentiate between the colors # Another possibility: zp2 <- ggplot(myData, aes(x = Year, y = Value)) zp2 <- zp2 + geom_line() zp2 <- zp2 + facet_wrap(~ Month) print(zp2) # This is fine, but it's hard to compare across facets # A third possibility; plotting reference lines across each facet: referenceLines <- myData # \/ Rename colnames(referenceLines)[2] <- "groupVar" zp3 <- ggplot(myData, aes(x = Year, y = Value)) zp3 <- zp3 + geom_line(data = referenceLines, # Plotting the "underlayer" aes(x = Year, y = Value, group = groupVar), colour = "GRAY", alpha = 1/2, size = 1/2) zp3 <- zp3 + geom_line(size = 1) # Drawing the "overlayer" zp3 <- zp3 + facet_wrap(~ Month) zp3 <- zp3 + theme_bw() print(zp3)  -  That's the same plot, repeated 12 times, but with different lines highlighted, right? Definitely an interesting way of looking at that data! Another way would be to plot only the original monthly timeseries, and then facet by month, and plot the month points on top of it. Same idea, but with the "real" timeseries in there. – naught101 Nov 9 '12 at 2:08 Like this: APdf <- data.frame(Time=c(time(AirPassengers)), Year=c(floor(time(AirPassengers))), Month=c(cycle(AirPassengers)), Value=c(AirPassengers)) ; APdf$Month <- month.abb[APdf$Month] ; ggplot(APdf, aes(x=Time, y=Value)) + facet_wrap(facets='Month') + geom_line(data=APdf[,c(1,4)], colour='gray') + geom_point(). God damn, I love ggplot2. – naught101 Nov 9 '12 at 2:26 Pairs plots: This is not a method of dimensionality reduction, but it is a really good way to get a quick overview of where some meaningful relationships might lie. In R, the base package contains the pairs() function, which is good for continuous data (it converts everything to continuous). A better function is ggpairs(), from the GGally package: library(GGally) ggpairs(iris, colour='Species')  - For the two-dimensional problem, I wonder if you could plot a map of your trace points with some symbol at the (x,y) coordinates. Then this symbol would either change color or oscillate around its fixed position (corresponding to$p=p_{mean}\$ for example). I can see both being relatively easy to do in matplotlib. The one where the symbol oscillate makes me think of a raster plot like this one:

This plot shows the velocity profiles at different axial locations, giving you a 2D map of the flowfield. The vertical lines represent 0 velocity. The regions without dots are not part of the computational domain. Of course this is not easily extensible to 3D data...

-
 What is the white square for? – naught101 Nov 9 '12 at 2:28 This is a flow field representation. It's a flow around a corner, the profiles represent the velocity at different axial locations... – FrenchKheldar Nov 9 '12 at 15:53 Ok. It would make sense to add some description to the answer. The plot is pretty opaque by itself... – naught101 Nov 10 '12 at 8:15