# How does one interpret histograms given by TensorFlow in TensorBoard?

I recently was running and learning tensor flow and got a few histograms that I did not know how to interpret. Usually I think of the height of the bars as the frequency (or relative frequency/counts). However, the fact that there are not bars as in a usual histogram and the fact that things are shaded confuses me. there also seems to be many lines/heights at once?

Does someone know how to interpret the following graphs (and maybe provide good advice that can help in general to reading histograms in tensorflow):

maybe some other things that are interesting to discuss is, if the original variables were vectors or matrices or tensors, then what is tensorflow showing in fact, like a histogram for each coordinate? Also, maybe referencing how to get this information to make people self sufficient would be nice because I've had some difficultly finding useful things on the docs right now. Maybe some tutorials example etc? Maybe some advice on manipulating them would be nice too.

As a reference, here an extract of the code that gave this:

(X_train, Y_train, X_cv, Y_cv, X_test, Y_test) = data_lib.get_data_from_file(file_name='./f_1d_cos_no_noise_data.npz')
(N_train,D) = X_train.shape
D1 = 24
(N_test,D_out) = Y_test.shape
W1 = tf.Variable( tf.truncated_normal([D,D1], mean=0.0, stddev=std), name='W1') # (D x D1)
S1 = tf.Variable( tf.constant(100.0, shape=[]), name='S1') # (1 x 1)
C1 = tf.Variable( tf.truncated_normal([D1,1], mean=0.0, stddev=0.1), name='C1' ) # (D1 x 1)
W1_hist = tf.histogram_summary("W1", W1)
S1_scalar_summary = tf.scalar_summary("S1", S1)
C1_hist = tf.histogram_summary("C1", C1)

• Whatever these plots are, they definitely are not histograms! By definition, a histogram depicts probability by means of areas.
– whuber
Jun 24, 2016 at 15:37
• The point is that by referring to them as "histograms" you mislead yourself, you risk misleading your readers, and you lose opportunities to research what is going on, because you will use the wrong keywords in your searches. The first thing you ought to do is consult your documentation to find out what it calls these plots.
– whuber
Jun 24, 2016 at 15:39
• @whuber I am not calling them histograms, they are calling themselves histograms! This is one of the commands that I used to collect that information W1_hist = tf.histogram_summary("W1", W1). It says histogram, what else am I suppose to call it? I don't know why they'd call it histogram when its something else. Jun 24, 2016 at 15:40
• I suppose a software developer can name her functions anything she likes. Regardless of the function's name, though, these simply are not histograms in any form. We can hope that the documentation uses recognizable, conventional names or--at a minimum--describes how these plots are constructed.
– whuber
Jun 24, 2016 at 15:45
• @Pinocchio, two minutes of googling brought me to github.com/tensorflow/tensorflow/blob/master/tensorflow/… where you can scroll down to read about "histograms". Have you already seen this documentation? Jul 1, 2016 at 11:47

Currently the name "histogram" is a misnomer. You can find evidence of that in the README. The meaning of the histogram interface might change some day as they said there. However, this is what it currently means.

The graphs in your question mix different runs of TensorFlow. Instead, look at the following graphs that display only one run:

First what I'd like to say is that the curves themselves represent percentiles. I will borrow the picture from here:

which means that the curve labeled 93% is the 93rd percentile, meaning that 93% of the observations were below the value ~0.130 at the time step 1.00k. So the graph gives 3 things of information, the percentage of observations bellow a certain value according to some think curve at every time step of the computation of the Neural network training (at least in this case its what the steps mean). This gives you a feel of the distribution of values of your network.

There are also the minimum and maximum values to get a sense of the range of values during training.

So the y-axis tells you the value you are interested and the curve tells you the percentile and the x axis at the step. So if you have:

$$(x,f_i(x) = y)$$

it means that at the step x according to the percentile curve $i$ there are less than $i$% of the values bellow the value y.

• but just looking at the graph without annotations, how would you guess the numbers 93%, 84%, 69%, etc...? Aug 28, 2023 at 7:56
• Okay, if $X\sim\mathcal{N}(0,1)$, then these are the values $\mathbb{P}(X\le -1.5)=0.07, \mathbb{P}(X\le -1)=0.16, \mathbb{P}(X\le-0.5)=0.31, \mathbb{P}(X\le 0)=0.5, \mathbb{P}(X\le 0.5)=0.69, \mathbb{P}(X\le 1)=0.84, \mathbb{P}(X\le 1.5)=0.93,...$, Then if $\Phi$ denotes the partition function of the standard gaussian, then they are the values $1-\Phi(1.5)$, $1-\Phi(1)$, $1-\Phi(0.5)$, $\Phi(0)$, $\Phi(0.5)$, $\Phi(1)$, $\Phi(1.5)$ which we can read in a gaussian table Aug 28, 2023 at 9:02