23
$\begingroup$

I recently was running and learning tensor flow and got a few histograms that I did not know how to interpret. Usually I think of the height of the bars as the frequency (or relative frequency/counts). However, the fact that there are not bars as in a usual histogram and the fact that things are shaded confuses me. there also seems to be many lines/heights at once?

Does someone know how to interpret the following graphs (and maybe provide good advice that can help in general to reading histograms in tensorflow):

enter image description here

maybe some other things that are interesting to discuss is, if the original variables were vectors or matrices or tensors, then what is tensorflow showing in fact, like a histogram for each coordinate? Also, maybe referencing how to get this information to make people self sufficient would be nice because I've had some difficultly finding useful things on the docs right now. Maybe some tutorials example etc? Maybe some advice on manipulating them would be nice too.


As a reference, here an extract of the code that gave this:

(X_train, Y_train, X_cv, Y_cv, X_test, Y_test) = data_lib.get_data_from_file(file_name='./f_1d_cos_no_noise_data.npz')
(N_train,D) = X_train.shape
D1 = 24
(N_test,D_out) = Y_test.shape
W1 = tf.Variable( tf.truncated_normal([D,D1], mean=0.0, stddev=std), name='W1') # (D x D1)
S1 = tf.Variable( tf.constant(100.0, shape=[]), name='S1') # (1 x 1)
C1 = tf.Variable( tf.truncated_normal([D1,1], mean=0.0, stddev=0.1), name='C1' ) # (D1 x 1)
W1_hist = tf.histogram_summary("W1", W1)
S1_scalar_summary = tf.scalar_summary("S1", S1)
C1_hist = tf.histogram_summary("C1", C1)
$\endgroup$
  • 2
    $\begingroup$ Whatever these plots are, they definitely are not histograms! By definition, a histogram depicts probability by means of areas. $\endgroup$ – whuber Jun 24 '16 at 15:37
  • 1
    $\begingroup$ The point is that by referring to them as "histograms" you mislead yourself, you risk misleading your readers, and you lose opportunities to research what is going on, because you will use the wrong keywords in your searches. The first thing you ought to do is consult your documentation to find out what it calls these plots. $\endgroup$ – whuber Jun 24 '16 at 15:39
  • 1
    $\begingroup$ @whuber I am not calling them histograms, they are calling themselves histograms! This is one of the commands that I used to collect that information W1_hist = tf.histogram_summary("W1", W1). It says histogram, what else am I suppose to call it? I don't know why they'd call it histogram when its something else. $\endgroup$ – Pinocchio Jun 24 '16 at 15:40
  • 1
    $\begingroup$ I suppose a software developer can name her functions anything she likes. Regardless of the function's name, though, these simply are not histograms in any form. We can hope that the documentation uses recognizable, conventional names or--at a minimum--describes how these plots are constructed. $\endgroup$ – whuber Jun 24 '16 at 15:45
  • 2
    $\begingroup$ @Pinocchio, two minutes of googling brought me to github.com/tensorflow/tensorflow/blob/master/tensorflow/… where you can scroll down to read about "histograms". Have you already seen this documentation? $\endgroup$ – amoeba says Reinstate Monica Jul 1 '16 at 11:47
21
$\begingroup$

Currently the name "histogram" is a misnomer. You can find evidence of that in the README. The meaning of the histogram interface might change some day as they said there. However, this is what it currently means.

The graphs in your question mix different runs of TensorFlow. Instead, look at the following graphs that display only one run:

enter image description here

First what I'd like to say is that the curves themselves represent percentiles. I will borrow the picture from here:

enter image description here

which means that the curve labeled 93% is the 93rd percentile, meaning that 93% of the observations were below the value ~0.130 at the time step 1.00k. So the graph gives 3 things of information, the percentage of observations bellow a certain value according to some think curve at every time step of the computation of the Neural network training (at least in this case its what the steps mean). This gives you a feel of the distribution of values of your network.

There are also the minimum and maximum values to get a sense of the range of values during training.

So the y-axis tells you the value you are interested and the curve tells you the percentile and the x axis at the step. So if you have:

$$ (x,f_i(x) = y)$$

it means that at the step x according to the percentile curve $i$ there are less than $i$% of the values bellow the value y.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.