# H2O's deep learning on R has a confusion matrix problem…?

I am testing H2O's deep learning in comparison with random forest in R. When I used (1) IRIS dataset [150 observations/samples by 4 predictors/features; the last column shows the class labels (iris types)] for training with default parameters, both shows the "correct" confusion matrix, in the sense that each class (Iris-setosa, Iris-versicolor, Iris-virginica) has the correct number of data points of 50. However, when I used (2) MNIST training dataset [60,000 observations (rows) by 784 predictors (columns); the first column shows the class labels (digits)], the deep learning shows "incorrect" confusion matrix, in the sense that each class (0, 1, ... 9) has incorrect number of data points. Each class should have (5923, 6742,..., 5949) data points. The random forest shows the correct confusion matrix. Dose anyone know if this is an expected result, or am I missing something here? The code fragment is shown below.

Any suggestions are appreciated.

(1) IRIS dataset

> irisdata
C1  C2  C3  C4          C5
1 5.1 3.5 1.4 0.2 Iris-setosa
2 4.9 3.0 1.4 0.2 Iris-setosa
3 4.7 3.2 1.3 0.2 Iris-setosa
4 4.6 3.1 1.5 0.2 Iris-setosa
5 5.0 3.6 1.4 0.2 Iris-setosa
6 5.4 3.9 1.7 0.4 Iris-setosa
....

> # Deep Learning
> model.dl <- h2o.deeplearning(
+  x = 1:4, y = 5,
+  data = irisdata
+)
>print(model.dl)
...
Confusion matrix:
Reported on iris.hex
Predicted
Actual            Iris-setosa Iris-versicolor Iris-virginica   Error
Iris-setosa              49               1              0 0.02000
Iris-versicolor           0              50              0 0.00000
Iris-virginica            0              10             40 0.20000
Totals                   49              61             40 0.07333

# Random Forest
> model.rf <- h2o.randomForest(
+  x = 1:4, y = 5,
+  data = irisdata
+)
> print(model.rf)
...
Confusion matrix:
Reported on iris.hex
Predicted
Actual            Iris-setosa Iris-versicolor Iris-virginica   Error
Iris-setosa              49               1              0 0.02000
Iris-versicolor           0              50              0 0.00000
Iris-virginica            0              10             40 0.20000
Totals                   49              61             40 0.07333


(2) MNIST dataset

> # Deep Learning
> model.dl <- h2o.deeplearning(
+   x = 2:785,
+   y = 1,
+   data = train.mnist
+ )
> model.dl
...
Confusion matrix:
Reported on Last.value.1
Predicted
Actual      0    1   2    3   4   5    6    7   8    9   Error
0      1005    0   0    0   0   0    0    0   0    0 0.00000
1         0 1118   1    0   0   0    0    0   0    2 0.00268
2         0    0 954    0   0   0    0    0   0    0 0.00000
3         0    0   1 1039   0   1    0    0   0    1 0.00288
4         0    0   0    0 981   0    1    0   0    1 0.00203
5         1    0   0    0   0 971    0    1   0    0 0.00206
6         0    0   0    0   0   0 1004    0   0    0 0.00000
7         0    1   0    0   0   0    0 1026   0    1 0.00195
8         0    0   0    1   0   0    0    0 970    0 0.00103
9         0    0   0    0   0   0    0    0   0 1063 0.00000
Totals 1006 1119 956 1040 981 972 1005 1027 970 1068 0.00128

> # Random Forest
> model.rf <- h2o.randomForest(
+   x = 2:785,
+   y = 1,
+   data = train.mnist
+ )
> model.rf
...
Classification: TRUE
Confusion matrix:
Reported on training data.        Predicted
Actual      0    1    2    3    4    5    6    7    8    9   Error
0      5826    1    9    7    3    7   26    4   32    8 0.01638
1         1 6633   33   14   17    5    5   17   14    3 0.01617
2        24   12 5699   39   30    5   25   53   58   13 0.04347
3        11    6   76 5748    5  108   12   51   77   37 0.06247
4        15   11   12    2 5602    3   41   15   19  122 0.04108
5        28    6   11   95   13 5107   50    9   70   32 0.05792
6        28    9    8    3    8   43 5793    1   25    0 0.02112
7         7   16   61   10   34    2    1 6013   23   98 0.04022
8        15   22   55   52   23   58   41    6 5506   73 0.05896
9        24    7   18   73   90   34    7   55   52 5589 0.06051
Totals 5979 6723 5982 6043 5825 5372 6001 6224 5876 5975 0.04140