I've learned ML and have been learning DL from Andrew Ng's coursera courses, and every time he talks about a linear classifier, the weights are just a 1-D vector. Even during the assignments, when we roll an image into a 1-D vector (pixels * 3), the weights would still be a 1-D vector.
I now have started O'Reilly's "Learning TensorFlow" book, and came across the first example. The weights initialization in tensorflow was a bit different.
The book says the following (Page 14):
Since we are not going to use the spatial information at this point, we will unroll our image pixels as a single long vector denoted x (Figure 2-2). Then $xw^0 = ∑x_i w^0_i$ will be the evidence for the image containing the digit 0 (and in the same way we will have $w^d$ weight vectors for each one of the other digits, d = 1, . . . , 9).
and the corresponding TensorFlow code:
data = input_data.read_data_sets(DATA_DIR, one_hot=True) x = tf.placeholder(tf.float32, [None, 784]) W = tf.Variable(tf.zeros([784, 10])) y_true = tf.placeholder(tf.float32, [None, 10]) y_pred = tf.matmul(x, W)
Why are the weights 2-D here. Are weights 2-D in softmax Linear Classifier? In the coursera course, when he taught Softmax Linear Classifier, he still says the weights are 1-D. Can anyone explain this?