I'm working through Andrew Ng's original Stanford course and ran into some numpy confusion. Basically, my main question is, if we dot product a 1D array with a 2D array in numpy (and the dimensions are not appropriate for matrix multiplication) does numpy automatically "transpose" the 1D array appropriately?
In other words, two examples below where I think I see this behavior:
X dimensions: 97 x 2
theta dimensions: (2,)
h-y dimensions: (97,)
Therefore, when I want to perform matrix multiplication between X and theta, Can i safely assume that theta is effectively treated as 2x1 during the matrix multiplication operation? Therefore I end up with 97x2 matrix dot 2x1 matrix which is acceptable?
A slightly different question applies to the (h-y)@X reference. Here we have dimension (97,) dot dimension 97x2. If h-y is indeed treated like a 97x1 matrix, I would expect a transpose to be required before matrix multiplication is allowed. However, this is not the case.
Do I understand correctly that numpy is effectively doing this through broadcasting? Numpy see's the need for 1x97 matrix instead of a 97x1 matrix and forces the 1D array to conform? Does h-y become 1x97?
And the big take-home question. This this mean, whenever performing matrix multiplication between a multi-dimensional matrix and a 1D array, don't ever worry about transposing the 1D array because numpy will handle this for you if needed.
Thanks in advance!
import numpy as np import os # Read comma separated data (97 rows, 2 columns) data = np.loadtxt('ex1data1.txt', delimiter=',') X, y = data[:, 0], data[:, 1] # Number of training examples m = y.size # Add column of 1's to X. X size becomes (97,2) X = np.stack([np.ones(m), X], axis=1) #Initialize first theta. Theta size is (2,) or (1,2) if broadcast required theta = np.array([0,0]) num_iters = 1500 alpha = 0.01 # Find best theta for i in range(num_iters): # X = 97x2 matrix # theta = (2, ) 1D array which, in this case, is treated similar to 2x1 matrix, allowing a dot product h = X@theta # h-y = (97,) or (1,97) if broadcast requried. Treated similar to 1x97 matrix, allowing a dot product # X = 97x2 matrix theta = theta - alpha*(1/m)*(h-y)@X ```