I'm working through Andrew Ng's original Stanford course and ran into some numpy confusion. Basically, my main question is, if we dot product a 1D array with a 2D array in numpy (and the dimensions are not appropriate for matrix multiplication) does numpy automatically "transpose" the 1D array appropriately?

In other words, two examples below where I think I see this behavior:



X dimensions: 97 x 2

theta dimensions: (2,)

h-y dimensions: (97,)


Therefore, when I want to perform matrix multiplication between X and theta, Can i safely assume that theta is effectively treated as 2x1 during the matrix multiplication operation? Therefore I end up with 97x2 matrix dot 2x1 matrix which is acceptable?


A slightly different question applies to the (h-y)@X reference. Here we have dimension (97,) dot dimension 97x2. If h-y is indeed treated like a 97x1 matrix, I would expect a transpose to be required before matrix multiplication is allowed. However, this is not the case.


Do I understand correctly that numpy is effectively doing this through broadcasting? Numpy see's the need for 1x97 matrix instead of a 97x1 matrix and forces the 1D array to conform? Does h-y become 1x97?

And the big take-home question. This this mean, whenever performing matrix multiplication between a multi-dimensional matrix and a 1D array, don't ever worry about transposing the 1D array because numpy will handle this for you if needed.

Thanks in advance!

import numpy as np
import os

# Read comma separated data (97 rows, 2 columns)
data = np.loadtxt('ex1data1.txt', delimiter=',')
X, y = data[:, 0], data[:, 1]

# Number of training examples
m = y.size  

# Add column of 1's to X.  X size becomes (97,2)
X = np.stack([np.ones(m), X], axis=1)

#Initialize first theta.  Theta size is (2,) or (1,2) if broadcast required
theta = np.array([0,0])
num_iters = 1500
alpha = 0.01

# Find best theta
for i in range(num_iters):
        # X = 97x2 matrix 
        # theta = (2, ) 1D array which, in this case, is treated similar to 2x1 matrix, allowing a dot product
        h = X@theta
        # h-y = (97,) or (1,97) if broadcast requried.  Treated similar to 1x97 matrix, allowing a dot product
        # X = 97x2 matrix
        theta = theta - alpha*(1/m)*(h-y)@X