I am watching a YouTube video on SVD, and attempting to recreate some of its examples to better understand the internal machinery of the algorithm. In one of the slides, the instructor mentions that the product of the left singular vectors and the singular values diagonal matrix, $U\Sigma$, will give the coordinates of the points in the projection axis:
I tried to replicate this using Python:
import numpy as np from scipy.linalg import svd x = np.linspace(1,20, 20) x = np.concatenate((x,x)) y = x + np.random.normal(0,1, 40) A = np.stack([x,y]).T
Very straightforward, linear relationship that should be easily encodable into one dimension:
D = 1 # define the number of desired concepts (dimensions) U, S, V = svd(A) S = np.diag(S)[:D,:D] V = V[:D,:] U = U[:,:D] #Examine the new points projected: U.dot(S).T
My output looks like this:
array([[ -1.74863878, -2.66276714, -2.98768631, -5.44595742, -8.02886504, -8.24785679, -10.35313385, -11.60196262, -12.80992842, -14.96076055, -15.28834347, -17.26752538, -18.9260174 , -21.06596511, -20.7882933 , -22.87015657, -23.95399225, -24.78878465, -27.31559524, -29.00987543, -1.28269614, -1.36805834, -4.15878884, -6.11100597, -6.34365475, -8.77205121, -10.79641343, -11.57002065, -12.20840018, -14.31109098, -14.34744527, -17.97651747, -18.40141581, -19.61488063, -21.9849372 , -22.08384878, -22.28500687, -24.91212881, -26.42387864, -27.39477121]])
I noticed that in my new projected axis, all of my values are negative. In the example in the YouTube video, it seems like the users with strong sci-fi affinity (the first concept or dimension) have large positive values in the "Sci-Fi" axis. However, users with strong affinity for romance (the second concept or dimension) have very large negative values in the "Romance" axis. My question is - what exactly is the reference point(0) of these new axis?
I notice that in the YouTube video, the only projected values that are close to 0 are the 3rd dimension, which the instruction says does not encode much variance and can thus be discarded. So am I right to interpret this as - it doesn't matter what the sign of the projected values is, but only their absolute value?