So I'm trying the implement the Backward Feature Selection algorithm using the Hilbert Schmidt Independence Criterion (a.k.a BAHSIC) but I'm getting an out of memory error when calculating the kernel's K and L in equation (5) (currently using a linear kernel) due to the amount of data in my data set (more than 100k observation). Reading in the internet there some methods to calculate the kernel when memory error happens like Nystrom or Random features. But the problem I found with it is that I wont end up with a squared matrix and as said in the paper "The diagonal entries of K and L are set to zero".
So anyone knows a turn around to this problem? In the same paper the say it is possible to do a low rank approximation of the kernel matrix using incomplete cholesky descomposition. But searching the web didn't find anything practical about (just found about basic cholesky descompostion).