@MartijnWeterings Ah, the images are actual images (shown to humans) which are transformed to the inputs, and I think "training/test images" just means the images from the training/set sets. The article is well cited in a reputable journal and is still undecipherable so I will try emailing the authors.

@MartijnWeterings What do you suspect is done in step 3? It says "Student’s t test across cross-validated training images" (and not test images) which seems that it has nothing to do with step 3. Taking one r from each fold is invalid as amoeba says, so regardless of what the paper does, do you have a suggestion for the best way to achieve this problem of determining Q? (Unsurprisingly, my attempt at reproducing the results of this paper is proving challenging.)

Thanks for your comments - I've added clarifications in the question body. The paper is jneurosci.org/content/35/27/10005, but it's got a lot more to it than just this so I've done my best to include all relevant parts in my question.

Great answer. This problem arose because a paper I was following suggested, since n ≪ p, computing the SVD in order to grid-search for the best alpha, since the resulting computation for each alpha only takes O(pn^2). However, my results/your answer shows that this would only take less time if there are greater than ~10 different values of alpha to try, due to the initial overheard of the SVD being huge.

IMO the amendment to this question was sufficiently different that it required its own question which I posted here: stats.stackexchange.com/questions/396914/…

Okay, so it seems LinearRegression is using SVD to find the L2 norm. In which case my question becomes: why is computing ridge regression with cholesky quicker than with SVD? (Meta question: should I adjust my question title to reflect this?)