Performing regression in terms of X'X and X'Y I've heard that some people like to operate directly on X'X and X'Y rather than X and Y. I think this may be in the context of big data, to save space, but I am not sure. What is the point of such approach, what does it achieve? Is this a standard trick I am not aware of?
 A: (Taking my best guess as to the point being made)
Aside from doing very large problems (n and p huge), most standard regression programs use QR decomposition, though some use SVD or Cholesky decomposition.
QR decomposition finds $X=QR$ where $Q$ is orthogonal and $R$ upper triangular, so $X^\top X b = X^\top y$ becomes $R^\top R b = R^\top Q^\top y$, which reduces to solving $R b = Q^\top y$.
Consider that if $n$ is say $2\times 10^8$ and $p$ is say $5$, $X$ has a billion elements but $X^\top X$ has $25$ and $X^\top y$ has $5$.
So when it comes to solving $(X^\top X)b = X^\top y$ is seems like it's a lot easier to compute $X'X$ and $X'y$ and solve for $b$ (perhaps via Cholesky decomposition).
However, to a extent that's partly illusory. While it is slower to use QR decomposition (and SVD) than Cholesky decomposition, a lot of the overhead tends to come because the usual implementations make a full copy of X before operating on it in pace to produce R. This is not strictly necessary (though it may involve considerable effort to avoid in some situations), and if I recall correctly a carefully implemented QR designed for a tall, skinny $X$ that doesn't try to take a copy of X (pass by ref, and only allocate/use the space it actually needs) should only take about twice as long as a standard Cholesky.
(e.g. see https://arxiv.org/pdf/1301.1071.pdf )
While Cholesky can be faster (and if the QR is not very carefully implemented, maybe quite a bit faster), it is not as stable. On particular kinds of problems it can be quite handy, though.
The Cholesky can also be made more stable, at a similar cost to that for a QR specifically designed for a tall, skinny matrix.
So in part it depends on which QR algorithm, and which Cholesky algorithm and what shape of problem you have and how much you care about stability. 
[As someone once said, if you don't mind about the accuracy of the answer, you can have it as fast as you like.]
