I'm trying to obtain an estimator $f(x)=y$ where $x \in \mathbb{R}^{D_1}$ and $y \in \mathbb{R}^{D_2}$, both are column vectors. So my training set $X$ and $Y$ are data matrices of size $D_1 \times N$ and $D_2 \times N$, respectively, and I want to learn $\beta$ that gives $\beta x \sim y$ in a least-squares fashion. I was doing this in MATLAB simply by beta_hat = Y * pinv(X); and it seems like working without a problem. Though I want to ask, is this correct?

My question:

Now I want to implement this without pinv because I want to add regularization to it, so I came up with this solution (this is without regularization) : $\hat \beta = Y (X^TX)^{-1}X^T$ is this correct? It also works but MATLAB complains about this :

Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND =  2.565271e-20.

And even crashes sometimes. So I think I'm making a mistake somewhere, but where?

Thanks in advance,