Proving Ridge Regression is strictly convex Definition of ridge regression
$$min_\beta||y-X\beta||_2^2+\lambda||\beta||_2^2, \lambda\ge0$$
you can prove a function is strictly convex if the 2nd derivative is strictly greater than 0 thus

But unfortunately I don't know if this is sufficient proof as it's possible for $X^TX$ to be negative and $\lambda$ can be 0. Unless I'm missing something.
 A: "you can prove a function is strictly convex if the 2nd derivative is strictly greater than 0"
That's in one dimension.  A multivariate twice-differentiable function is convex iff the 2nd derivative matrix is positive semi-definite, because that corresponds to the directional derivative in any direction being non-negative. It's strictly convex iff the second derivative matrix is positive definite.
As you showed, the ridge loss function has second derivative $2\lambda I +2X^TX$, which is positive definite for any $\lambda>0$ because

*

*$\lambda I$ is positive definite for any $\lambda>0$

*$X^TX$ is positive semi-definite for any $X$

*the sum of a positive definite and positive semi-definite matrix is positive definite

If you aren't sure about any of these and want to check in more detail it's useful to know that $A$ is positive definite iff $b^TAb>0$ for all (non-zero) column vectors $b$.  Because of this relationship, many matrix proofs of positive definiteness just come from writing the scalar proofs of positiveness in matrix notation (including non-trivial results like the Cramér-Rao lower bound for variances)
A: Less of a proof, and more of a convincing argument (that can lead you towards the proof): we all agree ordinary least squares with full rank covariance matrix $X^TX$ is strictly convex (see Convexity of linear regression), ridge regression is a form of OLS with augmented (virtual) data, thus it's also strictly convex.
The augmentation $X\text{aug} = \left[ \begin{matrix}X^T & \sqrt\lambda\mathbb I \end{matrix}\right]^T$ actually ensures that, in ridge, $X\text{aug}^TX\text{aug}$ is full rank, since it consists of concatenating a multiple of the identity matrix $\sqrt\lambda\mathbb I$.
So, if you can show that the equivalent OLS is strictly convex, so is ridge regression.
