12 better (new) tag added | link edited Jan 16 at 5:20 Carl 7,77344 gold badges2929 silver badges7979 bronze badges Notice removed Draw attention by Community♦ occurred Nov 28 '18 at 13:02 Bounty Ended with no winning answer by Community♦ occurred Nov 28 '18 at 13:02 11 added 91 characters in body edited Nov 21 '18 at 7:20 Cowboy Trader 1,2191717 silver badges4141 bronze badges For the solution of $$Ax = b$$, where $$A$$ is a square matrix, what is the difference between these two regularized solutions: $$x = (A + \alpha I)^{-1}b$$ -- coressponding to eq.3 below $$x = (A^TA + \alpha I)^{-1}A^Tb$$ -- corresponding to eq.2 below with $$K(w) = A, K(-w) = A^T$$ In the following reference: https://www.sciencedirect.com/science/article/abs/pii/0041555382900945# for positive symmetric difference kernels (for example a Gaussian kernel) the first form of regularization is promoted as a simplified Tikhonov regularization (wein statistical models we usually use $$p = 0$$ case for $$M(w) = I$$). Is the simplification just an approximation or is it equivalent to the second form above? Basically I am trying to understand the connection between discretized versions that we use in statistics and the continuous versions that are used in integral equations. This question is related to this: Regularized linear vs. RKHS-regression where the answer claims a substantial difference, in contrast to the above recommendation. For the solution of $$Ax = b$$, where $$A$$ is a square matrix, what is the difference between these two regularized solutions: $$x = (A + \alpha I)^{-1}b$$ -- coressponding to eq.3 below $$x = (A^TA + \alpha I)^{-1}A^Tb$$ -- corresponding to eq.2 below with $$K(w) = A, K(-w) = A^T$$ In the following reference: https://www.sciencedirect.com/science/article/abs/pii/0041555382900945# for positive symmetric difference kernels (for example a Gaussian kernel) the first form of regularization is promoted as a simplified Tikhonov regularization (we usually use $$p = 0$$ case for $$M(w) = I$$). Is the simplification just an approximation or is it equivalent to the second form above? Basically I am trying to understand the connection between discretized versions that we use in statistics and the continuous versions that are used in integral equations. This question is related to this: Regularized linear vs. RKHS-regression For the solution of $$Ax = b$$, where $$A$$ is a square matrix, what is the difference between these two regularized solutions: $$x = (A + \alpha I)^{-1}b$$ -- coressponding to eq.3 below $$x = (A^TA + \alpha I)^{-1}A^Tb$$ -- corresponding to eq.2 below with $$K(w) = A, K(-w) = A^T$$ In the following reference: https://www.sciencedirect.com/science/article/abs/pii/0041555382900945# for positive symmetric difference kernels (for example a Gaussian kernel) the first form of regularization is promoted as a simplified Tikhonov regularization (in statistical models we usually use $$p = 0$$ for $$M(w) = I$$). Is the simplification just an approximation or is it equivalent to the second form above? Basically I am trying to understand the connection between discretized versions that we use in statistics and the continuous versions that are used in integral equations. This question is related to this Regularized linear vs. RKHS-regression where the answer claims a substantial difference, in contrast to the above recommendation. Tweeted twitter.com/StackStats/status/1064896307846877185 occurred Nov 20 '18 at 15:00 10 deleted 13 characters in body edited Nov 20 '18 at 14:57 Cowboy Trader 1,2191717 silver badges4141 bronze badges For the solution of $$Ax = b$$, where $$A$$ is a square matrix, what is the difference between these two regularized solutions: $$x = (A + \alpha I)^{-1}b$$ -- coressponding to eq.3 below $$x = (A^TA + \alpha I)^{-1}A^Tb$$ -- corresponding to eq.2 below with $$K(w) = A, K(-w) = A^T$$ In the following reference (equation 3): https://www.sciencedirect.com/science/article/abs/pii/0041555382900945# for positive symmetric difference kernels (for example a Gaussian kernel) the first form of regularization is promoted as a simplified Tikhonov regularization (we usually use $$p = 0$$ case for $$M(w) = I$$). Is the simplification just an approximation or is it equivalent to the second form above? Basically I am trying to understand the connection between discretized versions that we use in statistics and the continuous versions that are used in integral equations. This question is related to this: Regularized linear vs. RKHS-regression For the solution of $$Ax = b$$, where $$A$$ is a square matrix, what is the difference between these two regularized solutions: $$x = (A + \alpha I)^{-1}b$$ -- coressponding to eq.3 below $$x = (A^TA + \alpha I)^{-1}A^Tb$$ -- corresponding to eq.2 below with $$K(w) = A, K(-w) = A^T$$ In the following reference (equation 3): https://www.sciencedirect.com/science/article/abs/pii/0041555382900945# for positive symmetric difference kernels (for example a Gaussian kernel) the first form of regularization is promoted as a simplified Tikhonov regularization (we usually use $$p = 0$$ case for $$M(w) = I$$). Is the simplification just an approximation or is it equivalent to the second form above? Basically I am trying to understand the connection between discretized versions that we use in statistics and the continuous versions that are used in integral equations. This question is related to this: Regularized linear vs. RKHS-regression For the solution of $$Ax = b$$, where $$A$$ is a square matrix, what is the difference between these two regularized solutions: $$x = (A + \alpha I)^{-1}b$$ -- coressponding to eq.3 below $$x = (A^TA + \alpha I)^{-1}A^Tb$$ -- corresponding to eq.2 below with $$K(w) = A, K(-w) = A^T$$ In the following reference: https://www.sciencedirect.com/science/article/abs/pii/0041555382900945# for positive symmetric difference kernels (for example a Gaussian kernel) the first form of regularization is promoted as a simplified Tikhonov regularization (we usually use $$p = 0$$ case for $$M(w) = I$$). Is the simplification just an approximation or is it equivalent to the second form above? Basically I am trying to understand the connection between discretized versions that we use in statistics and the continuous versions that are used in integral equations. This question is related to this: Regularized linear vs. RKHS-regression 9 added 129 characters in body edited Nov 20 '18 at 14:29 Cowboy Trader 1,2191717 silver badges4141 bronze badges 8 added 15 characters in body edited Nov 20 '18 at 13:24 Cowboy Trader 1,2191717 silver badges4141 bronze badges 7 added 15 characters in body edited Nov 20 '18 at 13:16 Cowboy Trader 1,2191717 silver badges4141 bronze badges 6 added 15 characters in body edited Nov 20 '18 at 13:10 Cowboy Trader 1,2191717 silver badges4141 bronze badges 5 added 132 characters in body; edited title edited Nov 20 '18 at 12:00 Cowboy Trader 1,2191717 silver badges4141 bronze badges Notice added Draw attention by Cowboy Trader occurred Nov 20 '18 at 11:56 Bounty Started worth 50 reputation by Cowboy Trader occurred Nov 20 '18 at 11:56 4 added 132 characters in body; edited title edited Nov 20 '18 at 11:55 Cowboy Trader 1,2191717 silver badges4141 bronze badges 3 added 220 characters in body edited Nov 15 '18 at 19:16 Cowboy Trader 1,2191717 silver badges4141 bronze badges 2 deleted 1 character in body edited Nov 15 '18 at 18:20 Cowboy Trader 1,2191717 silver badges4141 bronze badges 1 asked Nov 15 '18 at 16:36 Cowboy Trader 1,2191717 silver badges4141 bronze badges