1
$\begingroup$

I am currently having trouble with the results of a user defined ridge regression function (one that I have created) against that of the MASS::lm.ridge() function. Below is what is defined as my data:

x <- seq(-0.5, 0.5, 0.1)
b <- sin(2 * pi * x) - 0.1 + 0.2 * runif(11)
xb <- dataframe(x,b)

order <- 9
orders <- 1:order
X9 <- t(t(replicate(order,x))^orders)

The code above is describing my data, with x as the independent variable, b as the dependent, and X9 as my data used as ninth order polynomial matrix.

#MODEL USING MASS
lm_regularized_coef <- coef(lm.ridge(formula = b ~ poly(scale(x), degree = 9, raw = T), data = xb,  lambda = 0.001))
coef_fit <- as.matrix((lm_regularized_coef))

# USER DEFINED FUNCTION
ridge_line_fit <- function (x, b, lambda) {
    X = as.matrix(cbind(1,x))
    inv = solve((t(X) %*% X) + (lambda * diag(dim(t(X) %*% X)[2])))  #(X^T * X + lambda*I)^-1
    weights = inv  %*% t(X) %*% as.matrix(b) #(X^T * X + lambda*I)^-1 * X^T * y
    return(weights)
}

xs <- scale(X9) #For Scaling 
rlf <- ridge_line_fit(x = xs, b = data.matrix(b), lambda = 0.001)

cbind(coef_fit, rlf)

which gives me the results below:

                                            [,1]         [,2]
                                     -0.02513236 -0.004314105
poly(scale(x), degree = 9, raw = T)1  2.00391977  1.999360461
poly(scale(x), degree = 9, raw = T)2  0.02474866  0.017679371
poly(scale(x), degree = 9, raw = T)3 -1.19300070 -2.073829478
poly(scale(x), degree = 9, raw = T)4 -0.11014914 -0.200522427
poly(scale(x), degree = 9, raw = T)5  0.02744687  0.086002817
poly(scale(x), degree = 9, raw = T)6  0.09376673  0.396663975
poly(scale(x), degree = 9, raw = T)7  0.08204332  0.633710744
poly(scale(x), degree = 9, raw = T)8 -0.01781305 -0.170140701
poly(scale(x), degree = 9, raw = T)9 -0.01465847 -0.237401993

The left side are the coefficients from the MASS::lm.ridge() function, and the coefficients on the right are from the user defined function. I really cannot see why the two would be off. Wondering if someone could see if I may have an underlying issue that I may have missed.

Thanks

$\endgroup$
  • $\begingroup$ From a quick glance, you scale differently between the models; try changing to scale(poly(x, degree...)) or use lm.ridge(formula = b ~ xs, ... $\endgroup$ – user20650 Oct 22 at 23:31
  • $\begingroup$ also from ?lm.ridge`; "f an intercept is present in the model, its coefficient is not penalized." but it is in your function $\endgroup$ – user20650 Oct 22 at 23:38
  • $\begingroup$ Changing the the scale into the inside did the trick - thank you so much! That helped! Also, I see what you mean for the function - I will accommodate for that - probably exclude the first coefficient within the matrix after its calculated. $\endgroup$ – WorkInProgress_101 Oct 23 at 0:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.