I am regressing a continuous predictor on over 60 variables (both continuous and categorical) using LASSO (glmnet).
In examining the variable trace plot, I notice that as log lambda increases, one of the key variables has a coefficient that actually increases. Then, after a certain point, it begins to decrease like we would expect.
To make sure this wasn't a fluke, I ran 10 models using bootstraps and obtained very similar results.
Is this possible, or is there a problem with the data? If legitimate, what does this trend in the variable's coefficient tell us about the variable and the relation to the response?