In what ways are complex machine learning algorithms (e.g, random forests or support vector machines) different from simpler machine learning approaches (e.g., LASSO or ridge regression)?
Is this because with the more "complex" algorithms we have fewer constraints in implementation, i.e. for example models the regularized regression models can be considered to be constrained insofar as they only work if certain dependency assumptions are met? So with the more complex models we are able to capture non-linear relationships as well as linear?
In which case, what is best practice in selecting an ML algorithm. When do you go for say RF instead of regularized logistic regression?
