introductory machine learning concept questions I just started learning about machine learning and the concepts behind the different methods and I wanted to get some clarification on the couple concepts. I'm filling out a true or false handout and I was wondering if my answers were correct:


*

*Covariance matrix can have negative values


*

*I think it's true since co-variance can be negative


*The `1 regularization cannot shrink parameters to zero, hence
it can be used for the purpose of feature selection


*

*I'm not sure about this one, but I think it's true?
Also I'm trying to identify the hyper-parameters of the following machine learning methods:


*

*Deep Neural Networks:


*

*is it learning rate and number of layers?



Are my answers correct?
 A: 
The `1 regularization cannot shrink parameters to zero, hence it can
be used for the purpose of feature selection

Yes. You can refer to this answer.

Deep Neural Networks

Many other hyperparameters, like embedding dimension, layer dimension, input length, parameter sharing, reused layers in transfer learning, early stopping strategy, learning rate decay and many others. Here is a good article.
For the hyperparameters, you can refer to the APIs in Tensorflow or sklearn.
A: #2
A covariance matrix cannot have eigenvalues less than zero, as it is a real, symmetric matrix. However, there is no such restriction on positive/negative/zero in the matrix itself.
As you note, covariance can be less than zero. This happens when variables have correlation less than zero. Therefore, there can be numbers less than zero in the covariance matrix.
Zero is a theoretical possibility in a covariance matrix, if there is zero correlation between two variables. In practice, however, you will not observe this in most data (see Henry’s comment...categorical data could have zero empirical correlation, too).
A: Re: L1 regularization, I think that's a trick question. The conclusion is true, but the antecedent is false -- L1 regularization can shrink parameters to zero, and that's why it can be used for variable selection (any features associated with zero parameters are effectively cut out of the model).
However, as you know, the implication "if X then Y" is true when X is false -- that is, it is a vacuous implication. I don't know how tricky your instructor is. If you answer "yes", I think you should be prepared to explain why. I think a "no" answer would be more consonant with an informal interpretation of the question, but, again, you should be prepared to explain.
