After doing some research I couldn't really find a clear difference between those two terms when considering machine learning except that a covariate has to be continuous.
1 Answer
There's a lot of confusion of terms in moving between statistics and machine learning. These two terms are essentially synonyms.
Covariate implies a separate, related, focal variable (e.g., the dependent variable in regression). You can talk about features without a focal variable in mind. But sometimes people talk about covariates without a specified dependent variable, and there's nothing wrong with calling your independent regression variables "features".
a covariate has to be continuous
...that is not true.