When using ANOVA to compare means, I understand the null hypothesis is u1=u2=u3.... as we need to combine all groups into a single group when assuming their means are the same. But for ANOVA for Regression, I don't understand why the null hypothesis is : The model is ineffective(slope is zero).Why can't we have The model is effective(slope is non-zero) as the null hypothesis?
Linear regression and ANOVA are the same thing except that the former attempts to predict a continuous outcome using one or more continuous predictor variables, whereas the latter uses one or more categorical predictor variables to predict a continuous outcome.
So, if you have a continuous predictor you can test whether a model with slope is different from a model without slope. A model without slope is the simplest model we can fit and that's the mean and also the null hypothesis in a linear regression context. So if your regression coefficient $b_1$ is not significantly different from $b_0$ (the intercept), you might as well use the mean (the intercept) to describe your data, which could also be translated as a model with slope is ineffective.
In hypothesis testing, the null hypothesis is never what we want to demonstrate (translated in your case, that the model is effective), but the reverse, for which we know the distribution of the statistic.
Under H0 (slope is zero), we know that MSr divided by MSe will follow a F-distribution.