How strict the rule is that the m should be less than cases(C)? Or the observations(n)? Is dummy variable a considered under this rule, too? Can I break this rule? If so, can anyone please suggest any reference?
Observations $n$ vs. cases $C$: a very simple model of this could be a "hierarchical" data structure with is a variation $\sigma^2_C$ between cases, and an independent variation $\sigma^2_n$ between the (repeated) observations for each case. In this scenario, what matters is which of the two contributes the major variation: if $\sigma^2_C \gg \sigma^2_n$, you have an effective sample size of $C$, if $\sigma^2_C \ll \sigma^2_n$ you are fortunate and the effective sample size is almost $n$. I usually encounter situations with effective sample size $\approx C$, though occasionally, the $n$ repeated measurements I have add up to a bit of information as well.
In any case, $C$ is a conservative estimate of the sample size.
As Peter Flom explained, there are sophisticated models for such situations. However, you may get away with more simple modeling (such as
lm) if you can reasonably apply the simple variance model I outlined above and judge that one of the two sources of variance dominates the other. If $\sigma^2_C$ dominates, weighting the rows so that each case in the end gets the same weight may be enough. If you are lucky and $\sigma^2_n$ dominates, then the linear model should be OK.
lm may not be the most appropriate model (i.e. you may get better performance with more sophisticated modeling), you can still validate the model you got and find out how good that model is.
Violating the rule:
Such rules are rules of thumb. In practice, you may have no choice but to
violate it. The risk you run if you have too few cases is that your model overfits the data and does not generalize well to new cases. There are several things you can do to measure whether you are fortunate and there are no bad consequences or whether you do have a problem here. One symptom of overfitting is that the model and its predictions become unstable with respect to slight changes of the training data (e.g. exchange one case against another).
This you can easily measure during a resampling validation scheme by comparing the different surrogate models you calculate.
Here's a paper where we discuss this:
Beleites, C. and Salzer, R. Assessing and improving the stability of chemometric models in small sample size situations, Anal Bioanal Chem, 2008, 390, 1261-1271. DOI: 10.1007/s00216-007-1818-6
I did not use stepwise regression, rather I applied my knowledge to keep useful variables that are most relevant, observing the significance level and finally checking the VIF. Is it necessary to use stepwise regression.
IMHO: a very wise decision.
IMHO stepwise regression is rarely a wise direction to take, whereas applying your knowledge about problem and data can help tremendously. It is one of the most valuables types of information you have about the problem.
In any case, stepwise regression needs tremendous sample sizes, otherwise you have a high risk of overfitting. As you already know that your number of cases is far below the recommended sample size, I'd keep my fingers away from optimization techniques (such as selecting an optimal set of variables).
I am planning to validate this (Or, the updated one) by using leave-one-out cross-validation. If I leave one case, the new regression models may have new sets of m variables where all the variables may not be the same as the final model below. Is that right? Or, Do I have to keep the same variables in each leave out model that I have in the final model in order to estimate the MSE for validation.
- Make sure you do leave-one-case-out, even if you think that $\sigma^2_C \ll \sigma^2_n$.
- If you do resampling validation, any data-driven variable selection (didn't you say above that you decided which variables to use!? ) must be done inside the validation loop. Comparing which variables are chosen is one of the possibilities to judge model stability. Of course that needs to be interpreted with the domain knowledge about the problem.
Here's another paper, where we report such differing sets of variables (though from the experience with that study I concluded that model optimization - such as stepwise variable selection or the region selection scheme in our paper - is often not sensible):
Beleites, C. and Baumgartner, R. and Bowman, C. and Somorjai, R. and Steiner, G. and Salzer, R. and Sowa, M. G. Variance reduction in estimating classification error using sparse datasets Chemom Intell Lab Syst, 2005, 79, 91 - 10.