1
$\begingroup$

I am running a reg model with Weekdays as my dummy variables so I can find the weekday effects to the output metric. The picture below shows the results from this regression, and only Sunday is significant. In this case, I usually like to group the Mon, Sat, Thur, Tue & Wed into one group along with the baseline "Friday" as they are all insignificantly different to the base-line Friday (named as "Rest"), and re-run the model with two dummies "Sun" and "Rest". (one of the two will be automatically chosen as the base line by R). I wonder is there some package or method R can do this auto-regrouping and re-run the model by itself till all grouped dummy groups are significant.

thank you.

enter image description here

$\endgroup$
  • 4
    $\begingroup$ This sounds like an R question--it clearly is asking for R code--so it's sure to get close votes. But there's an important statistical question beneath it related to the inadvisability of abusing p-values in this way. It seems likely this procedure is going to be used inappropriately to determine some "optimal" way to group the days. Consider addressing that issue before you use this procedure. $\endgroup$ – whuber Aug 22 '17 at 17:14
5
$\begingroup$

As whuber indicated in the comments, this is a bad idea.

  1. The $p$-values of the new model will be inaccurate because they won't account for how you selected the model based on previous $p$-values.

  2. What's more, $p$-values of coefficients are a bad way to select models because they don't directly quantify any useful measures of model quality.

  3. Finally, while obtaining $p < α$ tells you that a null hypothesis is infeasible, obtaining $p > α$ doesn't much support its feasibility, so when a test fails to reject the null hypothesis, it doesn't tell you anything.

$\endgroup$
  • $\begingroup$ Thank you for the comments, then based on the results I posted, what action you will take? Will you discard the insignificant dummies? Will you still leave them in the model to address the effects from those weekdays? or anything else? $\endgroup$ – MeiNan Zhu Aug 22 '17 at 17:49
  • 3
    $\begingroup$ @MeiNanZhu Since you want to know how they're related to the dependent variable, you should leave them in the model. $\endgroup$ – Kodiologist Aug 22 '17 at 17:54
  • $\begingroup$ Look into fused lasso (search this site) $\endgroup$ – kjetil b halvorsen Aug 22 '17 at 21:05
  • $\begingroup$ (+1) Also merging other days with Friday (selected as reference because it's the first alphabetically) is quite arbitrary. $\endgroup$ – Scortchi Aug 24 '17 at 20:09
  • 1
    $\begingroup$ @Kanak Stepwise regression does not operate on a principle, it operates on a rule of thumb. By dropping or grouping variables you are claiming, with 100% confidence, that you wish to override the data and claim that the effect of the variable is zero (or the same as the effect of some other variable). A hypothesis test, which is based on the assumption that the effect is zero, can, by fiat of this assumption, give no evidence that the effect actually is zero. Dropping variables based on p-values is wrong, and a simple abuse of a statistic because the abuse is convenient. $\endgroup$ – Matthew Drury Aug 25 '17 at 1:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.