1
$\begingroup$

I am running a reg model with Weekdays as my dummy variables so I can find the weekday effects to the output metric. The picture below shows the results from this regression, and only Sunday is significant. In this case, I usually like to group the Mon, Sat, Thur, Tue & Wed into one group along with the baseline "Friday" as they are all insignificantly different to the base-line Friday (named as "Rest"), and re-run the model with two dummies "Sun" and "Rest". (one of the two will be automatically chosen as the base line by R). I wonder is there some package or method R can do this auto-regrouping and re-run the model by itself till all grouped dummy groups are significant.

thank you.

enter image description here

Improve this question
$\endgroup$
1
  • 4
    $\begingroup$ This sounds like an R question--it clearly is asking for R code--so it's sure to get close votes. But there's an important statistical question beneath it related to the inadvisability of abusing p-values in this way. It seems likely this procedure is going to be used inappropriately to determine some "optimal" way to group the days. Consider addressing that issue before you use this procedure. $\endgroup$
    – whuber ♦
    Commented Aug 22, 2017 at 17:14

1 Answer 1

Reset to default
5
$\begingroup$

As whuber indicated in the comments, this is a bad idea.

  1. The $p$-values of the new model will be inaccurate because they won't account for how you selected the model based on previous $p$-values.

  2. What's more, $p$-values of coefficients are a bad way to select models because they don't directly quantify any useful measures of model quality.

  3. Finally, while obtaining $p < α$ tells you that a null hypothesis is infeasible, obtaining $p > α$ doesn't much support its feasibility, so when a test fails to reject the null hypothesis, it doesn't tell you anything.

Improve this answer
$\endgroup$
6
  • $\begingroup$ Thank you for the comments, then based on the results I posted, what action you will take? Will you discard the insignificant dummies? Will you still leave them in the model to address the effects from those weekdays? or anything else? $\endgroup$
    – MeiNan Zhu
    Commented Aug 22, 2017 at 17:49
  • 3
    $\begingroup$ @MeiNanZhu Since you want to know how they're related to the dependent variable, you should leave them in the model. $\endgroup$
    – Kodiologist
    Commented Aug 22, 2017 at 17:54
  • $\begingroup$ Look into fused lasso (search this site) $\endgroup$
    – kjetil b halvorsen ♦
    Commented Aug 22, 2017 at 21:05
  • $\begingroup$ (+1) Also merging other days with Friday (selected as reference because it's the first alphabetically) is quite arbitrary. $\endgroup$
    – Scortchi ♦
    Commented Aug 24, 2017 at 20:09
  • 1
    $\begingroup$ @Kanak Stepwise regression does not operate on a principle, it operates on a rule of thumb. By dropping or grouping variables you are claiming, with 100% confidence, that you wish to override the data and claim that the effect of the variable is zero (or the same as the effect of some other variable). A hypothesis test, which is based on the assumption that the effect is zero, can, by fiat of this assumption, give no evidence that the effect actually is zero. Dropping variables based on p-values is wrong, and a simple abuse of a statistic because the abuse is convenient. $\endgroup$
    – Matthew Drury
    Commented Aug 25, 2017 at 1:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.