Below is the structure of the data,

1111          1        aa3a       x       100      1     0.5    0    35  100
1111          1        aa3a       x       100      0     0.5    0    36  100
1111          2        aa3a       x       100      0     0.6    1    36  150
1222          3        aa3a       y        20      0     1.0    0    20  200
1233          3        ab3b       z       1200     1     1.2    1    43  12000        

Data explanation : col1 : products, categorical. col2: Categorical. Levels:1,2 and 3. each product can be in any of the three regions, max in regions. col3:Categorical, multiple productId can be under this rangeID. col4: categorical,multiple productID can be under this brand. brand will be under prodrangeId. col5: continuous. col6: numeric, ranging from 0 to 7. col7: continuous decimals from 0 to 1.5. col8: numeric, varies from 0 to 14. col9: Week number of the year. col10: sales in number of units

Objective : To predict the sales of new product [new data] with good accuracy. [error less than 25% - error is measured as Weighted Average Percentage Error]

Note: sales are in number of units [not in dollar value]

After many iterations, my current model is,

fit= lmer(sales = number of stores+ number of stores*holiday*Seasonal*promotions + number of stores*promotions*SI+ number of stores*SI+ (1|ProductID/Region)+(1|Week number),historical data,control=lmerControl(optimizer="optimx",optCtrl=list(method="nlminb")),REML=FALSE)

predict(fit, new data, allow.new.level=TRUE)
  1. First parameter because to include number of stores as separate effect
  2. second param, because holiday, seasonal and promotions can not act directly on sales. So, with interactions
  3. third param and fourth, because when there is no holiday/no promotions, effect of other variables should be present. { ** Here** : I am not sure if I have understood the interactions correctly. If someone could suggest something here, would be better

Quick Note : Effect of Number of stores varies across regions. Ex: 10 stores in region 1 has 20 unit sales, whereas 10 stores in region 2 has 50 unit sales. I do not know how do represent this in the model. Need help here!

And as shown in the data, there are other factors, which I do not know how to represent or will be useful for the model. Any help here would be appreciated!

Results are 50 to 60% error. size and values of the dataset can vary significantly for different set of products.

Another Question : How do I ensure that my model is stable?

Attention : I have tried the following models, which did not yield good accuracy:

lmer(sales(in units) = number of stores + holiday + Seasonal + number of  promotions+(1|ProductID/Region)+(1|Week number)
lmer(sales = number of stores*holiday*Seasonal + (number of stores|ProductID/Region)+(number of stores|Week number)
lmer(sales = number of stores*holiday*Seasonal + number of stores +number of stores*Seasonal+ (number of stores|ProductID/Region)+(number of stores|Week number)

[My thoughts on above models :

  1. holiday can impact sales of a product, but indirectly on number of stores and same goes with Seasonal.

  2. In the second model, I have used number of stores as random slope, as It varies across products. I am not sure if a continuous variable can be used as random slope.

{ Here I have a question : Number of stores varies across different regions, how to represent that? How do add that interaction?}

  1. In third model, i have used number of stores as separate effect, because in few cases holiday will not be there and effect of number of stores should be taken care. ]

some errors which I am unable to understand : fixed-effect model matrix is rank deficient so dropping 10 columns / coefficients

And the was convergence errors: for this I have added control parameter, as suggested by Ben Bolker, Roland.

Finally : I am not sure, if mixed models is the right approach here. If someone could throw some light on this, would be good.

  • 1
    $\begingroup$ Welcome to stats.SE! Please take a moment to view our tour. It is preferred that each question is focused on one problem/question at a time. It can be appropriate to combine a few questions together, but as the number of questions increase, the difficulty in providing an appropriate answer also increases. Your question reads as if it may come from a textbook. If that is the case, please take a moment to review the wiki for our self-study tag and add the tag to your question as appropriate. $\endgroup$ – Tavrock Mar 2 '17 at 16:49
  • $\begingroup$ Thanks Tavrock! I had too many questions in my mind, had to put it all in the same context. I can understand it will be difficult to read through everything. From next time, I will split the questions. This is not from text book, these are the problems I am facing! $\endgroup$ – SinAt Mar 3 '17 at 3:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.