I am building a model for forecasting some number of activations. My data set has a panel structure.
Now, I want to come up with a forecast performance measure to assess the performance of my model and to compare it with other models. In this measure I want to punish too low forecasts differently from too high forecasts (so an asymmetric performance measure). I was thinking about the MAPE, but my data also contains 0 (and small) values, so the MAPE will not work. As an extra I would like to be able to change the degree to which too high predictions are punished differently from too low predictions.
I want the asymmetric measure because too high forecasts are worse than too low forecasts, since too high predictions leads to unused stock while too low stock allows for acquiring new stock which is less severe in my case.
Does anyone have a suggestion which performance measure I could use?