I have different pricing analysts who can set product prices (ecommerce) in random moment. I want to design algorithm which will rank these strategies on their effectiveness in any time period. All data available (prices, costs, types of products, total sales, etc). How to evaluate this strategies?

Thanks in advance.

  • $\begingroup$ The users can change the prices that are offered to them? Can you explain more about how the mechanism behind this? $\endgroup$ – DJohnson Nov 24 '15 at 14:24
  • 1
    $\begingroup$ I think "users" of the service that Василий Лукин wants to build, but they are actual ecommerce retailers. $\endgroup$ – mic Nov 24 '15 at 14:43
  • $\begingroup$ It is a pricing analysts who set prices on goods. $\endgroup$ – Василий Лукин Nov 24 '15 at 14:48
  • 1
    $\begingroup$ To rank according to effectiveness, you need to supply a sufficiently clear and precise definition of "effectiveness" to enable it to be quantified based on the data. What is your definition? $\endgroup$ – whuber Nov 24 '15 at 15:06
  • $\begingroup$ So, Василий Лукин is building an Amazon-like online retail platform? One factor that you didn't mention is marketing activity. As whuber notes, there are many possible approaches to evaluating pricing effectiveness strategies. This sounds like a big machine learning algorithm is required. Airlines have built ticket pricing models that react very quickly to a multitude of demand-related factors. Academics like Susan Athey at Stanford has many papers on the advent of machine learning methods in econometrics ... nasonline.org/programs/sackler-colloquia/documents/athey.pdf $\endgroup$ – DJohnson Nov 24 '15 at 15:07

To me it sounds like the real focus is on evaluating the abilities of pricing analysts to maximize gross profit and, in a second stage, comparing that to a metric derived from market growth. Finally, you want to rank the pricing analysts on their ability to outperform the market for a given time period. Gross profit can be defined as topline revenue less COGS but do you know how you want to define market growth? Is it at the product level?

To me, this sounds like a multi-level, mixed effects model with sales revenue as the target variable. The predictors could include price, product, product type, market (if more than one), marketing activity (promotions, discounts, advtg, etc.), factors relating to the pricing analyst such as age, gender, education, experience. The levels would be products within pricing analyst. After aggregating to the level of the pricing analyst, costs would be subtracted from the predicted revenue value to produce predicted gross profit.

I would leverage four summary metrics in ranking the performance of the pricing analysts:

  • The first metric would be predicted gross profit for each analyst. This overall performance metric is conditioned on the relevant factors for each analyst.

  • Next, assuming you've created a measure of market growth and potential gross profit, the difference between that and the pricing analysts' predicted gross profit would be the second summary metric. This would provide a relative measure of performance vis-a-vis the market.

  • The third metric could be based on the random effects slope for each analyst derived from the mixed effects model. This would provide a relative measure of analyst performance versus other analysts.

  • The fourth metric could be an analyst-specific price elasticity. This would be a measure of the analysts' ability to gauge the responsiveness of sales to their pricing strategy.

Then, standardizing each of these metrics into a z-score and adding them up to create a single, summary ranking metric could be created.

If you're concerned about equal weighting of the metrics or the correlations between, e.g., predicted gross profit and the differences in the z-score summation, you could also create a ranking based on scores from a principal components analysis of these four metrics. PCA would automatically compensate for any redundancies as well as differentially weight the metrics as a function of their communalities.

  • $\begingroup$ Sounds too complicated. It would be very interesting to know what algorithms are used in practice. $\endgroup$ – Василий Лукин Nov 27 '15 at 6:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.