# Reverse-engineering a (custom goods) pricing algorithm - each db row has: | 3 factors | 2 co-variates | price | (I have over 100k rows of data) [closed]

just wanted to mention up front that my question doesn't concern dynamic pricing, price optimization, revenue management, etc. No time series analysis either. It's just a simple multivariable regression problem- at least, I think so- for which price happens to be the dependent variable. But it has nothing to do with price theory or anything like that. All I'm really trying to do is look at a list of prices and determine the function or algorithm that produces them.

Again, it's simple pricing, nothing like Amazon for example, where they calculate market forces somehow in real-time and adjust their prices every 30 seconds, use crazy NLP algorithms, whatever it is they do.. This is just a simple service industry, with static pricing, where every order sold is basically just a given number of items, which are bought wholesale by the producer at a given cost, then customized by the company according to the customer's specifications, and sold to them once finished.

The cost of the actual goods is known reasonably well (maybe not with total certainty, since companies may pay slightly different rates based on their size, but the variation is slight). And as for the customization that is done, it is very easy to quantize. Typically something like:

• x layers of veneer at spot #1
• y layers of veneer at spot #2
• z number of total spots treated
• y/n - pretreatment performed?

That's basically it. The first two, I was considering factors with levels 1-6, since that's the range that 99% of all orders fall into. The binary "y/n" for whether pre-treatment was done I was considering as a categorical variable with two levels; together they are the 3 factors mentioned in the title of the post. z I didn't include in the count, because I'm not sure whether this particular company charges a separate fee for the numnber of spots treated, or simply adds up all the layers from all the different spots and charges for those. In most analyses I attempted, I left this variable out, hoping that, in those approaches that allow for it, it may be detected as a latent variable of some kind, if that's the correct term. Here's what a few lines of the dataset I have would look like:

|  Variable:  |           Type:          |           Possible Values:          |
|:-----------:|:------------------------:|:-----------------------------------:|
| C_S1        | Independent | Factor     | 6 Levels [1:6]                      |
| C_S2        | Independent | Factor     | 6 Levels [1:6]                      |
| Veneer      | Independent | Factor     | 2 Levels [Yes/No]                   |
| Cost        | Independent | Co-Variate | Continuous (generally $1 -$50)     |
| Quantity    | Independent | Co-Variate | Discrete (generally 12 - 3000 pcs)  |
| TOTAL PRICE | DEPENDENT VARIABLE       | CONTINUOUS (generally $50 -$10,000 |


Please remember this is only my best guess as to the correct representation of the actual problem. I'm not sure, for example, whether C_S1 (the number of varnish layers applied to spot 1 of the item) should rightly be considered a factor with 6 levels, and therefore require the incorporation of dummy variables for each of its levels, or simply be treated as a numeric independent variable that takes on integer values 1-6. Even more uncertain to me is whether I've properly classified 'Cost' and 'Quantity' as co-variates. Cost is what's paid for the original, unmodified good. Quantity is of course the number of items the customer orders on a given PO. This is of particular importance because the nature of the industry means that a large portion of the producer's cost comes from setting up the custom dyes, hand carving the templates and stencils, and so on - once that work is done, though, thousands of items could potentially be produced from that same set of custom stencils - so a large, fixed cost gets amortized over larger and larger quantities, and we see a pronounced 'bulk discount' effect, totally distinct from the typical "entice-you-to-buy-more" sort of bulk discounts that many companies offer.

This post is growing to an insane length so I'll try to wrap up fast- I've so far tried multivariable linear regression, GLM, PLS, MANCOVA (I think- whichever method in the ANOVA family it is that allows a mixture of categorical and continuous explanatory variables), and finally, non-linear regression with a whole host of different functional forms, ranging from those that come built in to the statistics packages I'm using (searched visually for those whose plots resembled the curves in my dataset, when it's plotted), to virtually every functional form I could think of that made semantical sense, (i.e., including explanatory variables for each of the factors/co-variates listed above, along with a fixed 'base' cost to model the amortized setup costs), which I typed in manually, set what I believed to be reasonable starting values for the parameters, and set it to run. And while I can get really good matches while holding one or more of the factors constant, I can't seem to find anything that comes close to being a 'universal' function for generating the correct price for most order specs. The only way I was able to get my goodness-of-fit and residuals in order was using a set of functions, piecewise, along with a Python script that provided the logic to select the correct function - inelegant, to say the least.

Are there any approaches/software packages that a relative novice like me might not be aware of, that are in use for solving problems like these? And was the information I gave sufficient, or should I be providing more concrete data here?

I know that the latter is certainly preferred, but unfortunately for legal reasons I can't exactly post entire chunks of my actual data. Any help would be greatly appreciated, not only in hopes that it might help me solve this problem, but also in that I would like to learn as much as possible and this seems like a perfect chance to do so.

## closed as too broad by Xi'an, gung♦, Nick Cox, Alexis, Glen_b♦Mar 24 '15 at 0:12

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.