How to model prices?

Question

Cross-posted at Math.SE.

I'm working on a hobby project and would need some help with the following problem.

A bit of context

Let's say there is a collection of items with a description of features and a price. Imagine a list of cars and prices. All cars have a list of features, e.g. engine size, color, horse power, model, year etc. For each make, something like this:

Ford:
V8, green, manual, 200hp, 2007, $200
V6, red, automatic, 140hp, 2010, $300
V6, blue, manual, 140hp, 2005, $100
...

Going even further, the list of cars with prices is published with some time-interval which means we have access to historical price data. Might not always include exactly the same cars.

Problem

I would like to understand how to model prices for any car based on this base information, most importantly cars not in the initial list.

Ford, v6, red, automatic, 130hp, 2009

For the above car, it's almost the same as one in the list, just slightly different in horse power and year. To price this, what is needed?

What I'm looking for is something practical and simple, but I would also like to hear about more complex approaches how to model something like this.

What I've tried

Here is what I've been experimenting with so far:

1. using historical data to lookup car X. If not found, no price. This is of course very limited and one can only use this in combination with some time decay to alter prices for known cars over time.

2. using a car feature weighting scheme together with a priced sample car. Basically that there is a base price and features just alter that with some factor. Based on this any car's price is derived.

The first proved to be not enough and the second proved to not be always correct and I might not have had the best approach to using the weights. This also seems to be a bit heavy on maintaining weights, so that's why I thought maybe there is some way to use the historical data as statistics in some way to get weights or to get something else. I just don't know where to start.

Other important aspects

• integrate into some software project I have. Either by using existing libraries or writing algorithm myself.
• fast recalculation when new historical data comes in.

Any suggestions how a problem like this could be approached?

Answer 1

"Practical" and "simple" suggest least squares regression. It's easy to set up, easy to do with lots of software (R, Excel, Mathematica, any statistics package), easy to interpret, and can be extended in many ways depending on how accurate you want to be and how hard you're willing to work.

This approach is essentially your "weighting scheme" (2), but it finds the weights easily, guarantees as much accuracy as possible, and is easy and fast to update. There are loads of libraries to perform least squares calculations.

It will help to include not only the variables you listed--engine type, power, etc--but also age of car. Furthermore, make sure to adjust prices for inflation.

Answer 2

I agree with @whuber, that linear regression is a way to go, but care must be taken when interpreting results. The problem is that in economics the price is always related to demand. If demand goes up, prices go up, if demand goes down, prices go down. So the price is determined by demand and in return demand is determined by price. So if we model price as a regression from some attributes without the demand there is a real danger that the regression estimates will be wrong due to omitted-variable bias.

Answer 3

What I'm looking for is something practical and simple, but I would also like to hear about more complex approaches how to model something like this.

After some sort of a discussion, here is my complete view of the things

The problem

Aim: to understand how to price the cars in a better way

Context: in their decision process people solve several questions: do I need a car, if I do, what attributes I prefer most (including the price, because, being rational, I would like to have a car with best quality/price ratio), compare the number of attributes between different cars and choosing valuing them jointly.

From the seller position, I would like to set the price as high as possible, and sell the car as quickly as possible. So if I set the price too high and am waiting for months it could be considered as not demanded on the market and marked with 0 comparing to very demanded attribute sets.

Observations: real deals that relates the attributes of a particular car with the price set within the bargaining process (regarding the previous remark it is important to know how long it take to set the deal).

Pros: you do observe the things that were actually bought on the market, so you are not guessing if there exist a person with high enough reservation price that wants to buy a particular car

Cons:

1. your assumption is that market is efficient, meaning the prices you observe are close to equilibrium
2. you ignore the variants of car attributes that were not purchased or took too long to set the deal, meaning your insights are biased, so you actually do work with latent variable models
3. Observing the data for a long time you need to deflate them, though the inclusion of the car age partly compensates this.

Solution methods

The first one, as suggested by whuber, is the classical least squares regression model

Pros:

1. indeed the simplest solution as it is the work-horse of econometrics

Cons:

1. ignores that you do observe the things incompletely (latent variables)
2. acts as the regressors are independent one of the other, so the basic model ignores the fact that you may like blue Ford differently from blue Mercedes, but it is not the sum of marginal influence that comes from blue and Ford

In case of classical regression, since you are not limited in the degrees of freedom, to try also different interaction terms.

Therefore more complicated solution would be either tobit or Heckman model, you may want to consult A.C. Cameron and P.K. Trivedi Microeconometrics: methods and applications for more details on core methods.

Pros:

1. you do separate the fact that people may not like some sets of attributes at all, or some set of attributes has a small probability to be bought from the actual price setting
2. your results are not biased (or at least less than in the first case)
3. in case of Heckman you separate the reasons that motivates to buy the particular car from the pricing decision of how much I would like to pay for this car: the first one is influenced by individual preferences, the second one by budget constraint

Cons:

1. Both models are more data greedy, i.e. we need to observe either the time length between the ask and bid to equalize (if it is fairly short put 1, else 0), or to observe the sets that were ignored by the market

And, finally, if you simply interested in how price influences the probability to be bought you may work with some kind of logit models.

We agreed, that conjoint analysis is not suitable here, because you do have different context and observations.

Good luck.

Answer 4

Besides what have been said, and not really quite different from some of the suggestions already made, you might want to have a look at the vast literature on hedonic pricing models. What it boils down to is a regression model trying to explain the price of a composite good as a function of its attributes.

This would allow you to price a car knowing its attributes (horse power, size, brand, etc.), even if an exactly similar mix of attributes is not present in your sample. It is a very popular approach for valuation of essentially non replicable assets --like real state properties. If you Google for "hedonic models" you will find many references and examples.

Answer 5

It looks like a linear regression problem me too, but what about K nearest neighbors KNN. You can come up with a distance formula between each car and compute the price as the average between the K (say 3) nearest. A distance formula can be euclidian based like the difference in cylinders plus the difference in doors, plus difference in horsepower and so on.

If you go with linear regresion I would suggest a couple things:

• Scale the dollar value up to modern day to account for inflation.
• Divide your data into epochs. I'll bet you'll find you will need one model for pre ww2 and post ww2 for example. This is just a hunch though.
• Cross validate your model to avoid over fitting. Divide your data into 5 chunks. Train on 4 and urn the model on the 5th chunk. Sum up the errors, rinse, repeat for the other chunks.

Another idea is to made a hybrid between models. Use regresion and KNN both as datapoints and create the final price as the weighted average or something.