How to build a regression model with just 5 datapoints with 5 or more variables?

I am attempting to build a simple multiple regression in order to estimate the value of a home. The 5 datapoints are 5 comparable home sales (similar in size, lot, location, # of bedrooms, etc...).

If you run such a model, the software will say you need one more datapoint than variables.

So, how about if I take my 5 comparable home sales and duplicate them 4 times. So, now I have 20 datapoints. The software should be able to run the related multiple regression.

I anticipate that you will answer that this methodology is flawed because underlying all that you still have only 5 differentiated datapoints. But, can you flesh out this rebuttal a bit further.

Also, can you suggest any alternative method to resolve this constraint.

-
If the five other homes are similar in size, lot, location, # of bedrooms, do you need a regression? Why not just take the average price as @EmreA suggests? What are your explanatory variables anyway? –  Peter Ellis Aug 8 '12 at 20:02
Peter, they are similar but far from identical. There are variations in size, lot size, # of bedrooms & bathrooms, etc... And, those characteristics are the independent variables. As mentioned in another comment, I have resolved this conundrum by using an optimization (minimizing the square of the errors in estimating sales prices of those homes). It worked well. And, I generated almost the exact same valuation for my house as the one generated by Value Appeal.com. The latter is a professional service that assists you in appealing property tax. –  Gaetan Lion Aug 8 '12 at 23:37

I can think of two ways.

The simplest is to assume that the price of those comparable homes follows a normal distribution. Then, to estimate the price, you just take the average of the 5 prices you have. You can also report a 95% confidence interval: average ± standard_deviation(5 prices)/sqrt(5).

The other one is more complicated. Since you don't have much data, it's good practice to regularize (or control the complexity/capacity) of your model. Examples include Lasso regression, L2-regularized regression, support vector (SVM) regression. I would first try the SVM regression using libsvm.

-
Your first method is the same as fitting a regression model with no explanatory variables, just an intercept. –  Peter Ellis Aug 8 '12 at 19:58
I like the idea of the first method. It is a bit challenging to truly implement it because every house is different (different square footage, lot size, # of bedrooms, etc... The second set of method are for the time being out of my league. I am also unclear why they would work with so few datapoints when simple linear regression can't handle that. For the record, I found another alternative. I simply run an optimization with different weights on each of the variables to generate estimates as close as possible to the home sales prices (reducing the sum of the square of the residuals). –  Gaetan Lion Aug 8 '12 at 23:27
I don't exatly know what your alternative method is but regularized model fitting is probably not more complicated than it. –  emrea Aug 9 '12 at 3:22

Adding duplicate data points will not even make the parameters estimable. Just think of it this way. It takes two points to fit a straight line and with two points you get a perfect fit. You have no information to estimate the variability in the determination of those points. Generally we assume that x is known and y is observed with some random error. Now repeat those two points 5 times or even 100 times. It does not matter. You gain no information about the variability of y. All the additional points are artificial and you still get exactly ythe same line with exactly the same fit. There is not new information. There would be if the new observations were really independent. They the data would be suggesting that y is observed with no error. Yet you would still want intermediate values for x so as to see if the function really is linear.

Even if you have more points than variables that does not really solve your problem. Yes it makes the parameters estimable and you may even have an estimate of the random error component from the residuals. But the estimates will not be accurate. You should have a lot more observations than variables in order to obtain a good model fit.

-
I agree that you should have a lot more data but what if you cannot have more? I believe that regularized model fitting is still applicable in these cases. –  emrea Aug 8 '12 at 17:57
Reducing complexity is always helpful when there is not much data. But isn't the easiest way to reduce complexity to reduce the numberof variables in the model. The OP was not suggesting cutting out any of the 5 variables. he only suggested adding data and in a very artificial way that does not help. –  Michael Chernick Aug 8 '12 at 18:03