How to build a regression model with just 5 datapoints with 5 or more variables? I am attempting to build a simple multiple regression in order to estimate the value of a home.  The 5 datapoints are 5 comparable home sales (similar in size, lot, location, # of bedrooms, etc...).
If you run such a model, the software will say you need one more datapoint than variables.  
So, how about if I take my 5 comparable home sales and duplicate them 4 times.  So, now I have 20 datapoints.  The software should be able to run the related multiple regression.
I anticipate that you will answer that this methodology is flawed because underlying all that you still have only 5 differentiated datapoints.  But, can you flesh out this rebuttal a bit further. 
Also, can you suggest any alternative method to resolve this constraint. 
 A: Adding duplicate data points will not even make the parameters estimable.  Just think of it this way.  It takes two points to fit a straight line and with two points you get a perfect fit.  You have no information to estimate the variability in the determination of those points.  Generally we assume that x is known and y is observed with some random error.  Now repeat those two points 5 times or even 100 times.  It does not matter.  You gain no information about the variability of y.  All the additional points are artificial and you still get exactly ythe same line with exactly the same fit.  There is not new information.  There would be if the new observations were really independent.  They the data would be suggesting that y is observed with no error.  Yet you would still want intermediate values for x so as to see if the function really is linear.  
Even if you have more points than variables that does not really solve your problem.  Yes it makes the parameters estimable and you may even have an estimate of the random error component from the residuals.  But the estimates will not be accurate.  You should have a lot more observations than variables in order to obtain a good model fit.
A: I'm curious as to why you've limited yourself to only five comparable sales.  If you're trying to estimate the price of a single house, then finding the five comparable sales and just averaging them makes a lot of sense.  This is the same as a K-Nearest Neighbors approach, which is the basic idea of a ton of machine learning algorithms.  But if you're trying to do regression, you're trying to estimate the cost as a (linear) function of your covariates, or inputs.  If you'd used all available data, you can make a much more robust estimate of what the parameters in your model will be, and your estimates will presumably be much better across the spectrum, not just for the single house you care about.
A: I can think of two ways. 
The simplest is to assume that the price of those comparable homes follows a normal distribution. Then, to estimate the price, you just take the average of the 5 prices you have. You can also report a 95% confidence interval: average ± standard_deviation(5 prices)/sqrt(5). 
The other one is more complicated. Since you don't have much data, it's good practice to regularize (or control the complexity/capacity) of your model. Examples include Lasso regression, L2-regularized regression, support vector (SVM) regression. I would first try the SVM regression using libsvm. 
