# Removing outliers and calculating a "lowest" attainable price from a pre-determined/fixed time series of prices

Just a foreword, I'm not a mathematician or otherwise statistically skilled. I know my way around calculating standard deviations, but it's all self taught. I'm a programmer with limited stats knowledge.

Now, Assume that you run a "Widget Rental" website, and on your application you allow owners of "Widgets" to rent them out. You let the widget owners create seasons, (such as May - June, and August - September ), and then pick a price that it would cost to rent out the Widget during that time period.

With the prices that are entered, I want to be able to :

• Sort all widgets by "lowest price" available, so low-to-high in a result list of all widgets.
• Provide a "From price" for a widget. That is to say, "You can rent this widget for $XXX per day". Each widget can have a different price based on the time of year. Some widgets will have dozens of different prices depending on the season as you get "high" seasons and "low" seasons. However, the sellers of the "Widgets" are especially mischievous, and have realised that if they set their widget to be really expensive for one day of the year, and also really cheap one day of the year, then they can easily appear at the low and high sort ranges. And they are also using this to manipulate their "from Price". Currently, I took a very naive solution in order to calculate the "lowest price" for a Widget, which is to just take the lowest( N ) value from a dataset. What I would like to is to get a "lowest from price" for a widget, which accurately portrays the price which it could be rented from.. and remove the lower/higher-band outliers. Take a look at this chart... with values... X Axis - Time (each significant interval is a day) Y Axis - Price The X axis is time, and the Y axis is the price. Now, this contains a normal distribution, and there aren't any real statistical outliers in that dataset. It's common to see the price between the lowest value and the upper value to fluctuate as much as 200% or a few standard deviations. However, take a look at this second chart... It contains a single day tariff, which is only 20 ēuros... I've played around with using Grubbs test and it seems to work quite well if you have a small dataset and don't take into account the duration of a date range. However, once you start trying to throw in a 3 days for a widget which is$1, it doesn't work well for removing the 3 clear outliers...

The important thing is that I want to get a "from price". That is to say, I want to be able to say, "You can rent this widget from XXXX". So it should be reflect the overall pricing taken as a whole and ignore clear outliers.

I'm going to implement this in PHP, so I don't have any amazing statistical tools at my disposal other than the core math libraries.

One scenario in which the median is less useful is a comparison between (A) a widget you can rent for 10\$a day for 182 days of the year and 100\$/day for another 182 days and 50\$/day for the last remaining day and (B) a widget available for 50\$/day all year round. The median price per day is the same but the actual price will differ depending on the period you want to rent the widget. Of course, you could create similar scenarios for the mean or the Winsorized mean. Fundamentally a single number cannot always capture the whole price distribution or correctly reflect each potential renter's situation.
Yet another solution would be to order widgets using the highest end of their price range (i.e. starting with the widget with the lowest highest daily price – hope that makes sense) or display the whole range (“Rent it between x and y\$/day”) or a little graph so that “cheaters” can be spotted immediately by the users. Basically, deal with this as a design problem rather than a purely statistical one. • Thanks for your input. I like the winsorized mean idea. This would definitely work better than a median value. I have thought about using rent between $X - \$Y.. will give more thought. Commented Jul 5, 2013 at 11:33