# Evening out a set of buckets

(I am not strong in statistics so please forgive me if I don't use the correct terms here)

Let's say I have a book with many (1000+) sections, and each section varies significantly in the number of words it contains. My goal is to divide the sections across a period of time (let's say 150 days) where the sections are read in order, but there is as little variance as possible in the number of words read per day.

For example, I need to read every word of a giant tax code book in three months. It's broken into sections that vary from a few sentences to multiple pages. I only want to read complete sections, and I want to spend as close to the same amount of time reading every day as I can.

I've written a script that works like this:
1. I divide the sections evenly across the number of days without regard to words per day. For the remaining sections, I select an evenly spaced collection of days and add a section to each one.
2. I then calculate the actual number of words that ended up in each day, calculate the average, and then select the day that is furthest from the average.
3. Based on that day's word count, I evaluate the standard deviation of the word counts based on 3 scenarios: a) the first section on that day is moved to the previous day, b) the last section on that day is moved to the next day, and c) no change. If a or b results in a lower standard deviation, I keep that change and go back to step 2. If not, I move to the day with the next highest difference from average in words per day and repeat step 3.
4. Eventually the algorithm will be unable to find a change that improves the standard deviation after running step 3 on each day in the set. At this point the script is finished.

I've tested this, it works and provides a set that is smoother than the result of step 1. But I'm not finding the optimal distribution of sections this way ... I still have big peaks that can be fixed manually.

My next thought was to allow for more complex scenarios in step 3 ... looking back/forward more days or moving more sections. But I'm not sure this is going to work.

Is there a name for what I'm trying to do? Has this problem already been solved? Is there a better approach than mine? (Practically speaking my script takes too long to run in real time on a website) I had no luck with searching, but I'm not even sure what I'm searching for. Has someone already created an algorithm to do this? Any better ideas? I'm not sure how to tag the question because I'm just not familiar enough with the terms.

• This is an interesting problem but not really a statistics problem, although your algorithm uses some statistical ideas; it's more of an algorithms problem. You might try asking it at cs.stackexchange.com or programmers.stackexchange.com . Commented May 4, 2015 at 18:20
• @MichaelLugo I started to wonder that after I'd read my own post a few times. I'm not sure though ... I don't feel like it's something that can only be done with scripting. Commented May 4, 2015 at 21:17
• Can you show an example (as small as possible) that has the features you describe (i.e. that your algorithm leaves peaks you can fix by hand) and stats how much improvement in s.d. you get when yo do the improvement? Commented Apr 26, 2016 at 23:07
• Have you thought of using the A* algorithm? I'm not sure there is a deterministic way to solve you problem, but a heuristic algorithm that uses standard deviation as the scoring-metric seems like a good way to go Commented Apr 27, 2016 at 5:12