How to best estimate the time remaining for a variable-length questionnaire? The greatest gain of the statistics classes in both school and university seems to be that I now have an inkling of which QA site to use for this question. :)
I'm a programmer and I'm making a questionnaire with a financial background. There will be many (probably over 100) questions, which will be presented to the user one-by-one, like in a wizard. Since it answering them all will be quite time consuming (likely over an hour), it would be best if the user interface could show the estimated time remaining - both for the current question and all remaining together.
As there is no way of determining beforehand how long a question will take, my idea was to gather statistical data about it (especially during alpha/beta stages). The naive approach would be to just average the times taken for each question, but (thanks again to the above mentioned statistics classes) I'm aware that this isn't really the best way to do it. There are all kinds of normal distribution curves, error calculations, approximation methods, and other arcane mathematical artifacts that I could not even begin to dream about.
Some more information:


*

*Questions are of very varying lengths. There are simple yes/no questions, and there are questions that will make you scratch your head. There are questions that ask to enter your contact details, and there are questions that require to meticulously fill a 30-row table. Some questions will ask you to calculate stuff. Some questions will require you to fetch some paperwork from your employer or something. Which brings up another point:

*The whole process can be suspended/saved and resumed at a later time. This shouldn't happen very often, but given the length of the process, and that some people don't pay attention to the pre-requirements, it can happen.

*Some questions may not be applicable based on answers to previous questions. For example, there might be a row of questions which are only applicable to businesses and not private persons.

*Some questions are essentialy duplicated based on previous questions. For example, if you entered that in the past year you worked for 4 employers, you will need to enter the same data for each of them.

*I can detect:

*

*When the user starts/finishes a question. Hence, I can calculate the duration.

*When the user goes idle. This is rather imprecise. I can only detect when there is no keyboard/mouse activity. This can be because the user has gone away, but it can also mean that he is digging through the papers looking for numbers. Or (since it will be a webpage), maybe he simply switched to another program - or another tab in the browser.

*Which questions have been answered, which definately will need to be answered, which definately won't need to be answered, and which might need to be answered. For the last type of questions, I'm not sure if it might be possible to calculate a meaningful probability based on previous answers. Perhaps.



So... any ideas on how to tackle this estimate? Remember - I'm going for maximum accuracy (or is it precision? better make it both), and would rather have better estimates than lower resource usage.
 A: I do not see why averages should not work.  You might want to use winsorized or trimmed averages  to  guard against outliers, which are likely due to inability to tell exactly whether user is idle or not. Then for the estimated time remaining show the sum of the averages for each question. You could build more complicated and precise methods if you collected data on the completed time together with user characteristics. For that though you will probably need another questionaire, so unless you are expecting a lot of feedback in alpha and beta phases, I suggest using the simple averages. Sometimes simplest methods work the best.
A: As @mpiktas points out, simple is good.  But I can't help thinking there will be substantial variation among users.  Roughly, we can conceive of the time to complete each question as depending on the question and on the user, so the trick is to tease out the question-specific timing factors from the experience of previous users and to estimate the user-specific timing factor from the user's experience with the questions they have already tackled.
This opens up the possibilities greatly.  A good answer would rely on exploratory analysis of preliminary data.  In general, though, a combination of (1) determining a suitable way to express question completion times (logarithms?  square roots?) and (2) robust analysis of variance techniques (such as median polish) or IRLS regression is likely to do far better than ignoring the user-specific factors ("simple averaging").
Once you have estimated question-specific timing factors, which could be as simple as computing median or mean times per question, you could easily estimate the timing factor for a new user by comparing their times to typical times for the questions they have already attempted.  For instance (this is truly simplistic, but gives the flavor of the idea) if they take twice as long to complete the first five questions as the sum of the typical completion times for those questions, you could estimate that this user will take twice as long as the average to complete the next 95 questions.  You can then add the average completion times for those 95 questions and double the sum.
