In my experiment, individuals assign probabilities to the likelihood of future events, and update their forecasts as frequently as they like. Most questions stay open (receiving new forecasts) for months. Forecasters are placed in different experimental conditions. Some forecast in isolation, some can observe forecasts made by others, and a final group can not only see other's forecasts, but is assigned to one of twenty-five teams of 15-20 people each, and encouraged to share information with their team.

One of my goals is to characterize social influence. e.g. "If an average person raises their forecast 10%, what effect does it have on their teammates subsequent forecasts?" But their are many other things to consider: if the forecast change is accompanied by a verbal justification, how does that change its influence? If the forecast change moves the forecaster closer to the team mean, does that (as I would expect) imply it will have less influence on subsequent teammates' forecasts than if it moves the forecaster further from the team mean.

I think it would be interesting to compare forecaster influence with forecaster accuracy. I assume they are positively correlated, but how strongly. Can we distinguish good/bad influences in terms of their effect on accuracy?

Accuracy can be measured easily enough using something like the brier scoring rule. But estimates of social influence will inevitably rest on some pretty heroic assumptions. How can I distinguish between a team reacting to a previous forecast, and a team reacting to some piece of news which also affected the previous forecast?

My questions for the crowd are as follows:

1) Given that I have this interest in influence, how would should I go about estimating it? Might I take advantage of random assignment to teams and/or to forecasting conditions without teams at all?

2) What other sorts of questions might I try to answer with this data?

  • $\begingroup$ I am interested in the result of your experiment! $\endgroup$ – qoheleth Jul 31 '14 at 4:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.