Are experts harmful? I'm reading "The Role of Chess in Artificial Intelligence Research" (pdf) and  interestingly, it says:

Experience [...] suggests that inputs from chess experts, while generally useful, cannot be trusted completely. 
A good example of this is Deep Thought's evaluation function. Several changes by capable human chess experts failed to produce significant improvements and occasionally even affected the machine's performance negatively. 
Here, human experts, along with their expertise, introduced their own
  prejudices into the program. One way of solving this problem is to
  limit the type and the amount of expert inputs allowed into the
  program; in other having an almost "knowledge-free" machine.



*

*How true is that in modern research and practice?  

*Is that a big problem, or just something specific to the game of chess?

 A: I think the key is consistency. The expert has not only particular knowledge, but a system within which that knowledge operates. They have a persona, an overall strategy, within which their tactics reside and evolve.
In some sense, a computer program playing chess is a Frankenstein's Monster created from a mashup of various bodies (programmers, experts, etc). So it's not surprising that any one expert's advice would not fit well with the system that exists.
I agree with other comments that experts may not know how they do what they do. In which case, being human, their conscious mind makes up a plausible story as to why they reached a particular decision. But I still think that expert advice to the programming team is always out of context (i.e. inconsistent with the context of the program's design and history).
EDIT: There may also be a Reinforcement Bias here. I can't find any good links to explain Reinforcement Bias, but the way I understand the term it's the effect you get when you update (refit) a supervised model using the previous results of the model -- usually indirectly -- as targets. It's similar to Confirmation bias, but it involves a level of indirection. Human experts would have their Reinforcement Biases, which could affect things.
A: Such issues are common in various areas requiring judgement.
The chapter on "Judgemental forecasting and adjustments" in Makridakis, Wheelwright and Hyndman Forecasting: Methods and applications has similar stories of expert judgement under-performing sometimes even very simple systems.
There's a paper (Dawes et al (1989) "Clinical vs Actuarial Judgement" Science, Vol 243, No 4899, p1668-74) about the failures of expert judgement in the medical area against what it calls 'actuarial' methods - basically fairly simple statistical models.
On the other hand there's a paper in the actuarial literature about the 'noisiness' and inconsistency of expert judgement in a particular problem in that area where expert judgement is often regarded by its practitioners as of paramount importance.
Makridakis et al discuss failures of expert judgement in many areas, as they relate to forecasting, and contains quite a bit of valuable advice.
And so on it goes. Cognitive biases abound and human experts suffer from them along with everyone else.
A: The question of whether or not experts are harmful is certainly not a problem restricted to the game of chess. 
An interesting question in the literature regarding the optimal design of Monetary Policy Committees (MPC) is whether or not committees should include outside experts who are not full-time employees of the central bank. 
To put this into perspective, first consider the Bank of England's MPC. It is composed of five internal executive members of the bank and four outside experts. On the other hand, the Federal Reserve Bank employs a committee composed solely of bank employees.
Outside expert members are included in the Bank of England's MPC as they are believed to bring in expertise and extra information to that gained inside the Bank of England. 
So, which MPC design is better? Experts in, or experts out? 
Well, this area of research is still active and it has been investigated recently by Hansen & McMahon (2010). I suggest consulting the references mentioned in this paper for further reading on this issue of "committees of experts".
Is this a big (important) problem? Taking into consideration the effects that a MPC's decision may have for the economy, I'd say this is a pretty important problem!
Lastly, I ought to mention that monetary policy decisions can, in theory, be delegated to a computer. For example, the computer could be programmed to implement, say, a simple monetary policy rule; for instance, one of committment. This would remove expert input after the monetary policy rule has been programmed into the computer. The use of the computer in monetary policy is mentioned in Svensson (1999).
Reference:
Stephen Eliot Hansen & Michael McMahon, 2010. "What do outside experts bring to a committee? Evidence from the Bank of England," Economics Working Papers 1238, Department of Economics and Business, Universitat Pompeu Fabra.
Lars E.O. Svensson, 1999. "How should monetary policy be conducted in an era of price stability?," Proceedings, Federal Reserve Bank of Kansas City, pages 195-259.
A: I think this is more about Engineering Problem Solving. Most successful engineering projects do not duplicate expert reasoning or or the expert's nature exactly. They solved the problem in a different way. 
For example washing machines use a different technique than humans, airplanes use different dynamics than birds.
If you are duplicating Expert Reasoning, their input is everything. But if you are solving same problem using different techniques (fast search, huge memory ...), their input is only helpful.
