Humans less likely to return to an automated advisor once given bad advice

The common ‘chat bot’ that pops up on websites asking if you need help has now become standard on many websites. We either dismiss it or engage with it, but do we trust the algorithm that is aiding our experience and providing us answers?

A recent study by US researchers looked into that question and found that participants were less likely to return to an automated advisor if given bad advice over a human advisor under the same circumstances.

In their experiment, researchers Andrew Prahl and Lyn M Van Swol from the University of Wisconsin asked participants to forecast scheduling for hospital operating rooms, a task they were unfamiliar with.

In order to complete this task, participants were given help from either an advanced computer system or a person experienced in operating room management. During the seventh trial out of 14, participants were given bad advice from either the computer or human advisor.

The researchers found that after participants received bad advice, they rapidly abandoned the computer advisor and did not use the advice on subsequent trials. This ‘punishment’ for giving bad advice was not nearly as strong for the human advisor.

“This has very important implications because time and time again we are seeing humans being replaced by computers in the workplace,” said Prahl.

Past research has looked at trust of automation through simple tasks like warning systems; however, Prahl and Van Swol said they wanted to look at more sophisticated automation that actually makes predictions about future outcomes, and compare that to human advice.

Unlike previous research, the researchers did not find that humans or algorithmic advisors were generally trusted more, the only significant differences arose after the advisor provided bad advice.

“This research suggests that any potential efficiency gains by moving towards automation might be offset because all the automation has to do is err once and people will rapidly lose trust and stop using it. This is one of the few studies out there that really show the potential downsides of automation in the workplace,” said Prahl.

Prahl and Van Swol will present their findings this month at the 66th Annual Conference of the International Communication Association in Fukuoka, Japan, from 9–13 June.