Digital Energy Journal- Can text summaries of data help people make better decisions than visualisations

Join | Existing Member Login

You are Home » News » View Article

Can text summaries of data help people make better decisions than visualisations

Wednesday, October 29, 2014

Research in the medical sector has shown that people can make better decisions when information is presented to them in text, rather than in graphs. Could this mean that if we present asset integrity information as text, people can make much better decisions with it?

Professor Robert Logie, a specialist in human memory at the University of Edinburgh, recently undertook a study in a neonatal unit to find out how doctors accessed information about a baby's health.

Doctors were asked what information they used when making a decision about the baby, including talking to colleagues, examining the patient, and using information visualisations from the computer system.

Then they were observed to see what information they actually used.

The research showed that although junior doctors claimed that they used the computer system, in reality they hardly ever used it.

The senior doctors, however, did actually use the data from the computer system.

A likely explanation is that it takes a great deal of experience to understand computer visualisations and to use them to make inferences about a baby's health, said Ehud Reiter, chief scientist with Arria NLG and also professor of Computing Science at the University of Aberdeen's School of Natural and Computing Sciences, speaking at the Digital Energy Journal Sept 24 event in Aberdeen, 'Using Analytics to Improve Asset Integrity'.

'A guy with 20 years' experience can understand what's going on - yes. The guy with 2 years' experience can't. People without much experience struggle to understand graphs."

In a follow-up experiment, doctors and nurses with all levels of experience were asked to make a decision about what was wrong with a patient after being provided both with graphical data, and a textural summary.

"Almost across the board, these doctors and nurses made better decisions from the text summaries than from the visualisations," Professor Reiter said.

Oil and gas

Arria is applying this observation to the oil and gas industry by developing computer tools that communicate with people using language, in the belief that it can lead to faster and better decision-making.

When oil and gas engineers try to solve a problem, they are typically sitting in front of large banks of screens displaying graphs, to try to work out what is going on, Prof Reiter said.

"They might spend half a day just looking through the data trying to find the really important nuggets. Once they find those bits they've got to decide what to do about them."

But if the engineer is working with a colleague to solve a problem, they will probably be talking about it and discussing what the graphs are showing them.

Arria NLG's vision is that the computer will act more like the engineer's colleague and help him or her make faster and better decisions by explaining what it (the computer) thinks is going on using everyday language.

The software aims to make the more routine work easier. Some engineers might get 100 asset integrity reports a day and have to distil them into an incident report, a task which can take two weeks. "What we're saying is - we already have the data, and we'll get it out in a minute," Prof Reiter said.

One delegate pointed out that perhaps graphs could provide much more precision than language - different people can interpret a word like 'concern' in different ways. "There are some words [which can be interpreted in many ways] that we avoid, selecting only those where there is no ambiguity in their meaning," Prof Reiter said.

Is medical research relevant?

One audience member suggested that perhaps oil and gas engineers are much more comfortable working with numbers and graphs than medical professions are - particularly younger oil and gas professionals who have grown up with computer games that include many visualisations.

Professor Reiter responded that he thought hospital doctors were in fact very comfortable with numbers, and live in a similar world to engineers. "Yes, the doctor has to talk to the patient, but most of the time they are living in a world of numbers. So I don't think there's a difference in culture from that point of view,' he replied.

"In terms of today's youngsters, you'd think they would be more comfortable in terms of visualisations, but that's not what my colleague Bob [Professor Robert Logie] found. If you put people in front of complex graphics, even young people struggle with them.

"I think there is a big difference between a computer game and the kind of set-up where the engineers and doctors are surrounded by complex screens full of data."

Examples

Arria's aim is to extract the key facts which someone needs to know, taken directly from the data, and present them as though written by a human.

Alternatively, the information could initially be presented as a simple graph, with 'key-fact annotations' (similar to those included in graphs of political polls describing key events). People can then click on the annotations to see more detailed summaries.

Here is an example of a statement which Arria's system can generate based on oil and gas data:

'There was an Acceleration alert on FGC Gearbox Casing at June 23 2014 15:22. The alert has been intermittently active since June 22 2014 07:34. An analyst previously examined this alert during the intermittent period and did not turn it into a service. Casing Drive End Vibration was unstable from June 23 2014 15:11 to 15:23 with a mean value of 1.0 g. CGC was on during this period.'

'There was 1 closed service that had examined this alert. Service 1102 was closed on May 22 2014 20:06. An action was taken: 'Gearbox vibration noise is consistent to what is seen currently. Going to close this for now'. Over the previous 90 days, the alert was marked as No Action 130 times and was turned into a service 3 times'

Arria NLG

The company's founders have strong credentials. Chief scientist Ehud Reiter is also professor of computing science at the University of Aberdeen's School of Natural and Computing Sciences, and founder of the University's natural language generation (NLG) research group. He also wrote the book 'Building Natural Language Generation Systems' together with Dr Robert Dale, published in 2000 by Cambridge University Press. He has a PhD from Harvard.

Dr Robert Dale is chief strategy scientist and chief technology officer at Arria NLG. He joined Arria after 17 years at Macquarie University in Sydney, where he was professor in the Department of Computing in the Faculty of Science, director of the Centre for Language Technology and former director of the University's Microsoft Research Institute. Dr Yaji Sripada, chief development scientist with Arria NLG, is senior lecturer in Computing Science at the University of Aberdeen.

Associated Companies
» Arria Natural Language Generation

comments powered by Disqus

CREATE A FREE MEMBERSHIP

To attend our free events, receive our newsletter, and receive the free colour Digital Energy Journal.