You are Home   »   News   »   View Article

Using graph databases in E&P

Tuesday, March 27, 2018

Graph databases, which use abstract data structures rather than rigid boxed data structures, could prove very useful in E&P, particularly working with complex data sets. Michelle Lim from PETRONAS explained.

Graph databases, which store data in a loose structure rather than in rigid tables (as the usual relational databases do), could prove very helpful in exploration and production, particularly working with the large complex data sets, said Michelle Lim, from the Digital Innovation, Strategy and Architecture department of PETRONAS.

She was speaking at the Digital Energy Journal conference in KL in October, 'Workforce of the Future: Improving Data Analytics & Knowledge Management.'

The word 'graph' means 'graphical', or arranged in free space (not a graph of the sort where x changes with y).

Data in a graph database is stored as 'nodes', and the database understands how the various nodes connect together as 'edges'.

The more common relational databases store data in tables, like data in a spreadsheet. They are much more rigid.

The development of graph theory is attributed to Euler, a mathematician, in 1736, who wanted a mathematical way to work out whether it would be possible to cross each of the 7 bridges in the town of Königsberg (now Kalingrad, Russia). He drew a map of the bridges and land connecting them as edges and nodes.

Today, graph theory is used by Google, to show how different pieces of information are related. This enables Google to present a wide range of different information related to the subject you are searching for (shown on the right hand side). For example if you search for Tom Cruise, you will be given different pieces of biographical information, quotes, movies, social media pages, and pages of former partners.

Graph databases are also used by Google Maps directions, to keep track of the different ways for getting from one point to another.

Facebook is also a large graph, where it keeps tracks of all kinds of relationships between people and things. Facebook is looking at ways to develop its social graph for workplaces, to help find the right people within the organisation.

The computer system can understand data similar to how a person would - for example there are certain pieces of data which could be connected to a person, such as their date of birth and their first school. Every company office should have an address


Graph databases were used to index documents by the North American Space Agency, showing how the different documents relate, she said. This can prove a much easier way to search for a document than by keyword.

In one project, engineers wanted to develop an 'up-righting' mechanism for a spacecraft, and they thought it was very likely someone had done this in NASA before, and wanted to see how they did it.

If they were searching through documents via keyword search, it could still take months to find the information, with tens of different terms which might be present in a useful document.

But with the documents connected in a graph, showing how they linked together, it was possible to find the relevant information in 4 hours, a significant cost saving from time saved.

The E&P industry has a similar challenge, with about 100 different data types. 'The more information we have, the harder it is to find information, this is our dilemma,' she said. 'When we have massive amount of data, they are sitting across disparate databases, that makes things even harder.'

E&P projects

Graph databases were used in the E&P industry to get a better understanding of well data and oilfields. The data could be visualised with green nodes for the oilfield, blue nodes for wells and purple nodes for the oilfield.

The data can be visualised with wells linked to the oilfield they are in, and wellbores linked to the wells they are in. This immediately provides a different and unusual way to visualise the company's wells, showing which fields are the largest.

This work can also show up problems or faulty data, for example if you have a wellbore which is not related to any well, or an oilfield which does not have any wells in it.

Graph databases could also be used to understand the software applications which the company is using, showing the types of data they work with. The data types are shown as green nodes and the applications are yellow nodes, and the data types are connected with the applications they work with lines.

This will generate a visualisation showing the most important data types, because they are used by many different applications. It can also help you spot where you two applications might be doing the same task because they use the same data types.

There are thousands of different software applications being used across PETRONAS and big savings if some of them can be taken out of use.

The technology might also be helpful in project management, when many different data types need to be bought together to better evaluate options for developing a field.

Graph data can also be useful for more complex data search systems (sometimes known as 'cognitive search') because it makes it easier for a computer to pull data from different data storage systems, and the graph database can better understand how the data connects together.

As the amount of data grows, and there are more different data storage systems, it is getting harder and harder for companies to get the right data together.

The data can be stored in different places - each 'node' in the database can be a separate data store.

Building it from documents

The work of building a graph database for documents would involve taking some of the main topics from the documents, and then mapping how the documents connect together.

Ultimately it could include millions of documents (as nodes) and connections between them (as edges). But you don't have to include all the documents to make a useful graph database - you can start small, say with 100 documents, and then build from there.

It usually requires domain expertise to know what fits together with what. 'Humans are always required,' she said.

Associated Companies
» Digital Energy Journal
comments powered by Disqus


To attend our free events, receive our newsletter, and receive the free colour Digital Energy Journal.


Exploration Data Management : North Borneo Grid
Wong Teck Hing
from Sarawak Shell Berhad


Latest Edition June 2019
Jun 2019

Download latest and back issues


Learn more about supporting Digital Energy Journal