Digital Energy Journal- Oracle; analysing complex real data streams

Join | Existing Member Login

You are Home » News » View Article

Oracle; analysing complex real data streams

Thursday, March 21, 2013

Oracle is offering the oil and gas industry tools to manage enormous streams of real time data, using 'complex event processing', 'extreme' data management and advanced analytics technologies.

Automated tools to analyse data streams from oil and gas activities are getting more and more important, as oil companies reach the stage where they have more streaming data than their staff can look at.

Computing giant Oracle is aiming to help, with its 'Exalogic,' 'Exadata' and Exalytics' machines, which can do data management tasks much more quickly, and at the same time be easy to program, said David Hattrick, Director Mining and Oil and Gas Asia Pacific at Oracle, speaking at the Digital Energy Journal KL conference 'doing more with drilling data' on October 24th.

'The oil and gas industry has done phenomenal things,' Mr Hattrick said. 'But there's this last mile in data management. If it gets cracked, or when it gets cracked, will return to all of us a bundle of energy for years to come - that's the social benefit.'

Oracle is 'trying to automate analytical processes, trying to augment human beings to address a whole lot of issues [in the oil and gas industry],' he said. 'We are trying to go across all those disciplines, bring all of this information together.'

'The industry today is not really managing real time data in real time,' he said.

'It is getting beyond the ability of one human being to monitor what's going on, even if they have got a whole lot of nifty visualizations.'

'My understanding is one human being can probably manage about 4 or 5 wells. If you are talking about an unconventional field, you have got to have an army of people.'

The proposition I am putting to you here is about [a better way to manage] real time data, including structured data, unstructured data, 'modelled' data and contextual data.

'Bringing it all together, and providing it in an aggregated way particularly to senior management who are going to probably going to want to see a whole lot of financial data beside that operational data.'

And providing the data 'in the way that specific disciplines want to see it.'

'I sat down one day and counted over 50 disciplines involved in upstream. I think that's the core issue: it takes 50 plus disciplines to run an upstream business. So we've got to figure out very smart technical ways to get all those people working together.'

'There's lack of visibility across operational, technical and financial data.'

'We need a mechanism to analyse massive real time data streams in real time, and not just looking at it but actually have some intelligence in there, that is augmenting the human beings.'

If everyone can see all the data, they can 'make smart recommendations based on the totality of the knowledge about the asset, using familiar petro-technical collection tools, including traditional interpretation, financial and operational data, using widely accepted (open)standards.'

Engineered systems

Following Oracle's acquisition of computer hardware company Sun Microsystems in January 2010, Oracle sees itself as both a software and hardware company, he said.

'Where we are actually going and what we are executing on is what we call engineered systems,' he said. 'This is a complete stack of software and hardware to provide an integrated function.'

'In many ways, it completely obsoletes the need for low level integration, from systems integrators or in-house team, to come in and store software on a server connected to a database to a network, all that sort of stuff.'

'One of the earlier adopters of our database machine, one of the big banks in Australia, did a consolidation exercise on the Exadata machine. The job ran in 8 seconds, it used to take 4 hours ran in 8 seconds. They actually put a service call in, because they didn't think that the job had even run.'

'These machines are capable of things that organizations just can't believe.'

The Oracle Exadata is a server, data storage and network solution all at the same time, which means that when Oracle staff visit companies, they don't know who in the IT department they should talk to about it.

Proposed system

In the proposed system, the data is communicated from the well or production site using Energistics standards WITSML and PRODML. Oracle is not doing anything to change the way the data is acquired.

'Open standards is the only way the industry can sort this out, there's no one big block of solutions from one vendor,' he said.

The WITSML and PRODML messages and converted into Java messages before entering into the Oracle system. Oracle works together with Infosys Oil and Gas practice to do this.

Java is a programming language which is made available on an open source basis. It is owned by Oracle, because it was developed by Sun Microsystems, a company Oracle acquired in 2010.

The data is managed on Oracle's 'Exadata' and 'Exalogic' engineered systems, which run on Java.
Asingle Exadata machine could run the entire SAP database of one of the world's largest mining companies, Mr Hattrick says.

SAP is written in Java.

The software applications can run on Oracle's 'Exalogic' machine, and analytics can be run on its 'Exalytics' machine.

'The value proposition is data discovery, data analysis, 360 degree view and high value information,' he said.

Demonstration

Mr Hattrick showed a demonstration of how it could work, managing real time data from drilling.
A sensor monitoring hookload generates WITSML data, which is converted into Java for the Oracle 'complex event processing' software.

The data is visualized using a product from INT, a software company based in Houston.

Normally the hook load graph would show a steady increase (as more weight of drill pipe is added), and if it doesn't, that probably indicates a problem.

'We are getting the real time data through the WITSML feed, we are taking it into the Oracle 'service bus', and it's being converted into Java.'

'The complex event processor is basically doing 2 things, taking the raw data and filtering out noise in that data and doing pattern recognition across the data,' he said.

'The real differentiator of the complex event processor (CEP) is that if you understand how to program SQL, you can program the event processor which uses a language called CQL,' he said.

'CEP is the one that's doing the pattern recognition, and it's spotting these issues with hook-load in terms of extracting the pipe.'

'The end user can basically design the interface the way they want.'

'It doesn't need any hard coding or programming skills, you are not going to have bring in someone and pay him a large amount of money to develop an integration to some other key system source system that you want to be involved.'

'The end user is moving between the raw data coming out of the real time data stream, and the filtered data from the event processor.'

'The end user is basically setting up various interface the way they want to see it.'

'All of this real time data is getting stored and archived. This is another issue, you know a lot of real time data is either not being kept.'

'If you have got the data archived, you are increasing your knowledge base for pattern recognition and trends.'

The data display uses HTML 5, which means that you can easily display it on different devices, including Apple and Android mobile devices.

'It's taking advantage of the extreme performance in these engineered systems that we are developing.'

Spooks, banks, telcos

Mr Hattrick said that his theory of industry of adoption means that spies are the first to use any technology, an example being SAIC, which was one of the key digital oilfield service providers (until being acquired in 2011 by Wipro).

SAIC is also defined on its Wikipedia page as an 'American defence company'.

'So there's a big correlation between electronic warfare and digital oilfield. They have very similar challenges,' he said.

'Then [new technology] moves into financial services because you make a buck out of the innovation.'

'Then telecommunications companies start using it, because they know their IT is their business.'

Now the technology, which has been used by banks and telecommunications companies for many years, is being offered to the oil and gas industry.

Associated Companies
» Oracle Corporation

CREATE A FREE MEMBERSHIP

To attend our free events, receive our newsletter, and receive the free colour Digital Energy Journal.