You are Home   »   News   »   View Article

Microsoft perspectives on oil and digital

Thursday, February 25, 2021

Kadri Umay, principal program manager energy industry with Microsoft shared his perspectives on how the oil and gas industry can make better use of digital technology, and what Microsoft is doing.

Kadri Umay, principal program manager, energy industry, with Microsoft, shared his perspectives on how the oil and gas sector can make better use of digital technology, speaking at the PIDX International Virtual Spring Conference in April.

Mr Umay is part of the energy industry team for Microsoft's Azure cloud computing service, and leads some of its data initiatives.

Mr Umay remarked that the Covid-19 means that industry is effectively having a 'digital transformation enforced on us'.

Digital transformation 'is no longer a project run by the office of the chief digital officer (CDO),' he said. 'We have to do it today. It has become a survivability thing.'

The oil and gas industry is also being pushed by both financial pressures and demands of the energy transition to digitise faster.

The drive to reduce emissions, and need to work with lower margins, is likely to push companies to be making much more real time decisions, he said. For example, if emissions somewhere are higher than anticipated, changes need to be made much faster, not after waiting a year for an emissions report.

'If you have a new service, or new software or new hardware that can provide you competitive advantage or an opportunity to reduce cost, you should be able to use it tomorrow,' he said.

Data quality

Data quality continues to be a big problem - and also only a small percentage of data is used in decision making. We can still consider ourselves 'wildcatters' in the data arena, he said.

Mr Umay cited a study from EMC saying that 'only 3 per cent of the potentially useful data is tagged.' If data is not tagged, it can't be found, and 'we can't run our digital workflows on the that data,' he said.

Mr Umay has been looking at 'patterns of inefficiencies in data management.'

'Finding [data] is the first inefficiency that we see,' he said. People in oil and gas companies might need to use 60 different search engines to find data they need, because the data is all stored in different places, or made from different applications.

Then the second problem is making the data available for your project. 'Some of it is sitting in Joe's desktop, some in a data centre in Australia, some in cloud servers in Singapore. You need to copy all this data in your cloud storage area before you start to process,' he said.

Then data can need a lot of processing and exporting before it can be used in for example a subsurface interpretation workflow. 'On a good day - it takes half of usable time of a geoscientist,' he said.

These problems were mentioned in a book written back in 1992, 'Enterprise Architecture Planning' by Steven Spewak.

Structuring data

For one client, Microsoft decided to look for a way to structure well data. It defined some data types (about the well and wellbore, well trajectory, well logs and documents), a set of data schemas defined in JSON, and released a demo.

'After the demo we had a lot of people interested in working with us and committing resources,' he said. 'We took all these schemas and operationalised them, professionalised them.'

It has a similar project with seismic data.


Microsoft is involved in the Open Subsurface Data Universe (OSDU) project, which aims to develop a standard data 'platform' or structure for all subsurface and wells data, so it can all be easily searched. There are over 200 members.

The OSDU defined a number of standard application programming interfaces (API) using REST (Representational state transfer) constraints, which define how software should be structured for ingesting, searching and delivering data from this depositor.

It has also defined JSON (human readable text) based schemas for metadata.

So you can have different data formats but with the same metadata schemas.

Anyone can build applications with these APIs, so it can connect with data stored in an OSDU structure. Some software companies are already doing this, he said. There could be something of an 'app store' where people can buy different apps which work with OSDU.

There can also be a role for Systems Integrator companies, which offer services to connect systems together.

Many customers are using 'power apps', such as dashboarding and data visualisation applications, such as Power BI and Spotfire, or analytics platforms like Azure Synapse.

It should be possible to plug in data in OSDU format into all of these applications. Data scientists and geoscientists need spend much less time on the 'plumbing', or 'data janitorial work'.


Companies might want to look at what he calls 'cloud native architectures' - software designed initially for the cloud, rather than software moved to the cloud.

They might want to explore managing data in data 'objects' rather than in proprietary file formats, known as 'object storage'.

'We can store any size of data in object storage,' he says. 'It's very low cost and very scaleable. It makes life super easy.'

No SQL databases can be used to store and read data 'any way we want', in a way which can be scaled to any size.

Having 'serverless architecture', where data storage is managed by the cloud, rather than by using specific server machines, makes it easy to be scaleable to any size. You can use gRPC (remote procedure call) to connect services between different data centres.

A 'microservices' based architecture can 'provides tremendous simplicity.' It is resilient and easy to upgrade he said.

Companies can be looking at better ways to gather data from sensors and developing 'edge' services (with computation done next to the sensor, so it does not have to send all of its data to the cloud).

An interesting emerging technology is 'secure multi-party computation', where data can be shared with others but kept encrypted the whole time. So you can run the same machine learning algorithm on different data sets owned by different people, encrypted with different keys. The algorithm can 'see' all the data, but no human can. 'Machine learning algorithms get better and better when you share data,' he said.

The Graph QL standard, a query language for APIs, could be useful. 'Being able to acess data with an API that makes sense for your use case is important,' he said. 'I think Graph QL will gain popularity in the next few years.'

Technologies like Secure multi-party computation might be useful in optimising scheduling, if it enables companies to share their requirements to a scheduling algorithm working across multiple companies, but without revealing secrets to their competitors.

Mr Umay mentioned Microsoft's Project Silica, which enables data to be 'written' onto a piece of glass, enabling data storage for 10,000 years. He showed a picture of a piece of glass a few inches square, which held a 76 GB movie. ''This technology is virtually impossible to destroy,' he said.

Associated Companies
» Microsoft Oil and Gas
comments powered by Disqus


To attend our free events, receive our newsletter, and receive the free colour Digital Energy Journal.


Latest Edition Jul-Aug 2021
Jul 2021

Download latest and back issues


Learn more about supporting Digital Energy Journal