Digital Energy Journal- Hadoop for oil and gas

Join | Existing Member Login

You are Home » News » View Article

Hadoop for oil and gas

Thursday, April 30, 2015

Hadoop, a new data storage technology, promises to make high volume data much easier to manage in the oil and gas industry. Alyssa Farrell of SAS explains how it works

Hadoop is a new data storage technology which may lead to a big change in how the oil and gas industry manages big data.

High-volume data, whether structured or unstructured, can be placed into an enterprise data hub (EDH) on Hadoop and prepared for further analysis without the time-intensive steps associated with the traditional relational database model.

With a Hadoop 'cluster', both structured and unstructured data can be managed for use over long time horizons, in original fidelity, and integrated with existing infrastructure and tools.

With Hadoop, organizations have a new way to think about data, transforming it from a cost to an asset.

Geophysicists are already evaluating Hadoop as a technology to support processing seismic data from a multidimensional perspective, complementing the traditional processing sequence.

'New technologies that reduce the latency of data for analysis, such as cloud applications and Hadoop, are significant game changers for the oil an dgas business,' said Moray Laing, SAS Executive lead for Oil and Gas, formerly at Baker Hughes.

'As a result, this industry is on the precipice of major change in their IT architectures.'

Big data trends

Recently, The Data Warehousing Institute (TDWI) released "Managing Big Data," a report that explored trends in big data management.

The report presents the findings from a survey of more than 400 practitioners about their big data efforts.

Respondents were asked which database management systems (DBMS) were in use for big data management efforts.

While traditional relational DBMS systems were at the top (38 percent), Hadoop was tied for second place (33 percent).

In addition, it was clearly evident that the phrase 'big data' was synonymous with Hadoop, in the minds of those surveyed. A meteoric rise for a technology that only became commercially viable in the last five years.

For many organizations, establishing an enterprise data hub using Hadoop will be a cost-effective solution for data capture of all data, structured and unstructured, in a secure, managed environment. When paired with additional technology applications to ensure data quality, and to visualize and analyze the data effectively, Hadoop is ready for prime-time.

Software companies, like Cloudera and SAS are working together to provide processes and technologies that accelerate data-driven insights.

Dave Cotten, whose team at Cloudera supports many US oil and gas companies, says that 'Cloudera's oil and gas clients are realizing multiple revenue generating and cost savings opportunities.

'From real-time field operations feedback improving reservoir yields, to full-fidelity electronic well record management, to mining internal and public data to determine optimal well spacing, customers are obtaining deeper insights at lower costs provided by Hadoop in an enterprise data hub.'

'In addition, our customers typically improve preventative maintenance, greatly reducing costly downtime. '

Analytics on Hadoop

SAS and Cloudera recently announced technologies that move the analytic functions directly within a Hadoop cluster.

Deploying models directly in Hadoop reduces data movement and replication, saving time and resources - while strengthening data governance.

With all your data in one place, simple tabular data can mix with more complex and multi-structured data to provide business insights never before possible.

Organizations can run a variety of enterprise workloads, from batch processing to advanced analytics, in a secure, managed, governed environment.

Early adoption of analytics on Hadoop has been popping up in seismology, asset optimization, commodity pricing strategies, and production optimization.

Because SAS data visualization on Hadoop allows companies to interactively explore billions of rows of data in seconds.

One common use case is for data validation, finding the outliers and flagging them for further explanation.

Because you can look at both structured and unstructured data, like Twitter feeds or web traffic, in one place over time, there are applications for cybersecurity as well.

As a platform for innovation, an analytics strategy that leverages 'big data', opens the door to new business models, partnerships, and information-sharing for oil and gas companies around the world.

Associated Companies
» SAS Global Oil and Gas

CREATE A FREE MEMBERSHIP

To attend our free events, receive our newsletter, and receive the free colour Digital Energy Journal.