Digital Energy Journal- The data analytics funnel

Join | Existing Member Login

You are Home » News » View Article

The data analytics funnel

Wednesday, April 22, 2015

The way the oil and gas industry works with data can be depicted as a funnel, with raw data going in one end and usable data coming out of the other.
By Ketan Puri, enterprise architect for big data, Infosys

I would like to depict oil and gas data analytics using a funnel.

Just as a funnel is used to channelize fluids and do processing like filtration, the data funnel depicts the controlled processing of data.

The data flows through the funnel, get channelized as desired and processed using various analytical models. The data can be simultaneously be fed in to other funnels for further processing.

Data cannot be analysed in its raw form. It needs to undergo lot of pre-processing before real analytics techniques can be applied.

Pre-processing may involve multiple operations on the datasets like rule based extraction, aggregation, splitting, transformation, filtration, truncation, noise reduction and much more depending on the business need.

The data logically need to traverse through multiple funnels before it can be analysed. This journey of the data from it raw form to analysable state is called Pre-processing.

The analytics funnel can be applied to streaming data (data in motion), staged data (data at rest), or to the data from various other funnels. The funnels can also re-stream the data and apply advanced analytics models. It all depends upon how we configure the properties of the funnel.

From purely analytics perspective we can divide the analytics funnel into three zones of data exploration / discovery, data modelling and analytics,

Data Exploration / discovery

This zone provides ability to explore the data at rest and identify data patterns. It allows the user to connect to multiple data stores with ability to plug and play various visualization tools. The key objective of this zone is to gain an understanding of the data sources, data sets and discover patterns that can help the business to gain deeper insights.

Data modelling

This zone leverages the understanding of the data from the exploration and discovery zone and extract the key patterns that help the business to meet their need. These patterns are used to create analytical models.

The models can be estimation models to better understand the business processes, or predictive models to forecast business behaviour under given set of parameters or prescriptive models to recommend actions and proactively respond to business situations. The key objective of this zone is to develop analytical models catering to a specific business need and identify associated data sets.

Analytics zone

In this zone the analytical models developed in the modelling zone are executed. The process of running analytical models on real data is called Scoring. The models can be deployed on the data lake to extract value from both the streaming data or from the data at rest.

This zone also provides the ability to perform pre-processing of the data and responding to the business with the insights. The key objective of this zone is to provide business direct access to the insights for enabling better decisions.

Finally the insights are fed back into the data lake providing business a better understanding of their data. The insights are then validated against the business requirements. This may lead to the discovery of more data sources, new data entities and refinement of the models.

The process repeats continuously churning and refining the data in the data lake. In due course of time with substantially large data sets these models evolve into predictive and prescriptive analytical models directly feeding into the business processes optimizing the processes and adding value to the business in multiple ways.

Processes

Here are some oil and gas processes which use a data funnel:

Oilfield Exploratory Analysis provides seismic data analysis creating the basis for exploratory drilling

The appraisal management process provides key inputs from the reservoir data analysis leading to drilling appraisal wells

The drilling and completion process optimization provides preventive techniques for blowouts, determining well bore trajectories for maximum reservoir coverage and avoidance of potential NPT.

Other processes include oilfield production optimization and performance forecasting techniques.

Associated Companies
» Infosys

CREATE A FREE MEMBERSHIP

To attend our free events, receive our newsletter, and receive the free colour Digital Energy Journal.