History doesn't repeat, it rhymes: challenging the role of data historians in predictive analytics

As the world continues to become more data-driven, it’s no surprise that organisations are turning to predictive analytics to gain insights and improve their operations and maintenance. Utility companies are starting to investigate the benefits of using predictive analytics to optimise energy usage, reduce maintenance costs and improve overall asset reliability. However, many businesses get lost in the hype of technologies, solutions and approaches that seem to make the same promises but in very different ways. One such example is the ongoing debate around the value of data historian technologies in enabling predictive analytics.

I want to challenge this notion and discuss why a data historian may not be as necessary to predictive analytics as some might believe.

First, let’s define what a data historian is. A data historian is a software system that collects, stores and retrieves time-series data from various sources. This system enables businesses to analyse historical data trends over time. Data historians are an old technology that has been around for decades and were an enabling component of many early data analytics applications. However, with advancements in cloud-based technology for data collection, processing, storage and analysis, the role of the data historian is diminishing.

One of the main arguments for using a data historian is that it provides a complete historical record of system data. However, with the significant cost reductions in scalable cloud services, cost-effective, high-performance data processing and storage is no longer a constraint. Businesses can store and process infinite amounts of data in the cloud, which means they can access their complete historical data without the need for expensive data historian software and dedicated hardware.

Another argument for using a data historian is that it provides real-time access to data, which is essential for making timely decisions. However, modern industry cloud platforms provide real-time data acquisition and processing capabilities. These elastic industry cloud platforms can be deployed to scale up and down, as and when required. This means businesses only pay for the storage and processing services they require to run predictive analytics applications when they require them. This modern approach eliminates the dead cost of storing large volumes of data not required for predictive analysis.

Finally, a data historian stores yesterday’s data, which is of limited value to artificial intelligence-enabled predictive analytics applications. As Mark Twain said, “History doesn’t repeat, it rhymes”, and this is true of failures in process plants. Historical data is required to train predictive models, but typically this involves no more than a few thousand rows of data extracted from an historian. Once a predictive model is trained, it rarely uses historical data: rather, it relies on real-time or live data. For operators this means once we have accurate and trusted predictions that allow us to take preventative action, the historian should, in theory, never write data outside of normal operating parameters.

In conclusion, while data historians have played a role in many early iterations of data analytics applications, they are not as necessary to predictive analytics as some might believe. With the advancements in industry cloud technologies, businesses can build predictive analytics applications that process massive amounts of live data, providing them with the far more valuable view of what is going to happen tomorrow rather than what happened yesterday.

Luke Davey is a digital transformation specialist and manager of the digital enterprise solutions team at Yokogawa Australia. Luke and his team assist customers in adopting new and emerging technologies to improve operational performance, security, safety and sustainability.

Image: iStock.com/metamorworks