Redefining the SCADA architecture to support substation flexibility

SCADA systems have been in use for many years at electric utilities. The basic architecture of the SCADA system has remained constant since the first systems were installed over 30 years ago, which were based on the most current information technology available. But as centralised information technology has advanced towards the use of decentralised PCs with servers, the utility’s SCADA architecture has not kept pace.

The challenge with the traditional SCADA architecture is the higher costs associated with new substations and limited flexibility when functional requirements, equipment additions and physical layout changes are required. As with most changes in the electric utility world, a valid business and operational case must be made to justify changes to designs and investments.

Current utility architecture

Today, the most common type of IT architecture is based on a central server located at the utility’s control centre that acts as the central core of the SCADA solution. Access to the real-time information, status and alarms of the utility’s substations is visible to control centre operators through local workstations (desktop PCs). However, non-real-time information, which is useful for analysing sequence of events (SOE), alarms and maintenance, is left in the relays (intelligent electronic devices - IEDs).

The real-time information is brought into a utility’s control centre through a variety of communications methods, including telephone, two-way radio, microwave and fibre. At the substation end of the communications connection is an RTU, which may be the data concentrator or may have an external concentrator. It is connected to the utility relays that control substation equipment such as breakers; the connections within the substation are generally done with a local area network (a fibre ring or copper cables).

Figure 1: Traditional SCADA architecture.

The technology opportunity

Looking back at the evolution of computing architecture, in the 1980s most computing was done on mainframe computers. The use of ‘dumb’ terminals was eventually replaced with PCs with the ability to do some computing; and today, we use PCs connected via wired and wireless connections to servers. Data can be transferred quickly with computing being done either at the server with a thin client PC or on the PC (a thick client).

The traditional SCADA utility architecture is similar to that of the mainframe computer, where all computations and memory exists in a central repository. In an age of cyber security attacks, the weakness of this type of architecture becomes obvious; having all the data stored in one location may enable easier protection of data and systems, but also significantly increases the devastation that can be caused by a successful attack. If a virus or other cyberattack knocks out the main SCADA system, the utility is left with no visibility into its operations.

The current utility SCADA architecture is now ripe for the same type of evolution that took place in computing - towards a distributed architecture. Utilities that embrace advancements in technology and act on this opportunity can realise the benefits that come through diversifying the computing and data while preparing the utility for a more intelligent grid.

Increasing visibility with local HMI

With the traditional architecture, there is typically no way to access the SCADA system at the substation. Using a laptop, data from modern electronic relays (IEDs) can be accessed using a port on the relay or through ports on the RTU. As a result, engineers that need access to alarm data, breaker operation history and SOEs (for example, the timing of breaker and backup breaker operations to verify timing settings) need to travel to the substation or utilise the main SCADA system. For security reasons, utilities may limit SCADA access, which then increases the complexity of reviewing data for standard engineering work.

As a result, some utilities have started to add local HMIs within the fence of larger substations to provide access to the data needed by engineers. The local HMI is usually constructed from a single board computer with no moving parts, including flash-only memory, HMI/SCADA software and a historian database; this is an interim step to moving to a fully distributed architecture.

While these systems can be blocked from operating breakers, a web interface can provide engineers with access to the HMI remotely. The HMI software is generally the same or similar to the software installed at the control centre, and an entirely different communications method from the primary SCADA can be used for these installations, including secure internet access.

A local HMI connects to the LAN within the substation and constantly retrieves and stores data from the IEDs. In the traditional architecture, only about 30% of this data is backhauled to the control centre through the RTU and consists of data needed for real-time operations. The other 70% of data available, which is not needed for real-time operations, is not captured - leaving behind a mine of historical information that can provide critical intelligence to engineers and maintenance planners.

Having a local HMI allows all of this data (both real-time and non-real-time) to be collected into a single repository with a common clock. The information provides the foundation for root-cause analysis, trending and predictive diagnostics, providing deep visibility and valuable insight into areas such as breaker operations and maintenance requirements - enabling greater reliability and efficiency.

Figure 2: Substation HMI architecture with local HMI.

Benefits of distributed architecture

The use of a local HMI enables a distributed architecture SCADA system, which presents an entirely new way to configure the utility substation with less complexity and more accessibility for engineers and technicians. It consists of local HMIs with full SCADA control and a central control room that provides visibility, command and control, and data retrieval by utilising the HMIs - providing the infrastructure for a smarter grid that delivers improved operational and financial results.

Figure 3: Distributed SCADA architecture.

The new architecture allows utilities to expand functional operations by preparing for smart grid functionalities without major reprogramming of a large SCADA system. From a physical layout perspective, this architecture allows for greater physical flexibility in substation expansions and eliminates the need for the RTU/data concentrator.

At new substations, control houses are often built to house the relays that control the breakers, allowing technicians to work on the relays in all types of weather - an expense that has become accepted but should now be questioned. Using a distributed architecture, a control building is no longer necessary because the local SCADA acts as a gateway, allowing for reprogramming of all points accessible to the local SCADA.

Therefore, the relays can go back into the breaker cabinets and be interrogated and programmed from the engineer’s desk - thus eliminating travel time to the substation. Without the need for the house, the panels and racks in the house can be eliminated, as well as the time and costs associated with pulling and landing multiple sets of copper relay cables.

And finally, standardisation plays a key role. The expensive custom SCADA systems with their high annual maintenance fees can be eliminated as a version of the SCADA software is installed on common Windows servers in the control room. This version of the software views the thick clients in the substation through traditional communications channels or newer ones as the utility creates them.

Figure 4: Substation physical layout expansion conflicts.

Improved insight and reliability

The use of the local SCADA as a thick client allows for simpler creation of the screens and point connections using the IEC61850 communications protocol, which helps decrease install time, and the screens can be easily replicated in the control room. The local SCADA will have high-speed storage of all events available to the relays, supported by an appropriate database.

The high-speed data historian can capture all the points and replicate the data into the control room at any time for analysis, or the analysis can be done via the web on the thick client at the substation from an engineer’s desk. The local SCADA can also be used to review the data from phasor monitoring units (PMUs) to detect anomalies on the transmission grid and transmit the data to phasor data concentrators (PDCs). It can utilise a common clock for all relay operations, thus properly ordering the SOEs of breakers without trying to account for time stamp errors.

Furthermore, the local SCADA puts computing power out into the field where the next two areas of evolution to the grid will occur: distributed generation and micro-grids. With the computing power put closer to the distribution assets, the system can utilise meter data without extensive backhaul and processing delays. It can automatically balance the substation’s loads and resources, and island the substation or sections of the distribution system automatically for continuous operations. For operations involving multiple substations, the control room SCADA system will oversee the coordination of activities.

Impacts on substation design and construction

The changes espoused here have a profound impact on the physical layout of the substation. Traditional substation layouts include a building to house the battery bank, IED relays, low-voltage breakers, RTU and communications equipment. These block buildings needed to be centrally located within the substation due to signal attenuation and voltage drops; the extensive cabling required for bringing each point from the breaker to the relays, the low-voltage AC power, and DC power can now be reduced to the LAN cables and the AC and DC power.

Reduced cabling eliminates the need for the cable trench system that occupies large areas and limits material locations, truck accessibility and expansion of the substation. Without the building and trench system, the substation also becomes less costly to construct and easier to modify for expansions and new equipment installations.

The few remaining items from the house, such as batteries installed in large cabinets with local air conditioning (if required), can be relocated should a substation redesign be initiated. The HMI and other communications equipment can also be located in free-standing cabinets, enabling a much smaller footprint.

Migration strategy

For utilities with an existing large, centrally controlled SCADA system, a planned migration strategy is required to evolve towards a distributed architecture system.

• The first phase will be to install the local HMI on single board computers with flash memory in each substation. The system can be completely configured but used only as the local HMI until these single board computers are installed in every substation; these can be accessed via existing large-scale EMS systems.
• The second phase will be to install redundant (or quad) servers in the control room operating in parallel with the existing central SCADA system.
• The third phase will be to complete the communications links to the HMIs. These can be done through any of the various secure communications methods currently available such as fibre, radio, microwave or even internet, but larger bandwidth systems are preferred.
• Finally, the HMIs will be converted to local SCADA with the functionality tested before going live. The old systems can then be used as backup or removed to eliminate any unauthorised access into the substations.

Servers in the control centre should be isolated from the enterprise data systems by unidirectional firewalls that allow data out of the control centre into enterprise systems but not data from the enterprise system into the control centre, as this will help protect against hacking and unauthorised use.

Conclusion

While substation SCADA architecture has not changed much since utilities first employed SCADA in the 1980s, computing technologies and architectures have evolved, making it possible for utilities to rethink how they design their substations and operate their electric transmission and distribution systems. By embracing a new distributed architecture with local HMI, utilities can take a significant leap forward towards a smarter grid.

The advantages of leveraging a distributed architecture are twofold: it increases the functionality of the SCADA system to allow for comprehensive data access by engineers and it allows for a much simpler and more flexible substation layout. Adding to the ease for migration, the replacement of legacy ‘mainframe’ SCADA systems no longer requires extensive consulting assignments and customised programing, and standardisation based on IEC61850 makes SCADA system configuration faster and more robust.

The changes may seem radical but utilities that can leverage the technologies available today can reap ongoing benefits, including reduced costs, greater flexibility and ease of use. The business case to drive this new distributed architecture off the drawing board and onto the grid is more compelling now than ever as utilities focus on improved operational performance and financial sustainability.