Determinism in industrial ethernet:
A technology overview — Part 1

The benefits of using well-established ethernet and internet technologies in industrial plants are too great to ignore, but overcoming the requirement for real-time operation has led to some innovative approaches to how this ubiquitous technology can be used in process plants and factory automation.

Automation and process control technology has been undergoing continuous change for some decades now. The advent of digital fieldbus technology has enabled new innovative and more powerful devices with access to more information, greater accuracy and reliability, and lower wiring costs than previous analog current loop and direct serial technologies.

At the same time, established de facto standards in information technology (IT), such as ethernet and the TCP/IP protocol suite, are dictating changes in relation to the integration of industrial systems with business systems and opening up opportunities to apply these technologies in industrial environments. The introduction of ethernet into the industrial landscape allows for new flexibility of deployment and new transmission media (such as wireless), and established IT protocols over TCP/IP such as OPC and DCOM allow new data management applications to revolutionise the efficiency and sustainability of the plant as a business.

IT technologies, however, were never designed with industrial applications in mind, and so their implementation needs to be modified in various ways to make them suitable. Not only do hardware devices such as switches and connector technology need to be ‘hardened’ for hazardous and harsh industrial environments (this is a given), but the standard IT communication protocols are, in many cases, not suitable for demanding, time-critical applications (such as motion control).

The solution to these problems has taken a few different forms, with vendors developing different approaches and with standards being written to encompass those technologies that have established themselves in the market. In this article, we will briefly cover some of the more widely adopted industrial ethernet technologies.

The benefits of ethernet and TCP/IP

The benefits that ethernet and the internet protocol (IP) suite deliver that were previously unavailable in industrial environments are easily recognised as:

• Ubiquitous technology — The broad spectrum of applications and tools that already support IT networking technologies, and the well-established technologies for building large, managed, distributed and redundant networks at relatively low cost can now be applied.
• Simplified communication — The combination of ethernet and TCP/IP provides open, well-known technologies, such as HTTP, SMTP and FTP, which can be easily utilised to enhance communication and management in an industrial environment.
• Enterprise connectivity — The simpler integration with business IT systems to provide better plant intelligence and product management.
• Bandwidth — With speeds of 10 Mbps, 100 Mbps and 1 Gbps, and data payloads up to 1500 bytes, ethernet provides the ability to transfer data that could not previously be transported in industrial environments, and delivers data at higher speed.

The challenges

Ethernet was originally developed in the 1970s on a shared bus concept (much like many fieldbus technologies), only one in which all parties were free to transmit, albeit one at a time, and all devices would ‘see’ all the data on the bus. If two devices started ‘talking’ at the same time, then the CSMA/CD (carrier sense multiple access/collision detection) access method would mean that the nodes would ‘back off’ and retransmit again after a random delay. This method, while suitable for the business networks of the day, provides absolutely no guarantee of exactly when an ethernet frame will arrive at a destination.

Today, ethernet’s underlying access method is still the same, but networks utilise switches (turning the bus into a star topology), meaning that devices only see data that is addressed to them. This has improved the latency and predictability of native ethernet, so much so that it can be used for deterministic applications up to a point. This is still, however, dependent on the overall network design — overloaded switches, or networks with a larger number of ‘hops’ (multiple switches and routers) between communicating devices can seriously impact the reliability of ethernet as a deterministic medium.

The TCP/IP suite also provides additional overhead that can limit the use of this technology in time-critical applications. The reaction time on TCP/IP networks can often be higher than 100 ms. Figure 1 shows the ISO’s layered OSI Model (open systems interconnection) of data communication, and how ethernet and the TCP/IP protocol suite map to the OSI layers.

Figure 1: The OSI Seven Layer Model and the TCP/IP suite. User data is encapsulated in successive layers down the ‘stack’, each layer adding addressing and error checking information.

All this is not to say, however, that ethernet and TCP/IP cannot be used for real-time applications. The concept of ‘real-time’ is relative — whether the communication is for a motion control system, where response is measured in microseconds or milliseconds, or whether the control system is tracking the sun in a solar power plant, both could be called ‘real-time’ control. Many industrial ethernet protocols also attempt to manage determinism by using a master/slave communication model at Layer 7, in which a master device, such as a controller, determines when communication with devices is to occur, making the timing of communication more predictable.

Relative real time

In relation to ‘real-time’ networking, applications can be divided into three broad categories:

• Non-real time — Normal data processing environments where timeliness of data delivery can vary widely and can be measured in seconds or minutes without detrimental impact, such as data transfers between higher level systems like MES and ERP — in other words, higher level plant management and business applications.
• Soft real time (SRT) — Providing real-time deterministic messaging in the range up to 100 ms response time, using higher level control of (mainly) UDP packets, where a strict master/slave communication is maintained.
• Isochronous real time (IRT) — Provides deterministic messaging for high-speed equipment such as motion control systems, where sub-millisecond timing is depended on.

For non-real time applications, standard ethernet and TCP/IP technologies should suffice. For the other two categories there are different approaches used in different applications. In any case, we must remember that ethernet and the higher-level communication protocol suite does not provide the functionality in any application to actually do anything but provide communication. The actual usefulness of the network depends on higher-level application design, either in the Application Layer (Layer 7) of the ISO model, or above. For example, TCP/IP may provide the HTTP protocol, but is no use without web server and browser applications to make use of it. IT network technology does not provide the necessary high-level functional models to enable industrial applications, so a large part of the development of industrial ethernet has been in the development of higher-level protocols and models to meet the requirements of industrial applications.

Varying approaches

Since industrial field networks are deployed in different environments for different purposes, there have been some varied approaches to utilising ethernet as a field network technology. These can be divided into two broad categories we can call encapsulation and modified ethernet. Some technologies use a combination of both approaches to achieve different levels of determinism for different applications.

Figure 2: Three approaches to implementing industrial protocols over ethernet.

Encapsulation

Encapsulation is where a high-level fieldbus data packet is encapsulated, or embedded, in a standard TCP or UDP packet. In this case, the ethernet and TCP/IP layers of the protocol stack (OSI Layers 1 through 4) are left to function in the standard way and functionality is added at Layer 7. The fieldbus data is carried by TCP/IP as ‘user data’, and standard ethernet hardware can be used.

This is the most straightforward way to use ethernet as a fieldbus transport, adding it as a new transport mechanism that provides enhanced integration with business systems and enhances the deployment and management of larger enterprise industrial control systems.

In order to establish an acceptable level of determinism, control is normally applied to the ethernet communication process from higher-layer functions that determine the flow of information at the application level. This can be achieved by the development of new Layer 7 protocols, or by adding higher-layer application functionality outside the OSI model, such as in FOUNDATION Fieldbus HSE. In most cases, this is achieved by enforcing a strict master/slave communication model, where controllers initiate all communications in a controlled manner (with the exception of alarms).

Direct Layer 2 encapsulation

Some protocols use standard ethernet infrastructure at Layer 1 and standard ethernet media control at Layer 2, but encapsulate the data directly, not making use of the TCP/IP protocol suite. This lowers overhead and improves determinism further.

In practice, this option is usually found alongside the use of TCP/IP anyway. SRT applications can use the direct encapsulation, but the TCP/IP stack allows additional functionality for network management using standardised tools. Luckily, ethernet provides a prioritisation scheme whereby SRT traffic can be encapsulated with a higher priority than TCP/IP.

Modified ethernet

The only way to ensure IRT functionality with sub-millisecond response time and jitter is to change the way ethernet itself works. Obviously an entirely new technology could be developed instead, but there are already real-time device control network technologies available, and this would not take advantage of the integration benefits that ethernet can provide.

In these designs, the physical connectivity of ethernet is maintained (the same cables and basic infrastructure), but the functionality of the ethernet protocol is directly modified at Layer 2 to provide deterministic communication capabilities. This requires hardware interfaces with special ASICs or FPGAs to be developed.

Compatibility with higher-level protocols such as HTTP and SNMP can be achieved by encapsulating this higher-layer data in defined timeslots.

Common features

Despite high-level differences (at Layer 7), all systems have some or all of the common central elements of an IT network infrastructure. This common functionality includes the well-established standards for Ethernet IEEE 802.x data transmission technology (Layer 1), the medium access method (CSMA/CD, Layer 2), the internet protocol (IP, Layer 3) and the TCP and UDP protocols (Layer 4). In addition, common elements can be found in Layer 7 for non time-critical functions. Here, accepted IT standards such as HTTP, FTP and SNMP are commonly used.

Differences

The differences between the current systems can be found in the general communication system architecture, the industrial application protocols in Layer 7, and the object modelling and the engineering model for system configuration. There are also differences in whether the full TCP/IP suite is implemented and whether the Layer 2 architecture has been modified for IRT applications.

In part 2

There are something like 20 protocols that can be called industrial ethernet protocols, so an exhaustive description is not possible. In part 2 of this article we will describe how the five most well-known field and device protocols have been deployed over ethernet.