Network infrastructure: your future business foundation — Part 1

Network infrastructure is one of the most vital yet undervalued business assets. Lose the network and you lose phones, email, internet, access to business systems or control/visibility of the manufacturing process. Many businesses strive to provide optimum versions of the devices connected to the network, such as computers, phones, machines, etc, yet attempt to economise on the network infrastructure that supports these devices.

Current-day industrial networks

Today’s industrial networks are a composite of Ethernet protocols and what industry experts term legacy protocols. ‘Legacy protocol’ is a telling term because, like other legacies, we must live with them for a while. Legacy protocols age and become more difficult to support over time. This issue is further exacerbated by the ageing workforce megatrend. A large portion of the support staff for legacy protocols has reached retirement age. Forward-thinking organisations instituted plans to retain this outbound knowledge. Other companies meet the need by engaging professional services organisations backed by major automation manufacturers.

Protocol distribution in industrial networks

Legacy industrial protocols (fieldbus) account for almost half (48%) of the industrial network nodes sold. Ethernet variants account for 46% of nodes while wireless nodes have 6% share. The telling aspect of this story is that Ethernet and wireless are growing at double digit rates while Fieldbus is growing at a shrinking single-digit rate.

Network refresh rates

Industrial networks have added expectation versus their enterprise counterparts. Not only does the business require them to operate at peak levels, industrial networks have the longest refresh rate of any business network. Where data centres are refreshed every three to five years, industrial networks are refreshed every 12 to15 years. Further, the supporting physical infrastructure is often in place for more than 20 years. A major capital expenditure is required to install and commission a new network. ROCE expectations are extremely high for all businesses (Figure 1).

Industrial network refresh rates are accelerating over time as companies work to balance investment performance and network performance. However, we must still anticipate longer than desirable refresh rates to make planning effective.

Figure 1: Infrastructure investment versus longevity.

Network planning and management

Like other business assets, a rigorous process governing the network ensures efficacy and availability as time goes by. There is not a particular methodology that is superior to others. Realistically, the best run business networks result from a collection of elements intertwined with the governing process. Justification for creating the process is simple:

• Lowers the total cost of network ownership.
• Improves business agility.
• Helps the business respond quickly and effectively.
• Increases availability.

Figure 2: The PPDIOO process.

The PPDIOO Process is a design and management methodology that spans the entire network lifecycle (Figure 2).

1. Prepare: Business agility is a result of good preparation. This phase is used to consider the broad vision, requirements and technologies you can employ to make your business more competitive.
2. Plan: Successful technology deployment must have an accurate assessment of the current state network, its security posture and the business readiness to support the chosen solution.
3. Design: A detailed design reduces risk, avoid delays and controls the total cost of network deployments.
4. Implement: Here the business works to integrate devices and new capabilities in accordance with the design phase without compromising network availability or performance.
5. Operate: The business proactively monitors the network to improve service quality, reduce disruptions and mitigate outages while maintaining high availability, reliability and security.
6. Optimise: Best-in-class businesses never stop looking for a competitive edge. So continuous improvement is a mainstay of any network lifecycle.

Network documentation

Does comprehensive, up-to-date documentation exist for your network? Most companies do not have the required documentation, but all of them should consider it an absolute necessity. Accurate documentation and identification substantially shortens the time to recover from a network issue. Many methods to generate this documentation exist, ranging from a summer intern project to engaging a professional services organisation to assess and document the network.

Network assessment

Professional services organisations, backed by major automation manufacturers, perform cabling and network infrastructure assessments. Many use software that ‘crawls’ unobtrusively through the network, discovering and visualising the network footprint (Table 1).

Table 1: Network design assessment criteria. For a larger image click here.

Legacy protocols

During the assessment process, pay attention to legacy protocols, ie, fieldbus. If legacy protocols are present, plans to migrate them to a modern technology must be at the forefront. Legacy protocols migrate out of the network as it ages because they become difficult and expensive to support, even if their performance is adequate. Their replacement is infrequently a ‘rip-and-replace’ proposition.

Network longevity

Consider the age of the existing network and physical infrastructure. Are any of the active components, network switches, servers, programmable controllers, drivers or other end devices approaching ‘end-of-sale’ or ‘end-of-support’ from the manufacturer? Aged active components have support costs that grow exponentially after a certain age, so they need to retire before your business is frantically searching for good used replacements to get manufacturing up and running.

Evaluate the age of transmission media and its condition. Consider wire speed as well. Category 5e is adequate for 10 Mbps and 100 Mbps traffic but is insufficient in the long term. Pay attention to the connections and the cable. Jacket materials are commonly thermoplastic, which ages over time, particularly in challenging environmental conditions like UV exposure, temperature extremes and chemical exposure. The same exposures age the metallic portion of connectors. Along cable routes, look for sharp bends and areas where cables appear to have been struck or deformed. With multi-pair copper Ethernet media, these physical deformations displace pairs in the cable, damaging its performance. With fibre-optic media, there can be microfractures from physical deformation that attenuate or, in severe cases, interrupt signal flow.

Cabled infrastructure

As the network grows the cabled infrastructure must evolve. There are two cabling topologies used for industrial networks: point-to-point and structured cabling. Older industrial networks employ a point-to-point cabling topology where each connected asset has a home run cable to the control room or data centre. Engineers chose this solution in the belief that connection points were vulnerable and would cause reliability issues. More connections meant more risk. In the very early days of Ethernet connectors there may have been some credence to that conclusion. Connector and media design as well as manufacturing processes are significantly more robust today. So, in modern networks, this argument is no longer valid. Also, the flexibility and network resiliency gained from a structured cabling topology far outweighs the point-to-point reliability argument.

Enterprise networks were once point-to-point cabling. They quickly evolved to structured cabling for several reasons, notably:

• Structured cabling provides the needed flexibility to accommodate moves, adds and changes (MACs).
• Structured cabling can adapt network topology and configuration to business needs without pulling new cabling and resultant disruption of business activities.
• Structured cabling topologies enhance network reliability and recovery speed from outages.
   

Industrial networks are on this evolutionary path because the value proposition for structured cabling networks is so strong. Elimination of downtime is the strongest argument for structured cabling topologies in industrial networks. Industrial network downtime is easily monetised in lost production dollars. As such, a business can readily justify adoption of this topology.

Structured cabling enables patching or otherwise redirecting network traffic to rapidly address infrastructure-related outages. It allows outages due to fault in the horizontal cabling to be immediately addressed by patching to a different horizontal link. After the outage is resolved, the patching infrastructure permits technicians to quickly attach diagnostic instruments to the failed link. The link can be returned to normal service with minimal disruption in network operation. Table 2 shows the functional comparison between enterprise and industrial networks.

Table 2: Functional comparison – enterprise and industrial networks. For a larger image click here.

Installing structured cabling

When installing structured cabling, it’s good to have a few guidelines in mind to ensure maximum viability from the new installation.

• Require the cable installer to connect a network analyser to each link installed, including spares.
• Measured link performance becomes one of the job completion deliverables; doing so establishes that the link delivers expected transmission performance, not just electrical continuity. Further, if there are problems with a link in the future, baseline performance data exists in your files.
• Premium cable manufacturers extend a generous warranty in exchange for fidelity to their offering using a certified installer; it is a worthwhile investigation when selecting materials and installers.

Network management software

Another important topic for the current-day network discussion is a three-letter acronym: NMS. NMS stands for network management software. It is an emergent category of software for industrial networks. Just as the name implies, it is purpose-built software used to manage networks. There are Enterprise NMS solutions and have been for a while. Due to the unique properties of industrial networks, these tools are not suited for the job. When selecting this category of software, make certain the NMS solution you consider is purpose-built for industrial networks.

Industrial networks have two main NMS use cases.

• A consultant or system integrator working with the business uses NMS software located on their computer to discover and visualise the network; this application is to assess and document the current state network in anticipation of service activities that the consultant undertakes on behalf of the business.
• An NMS solution is installed permanently in the network, typically on a server in the DMZ so the entire industrial network is visible and monitored; this application acknowledges the dynamic nature of the network and acts as a watchdog sniffing out problems; visualises the network so a common understanding of status is provided for varying worker experience levels and dependent on the NMS solution; and provides a portal for secure remote access when needed.
   

The first use case focuses on the needs of network maintenance. An expert uses the NMS tool to discover and visualise the network. This step generates a baseline documentation package for the network. Typically, businesses retain this expert to perform maintenance, usually to upgrade or expand to the network. Up-to-date documentation for the network is a welcome latent outcome of the exercise. While the expert’s NMS package is connected to the network, performance and health metrics can be seen, helping the expert spot deficiencies that must be corrected. However, these values are a snapshot, in that the NMS solution does not remain connected to the network long term.

The second use case addresses a greater portion of the network lifecycle. In this use case, the NMS solution resides in the network, typically on a server in the DMZ or the Manufacturing Zone. Residence in the network allows the NMS software to act as a dashboard, allowing network users to see network health and performance. Further, more members of the workforce interact with the network nowadays, all with varying levels of network knowledge. These workers need information out of the network to ascertain if there is a network-related problem slowing down production. A production planner can use that information to make better decisions but may not have the needed skills to access the information.

Network building blocks

Another practice that has risen to prominence is the use of pre-populated and preconfigured network enclosures. The solution allows companies to rapidly deploy or expand without bearing the time penalty and expense of a bespoke enclosure.

Enclosure designs are validated electrically and thermally to eliminate risk during installation and commissioning. Active component placement within the enclosure is optimised for function, thermal performance and maintainability.

Since the network building blocks are built to a validated design, companies gain enhanced supportability through their use, avoiding the ‘snowflake’ scenario where each bespoke enclosure is ‘just a bit different’. This factor is important in a local deployment but becomes vital when multiple locations across a global footprint are considered.

Worker education

Businesses need to invest in their personnel as certainly as they make other business investments. Creating productive collaboration between IT and OT staffs is of immense benefit to businesses. The progression towards more ubiquitous use of Ethernet sets the stage for this collaboration to occur. To hasten engagement, worker training investments are necessary.

Industry experts have long provided training and certification for IT staff in Ethernet-based enterprise applications. Training materials to support industrial staff in a similar realm have been sparse at best.

In Part 2

In Part 2 of this article we will look at emerging technologies and industrial network planning for both the near term and the long term.

References

1. Cisco Systems and Rockwell Automation 2017, Converged Plantwide Ethernet Design and Implementation Guide.
2. Colville, RJ, Spafford, G 2010, 'Top Seven Considerations for Configuration Management for Virtual and Cloud Infrastructures', Gartner RAS Core Research Note G00208328, <<https://img2.insight.com/graphics/no/info2/insight_art6.pdf>>

Top image: ©stock.adobe.com/au/Grispb