Design Components of an Industrial Network

Every industrial process has its own constraints and needs specific consideration while designing the network. For instance, electricity transmission and distribution substations are typically designed around IEC61850 standard, expected to withstand high electromagnetic interference, support micro-second level time synchronization, and are designed for high level of redundancy with close to zero millisecond network recovery time. On the other hand, a discrete manufacturing plant may not need that level of redundancy or time sync, and we would be expecting a robust network to drive end-to-end process for availability of raw material, assembly, and packaging. The differences are therefore unique and would have to be factored in for industry-specific installations.

The key components of a typical industrial network are following:

3.1 Physical Connectivity

Planning appropriate physical connectivity is one of the basic aspects of designing any network. It is important to take some time for due diligence in identifying the physical medium of the connectivity for an industrial network to prevent future bottlenecks, especially when 60-70% of all network down time is due to cable/connectors issues.

There are three mediums of communication: electricity, light, and air.

Electricity: Copper cables are used for the majority of electronic communication between two devices. There are separate standards for industrial cabling for protocols like Foundation field bus, Modbus, Profibus, etc.

However, there are network versions of these protocols, which run on ethernet cable, defined as Cat3/4/5/6/7/8 cable with 4 pairs of cable twisted within a jacket, also known as “Twisted Pair Cable''. In an industrial environment, it is highly recommended to use Shielded Twisted pair cable along with shielded RJ45 connectors, which include an additional layer of metal shielding to counter electromagnetic interference.

Figure 1

Figure 2

One of the key limitations with electrical communication is range, which is defined as 100 meters between two devices.

Light: Light is another medium of communication commonly used with fiber optics cables in an industrial setup. A key purpose of using a fiber optics cable is to enhance point to point communication range beyond the conventional 100 meter range.

An optical fiber is made up of a light carrying core surrounded by cladding. The cladding prevents light from escaping the core, effectively keeping the signal moving down the glass. There are two main categories here, viz.:

A. Single mode fiber: a fiber featuring a small light-carrying core of about 9 micrometers (µm) in diameter. For reference, a human hair is closer to 100 µm. The core is surrounded by a cladding that brings the overall diameter of the optical fiber to 125 µm.

B. Multimode fiber: a fiber with a core of 50 µm or above. A larger core means multiple modes (or rays of light) can travel down the core simultaneously. Just like single mode, the core is surrounded by a cladding that brings the overall diameter of the optical fiber to 125 µm.

Figure 3

Figure 4

Air: In the last couple of decades wireless communication was used in industrial setup for non-critical monitoring application and overall its adoption has increased in the industrial automation process. And going forward, with the emergence of 5G technology, we can expect a further endorsement of wireless communication. ISA100, WirelessHART, LoRaWAN, Sigfox, Zigbee, WIMAX, 6LoWPAN are some of the mature wireless protocols being used in the industrial applications for small to medium range wireless connections.

3.2 Industrial Network Infrastructure

An industrial network is built with the combination of multiple active components to create a robust backbone and enable data communication between required endpoints. The OSI model defines seven layers of communication and Infrastructure devices are also classify based on the operating layers of the OSI model. As we move forward to discuss various industrial network devices, there few common parameters application to all industrial network device, as mentioned below:

Mounting	Din Rail or 19” Rack Mount
Operating Temperature	0 to 60 C or -40 to 85 C
Ingress Protection	IP20/30 or IP65/66/67
Power Supply	24/48 VDC (Redundant) AC 100-240V or DC 100-250V

Industrial networks include the following type of infrastructure components:

Hub and Unmanaged Switches: Hub is a category of devices which are becoming obsolete due to its inefficient method of broadcasting the data to all connected devices. Whereas unmanaged switches can learn and create a forwarding table to efficiently forward data to a given destination address and are still considered a cost effective connectivity solution at the edge level. However, both hub and unmanaged switches do not include any management and security features.

Protocol and Media Converter: Industrial automation requires a complex network that enables machines/controllers operating on different protocols to communicate with each other over a large, distributed network. A common device used in this type of setup is a protocol converter like RS232/485 to Ethernet, and media converter for translating signals from electrical to fiber optics communication and vice versa. These converters are typically located at the edge of an industrial network and support limited network management and security features.

Managed Layer 2 and Layer 3 Switches: A typical industrial network is primarily built with industrial grade Layer 2 managed switches, which are further connected with Layer 3 switches to route the data between multiple subnets. In reference to Purdue Model, managed switches are spread across all levels except Level 0, which is categorized for the physical process of the systems. The evolution of automation within industrial setup, demand to employ secure and effectively managed infrastructure. And managed switches provide the needed structure to build necessary use cases for various industries. In addition to conventional enterprise switches, managed industrial switches support additional redundancy and communication protocols like Modubus/TCP, Profinet, PRP, HSR, MRP etc.

Industrial Firewall: Firewalls are considered to be a gatekeeper for a network, where the operator can define the data flow. In advanced industrial networks, firewalls are used at multiple levels to secure each segment of a network. Industrial firewalls support most of the industrial protocols used by PLC, HMIs and SCADA system and allow users to set rules based on specific protocols to ensure only relevant data flow within a cell.

Industrial Routers/Gateway: Gateways and routers support multiple communication protocols and physical interfaces to enable connectivity with both advanced as well as legacy systems. Industry 4.0 use cases demand multipurpose gateways for not just to communicate between interfaces like RS232/485, Ethernet, E1/T1, Cellular, etc., but also provide a network level compute to a run edge compute logic. This is typically defined as Edge / Fog computing.

Industrial Wireless: Industrial wireless AP’s and devices typically adhere to specific environmental norms and used at the edge of the industrial network to connect wireless sensors. Industrial application does not demand a high bandwidth, rather they expect better latency and range. A typical wireless gateway is expected to support protocols like ISA100 or WirelessHART or LoRaWAN or WiFi with environmental certifications like ATEX Zone 1/2 and IP67.

3.3 Industrial Network Topology and Redundancy

Often, industrial networks are developed in phases as the industrial setup expands and required to connect devices spread across a large, distributed area, for instance, a power plant or manufacturing facility. One of the most common topologies used is Ring, which allows architects to connect distributed devices across the facility with minimum cable length and with at least one cable level redundancy. Actual network topology depends upon multiple factors, such as criticality of the system; latency requirement; network recovery time. Common network architecture and key features are mentioned below:

Figure 5

Figure 6

Industrial networks enable mission critical communications, where volume of data flow could be less, but latency and availability is of paramount importance. Considering availability of the network is one of the key factors for many industrial applications and even with multiple redundancy, the system should be designed with minimum failover time, which is also referred as network recovery time recovery time. Conventional Layer 2 network uses Rapid Spanning Tree protocol (RSTP) for redundancy. RSTP was originally designed to avoid a network loop, which intern stops a broadcast message loop within a network. and the recovery time could be up to 30 seconds in case of network failure. Over the period, various redundancy protocols like Media Redundancy Protocol (IEC62439-2) and PRP/HSR (IEC62439-3) developed for reducing the network recovery to sub 100 milliseconds and 0 milliseconds, respectively.

Figure 7

3.4 Network Management

The process and tools involved in network management determine how efficiently an industrial network is operated and maintained. One of the key tools used for managing a network is Network Management Software (NMS), which mainly employ network protocols like Simple Network Management Protocols (SNMP), Link Layer Discovery Protocol (LLDP), Management Information Base (MIB) etc. These are standardized protocols defined by IEEE to exchange management related information within network infrastructure. As NMS tool can provide following key functionalities:

Network Topology Discovery
Network Physical Connectivity Maps/Graphs
Network Infrastructure Configuration Management
Network Bandwidth Consumption
Network endpoint connectivity
Network events and alarms monitoring

Convergence of IT and OT also enhances the management capability for OT networks. Concepts like Software Defined Networks (SDN), will also be adopted in future industrial setup and enable centralized IT team to drive the policy based network management for OT networks.

3.5 Industrial Network Security

Industrial network security in many ways is considered to be an obstruction and overhead for industrial processes and operations. And conventionally, since industrial networks were isolated, having an air gapped network was considered to be a safe network, which has proven to be a false notion. With the evolution of communication technology, it is almost impossible to keep any network isolated.

Network cybersecurity is more about process and best practices adoption, along with appropriate tools and training.

The below architecture represents how an industrial network should be segment at each level of the Purdue model.

Figure 8

Key aspects for a secure industrial network include:

The control level network should have segmentation to protect each cell.
Each segment should be protected with an Industrial firewall.
Integration between Industrial Network and Corporate IT network should be via Industrial demilitarized zone (DMZ).
All network switches/firewalls should have management capability; Limited unmanaged devices may be used with strict segmentations.
Centralized network management and OT visibility tools should be deployed.
Tight integration between various network tools including NMS, Firmware Management, Inventory management and SIEM, to drive security policies, without affecting operation.
Periodic network assessment based standard frameworks like IEC62443, NIST.