Experimental assessment of network design approaches for protecting industrial control systems

international journal of critical infrastructure protection ] (] ] ]]) ] ] ]–] ]]

Available online at www.sciencedirect.com

www.elsevier.com/locate/ijcip

Experimental assessment of network design approaches for protecting industrial control systems Be´la Gengen, Flavius Graur, Piroska Haller Department of Informatics, Petru Maior University of Tîrgu Mureş, Nicolae Iorga, No. 1, Tîrgu Mureş, Mureş, 540088 Romania

art i cle i nfo

ab st rac t

Article history:

This paper surveys and provides experimental results related to network design techni-

Received 3 July 2014

ques focused on enhancing the security of industrial control systems. It analyzes defense-

Received in revised form

in-depth strategies, network segmentation, network firewall configurations and the role of

11 June 2015

intrusion prevention systems, intrusion detection systems and anomaly detection sys-

Accepted 28 July 2015

tems. The paper also studies the applicability of emerging technologies in the area of IP networks, including software-defined networking, network functions virtualization and

Keywords:

next generation firewalls in securing industrial control systems. The main contribution of

Industrial control systems

this paper is the experimental assessment of existing and future network design

Network security

approaches in the presence of real malware (e.g., Stuxnet) and synthetic attacks (e.g.,

Defense-in-depth

denial-of-service attacks). The experimental results confirm the importance of defense-in-

Software defined networking

depth strategies and also highlight the embryonic state of software-defined networking

Stuxnet

security, which requires profound transformation and validation in order to be embraced by the industrial control system community. & 2015 Elsevier B.V. All rights reserved.

1.

Introduction

The pervasive adoption of commodity, off-the-shelf information and communications hardware and software in modern industrial control systems has led to significant cost reduction as well as greater efficiency, flexibility and interoperability between components. At the same time, it has enabled the implementation of new services and features such as remote monitoring and maintenance, energy markets and the emerging smart grid. However, the technological shift from completely isolated environments to “system of systems” integration has had a dramatic impact on industrial control system security [3]. By leveraging attack vectors that are commonly used to compromise traditional computer systems (e.g., phishing and USB infections), malware aimed

at disrupting critical infrastructure systems have become effective cyber weapons [13,50]. Industrial control systems are also subject to a new breed of cyber-physical attacks. These attacks, which are more complex and sophisticated than traditional cyber attacks, can exploit the cyber and physical dimensions of industrial control systems to significantly impact their normal functioning. Stuxnet [13] is believed to be the first malware that was specifically designed to attack industrial control systems. Its ability to rewrite the control logic of industrial hardware and, more importantly, to hide its presence from system engineers, showcased a new class of threats in which disturbances originating in the cyber dimension propagate to the physical dimension. Stuxnet's follow-ups, Duqu [58], Flame [50] and Dragonfly [59] (reported in June 2014), have revealed

n

Corresponding author. E-mail address: [email protected] (B. Genge).

http://dx.doi.org/10.1016/j.ijcip.2015.07.005 1874-5482/& 2015 Elsevier B.V. All rights reserved.

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

2

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

the true dimension of cyber espionage where specially forged malware can strategically compromise significant organizations and, if needed, cause damage in various industrial sectors, including defense, aviation and energy. Given the escalating threats, this paper provides an experimentation-based survey of existing network design techniques aimed at enhancing industrial control systems security. The main goal is to experimentally evaluate the impact of network design choices on the successful outcome of cyber attacks on industrial control systems. The paper starts with an overview of the main approaches, including wellestablished techniques such as network segmentation, firewall-based traffic filtering and the deployment of intrusion prevention systems (IPSs), intrusion detection systems (IDSs) and anomaly detection systems (ADSs). Next, the importance of defense-in-depth strategies is emphasized and the applicability of emerging IP network technologies such as softwaredefined networking (SDN), network function virtualization and next generation firewalls to securing industrial control systems are discussed. This discussion is followed by experimental evaluations of the principal security measures. The evaluation is conducted using real malware (Stuxnet) and a synthetic denial-of-service attack that causes severe disruptions to communications and services. The results confirm the importance of defense-in-depth strategies and highlight the applicability of novel IP network technologies to enhance the security of modern industrial control systems. This paper also presents an approach for experimenting with the cyber and physical dimensions of large industrial control systems. The approach embodies software-defined network controllers based on Floodlight [46], real sensor networks, an implementation of the emerging Sensei/IoT* proposal [45] and the Mininet network emulator [26]. This is an important step in industrial control system security experimentation because it paves the way towards testing softwaredefined-network-enabled industrial control system configurations. In fact, software-defined networking leads to flexible and dynamic industrial control networks in which accidental failures and malicious attacks can be mitigated through dynamic network reconfiguration. However, throughout this work, it is noted that software-defined networking is an emerging technology, built on an architecture that does not embody security. Thus, while software-defined networking may bring certain advantages, the absence of security features requires profound research and extensive validation before it can be embraced by the traditional IP networking community, let alone the industrial control systems community. This paper has three major contributions. First, it conducts an analysis of network design methodologies focused on securing industrial control systems, and experimentally tests the applicability of emerging networking paradigms to industrial control systems. Second, despite the many articles about Stuxnet [9,13,17,25,33], this paper is the first to describe systematic experiments that evaluate the effects of various network design strategies on Stuxnet's ability to propagate in an industrial control network. Third, the paper presents a novel approach for experimenting with software-definednetworking-enabled industrial control networks in order to provide laboratory-scale infrastructures for conducting security and resilience studies.

2.

Related work

Securing industrial control systems used in the critical infrastructure is an important area of research [30,34,36,55]. A survey of the scientific literature reveals a large collection of articles dealing with security assessments of industrial control systems. Several guidelines and tools have been developed to assess the security risks in industrial control installations. The U.S. Department of Homeland Security guide, Cyber Security Assessments of Industrial Control Systems [61], provides an overview of the cyber security assessment process. It discusses typical cyber security assessment steps such as establishing an assessment team, creating a test plan, identifying attack vectors, executing the assessment and reporting the results. The Cyber Security Evaluation Tool (CSET) provides a systematic approach for conducting cyber security assessments of industrial control systems [28]. CSET is a question and answer based tool that helps establish whether the configuration of a specific installation adheres to industry standards and best practices. It is currently available as a standalone software tool. The InSAW tool provides an integrated approach for modeling actors, assets, relationships between assets and relationships with external entities [42]. InSAW helps construct dependencies and data-flow graphs that aid in the identification of possible vulnerabilities. The tool has been used extensively in recent assessment approaches proposed by Leszczyna et al. [34,36]. The aforementioned approaches are designed to conduct cyber security assessments of industrial control systems without interfering with the actual installations. In contrast, the work presented in this paper experimentally assesses the impact of different industrial control network configuration decisions on the outcomes of cyber attacks involving real and synthetic malware. Several researchers have conducted experimental assessments using synthetic attacks against industrial control systems. Cardenas et al. [11] have studied various attacker models and their impacts on a simulated physical process. Other researchers [21,49] have investigated the impacts of attacks such as spoofing, replay and denial-of-service on the functioning of physical processes. Leszczyna et al. [36] have leveraged the MalSim agent-based architecture [35] to simulate the behavior of a wide range of malware. Moving to large-scale infrastructures, Bilis et al. [8] have documented the impacts of deliberate attacks on the performance of electric power grids. The attacks, which modified parameters in a simulated electric grid, could be launched by an attacker after bypassing protective security measures to cause significant damage to the underlying physical infrastructure. In contrast to the aforementioned assessments, this paper provides valuable insights on the impacts of industrial control network design decisions on the behavior and successful replication of real malware (i.e., Stuxnet). The paper does not focus on the actual impact of malware on the functioning of physical processes because this aspect has been covered extensively in previous studies [8,21,31]. The novelty and principal contribution of this paper is the experimental assessment of the impact of Stuxnet on a typical industrial

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

control network and the impact of different network design decisions on the successful execution of Stuxnet's replication routines. Several researchers have examined Stuxnet's building blocks and its core features and purpose. Falliere et al. [17] published the first detailed report, W32.Stuxnet Dossier, about Stuxnet. Their report detailed Stuxnet's complex features, infection mechanisms, software exploits used to escalate privileges and execute its self-replicating routines, as well as its communications with command and control servers. Chen and Abu-Nimeh [13] have analyzed Stuxnet's high level of sophistication and discuss the massive effort and resources required to develop such malware. Byres et al. [9] describe the possible propagation of Stuxnet in a “high security” industrial control facility. Based on Stuxnet's features, the authors discuss several scenarios in which Stuxnet could copy itself to other networks. They also demonstrate alternative infection pathways that may be used to reach industrial control hardware from the outside world. Finally, Langner [33] reveals that Stuxnet, in fact, executed two different attacks. The first attack targeted the centrifuge over-pressure protection system while the second attack, which occurred a few years later, attacked the centrifuge speed control system. However, one of the most significant aspects of Langner's work, as also highlighted more recently by Hagerott [25], was the human dimension of Stuxnet's attacks and that the attackers opted not to induce a catastrophic failure of the centrifuges. Instead, the attackers sought to erode Iran's technical confidence in their nuclear enrichment program and to induce confusion in an attempt to significantly delay the creation of nuclear weapons. Stuxnet has already received considerable coverage and its features have been extensively documented. Nevertheless, this work makes significant contributions to the state-of-the-art. The research leverages real network architecture implementations to experimentally confirm Stuxnet's ability to bypass security mechanisms such as network segmentation, firewalls and intrusion protection systems, and its ability to propagate from an enterprise network to process controllers as discussed in previous studies [9,13,25,33]. Also, it experimentally demonstrates the ability of intrusion-prevention-enabled firewalls to block malware with known signatures. Additionally, the work experimentally demonstrates the use of novel IP networking paradigms such as software-defined networks to implement dynamic and reconfigurable industrial control networks. Finally, the work shows that Stuxnet, because of its rich feature set, is a valuable asset in experimentation-based security. Indeed, Stuxnet and other similar malware should be used to validate novel techniques and architectures focused on enhancing the security of modern industrial control systems.

3. Classification and analysis of security measures Given the various dimensions of industrial control systems, the different aspects of typical security life-cycle design (e.g., training human personnel and establishing and maintaining policies), and the features and requirements of industrial control installations in specific industries, providing an exhaustive review of the

3

myriad security measures is beyond the scope of this paper. Instead, this paper focuses on general security-aware network design principles and security services, since these principles and services can find applications in the various industry sectors. More specifically, the following techniques for securing industrial control networks are considered:


Network segmentation techniques that separate and iso





late vital network components. Defense-in-depth strategies. Firewall configurations and traffic filtering policies. Intrusion detection and anomaly detection systems. State-of-the-art IP networking solutions.

3.1.

Applying network segmentation techniques

Network segmentation is commonly used in traditional information technology security to isolate and better protect vital services. The technique involves splitting a network into sub-networks and creating separate network segments. The National Institute of Standards and Technology (NIST) Guide to Industrial Control Systems Security [53] and its revisions [54,55] provide a detailed analysis of different network segmentation techniques. As a first step towards a complete network segmentation solution, the documents discuss the use of hosts with multiple network interface cards serving as firewalls. If possible, however, the documents also state that dedicated hardware firewalls should be used to reduce the possible exposure to additional threats and to ensure high performance. Hardware firewalls can also be used to filter and reduce the amount of overall traffic such that only preconfigured traffic is allowed to pass. Additionally, a firewall provisioned between a control network and a corporate network can significantly reduce the probability of successful attacks. However, this architecture requires the configuration of firewall rules to permit direct access to control network devices from a corporate network and careful planning and distribution of services across networks to limit the number of firewall rules, which could lead to significant security breaches. A slight improvement of this architecture is the coupling of routers with firewalls. This approach is commonly used in Internet-facing systems to reduce the load on firewalls, which can then be configured with more complex filtering rules and in-depth packet analysis. Routers can be configured to handle the bulk of the packets with basic packet filtering rules. Positioning firewalls and demilitarized zones (DMZs) between control and corporate networks can provide a greater level of security. By placing critical services (e.g., a data historian) on a different network, no direct communications are required between the corporate and control networks. The demilitarized zone firewall can block arbitrary packets originating from the corporate network and can allow specific packets to reach the demilitarized zone. However, by compromising a host in the demilitarized zone network, an attacker can still launch an attack against the control network by exploiting packets permitted by the firewall. In such scenarios, the NIST documents recommend the use of hardened firewall rules (e.g., to only allow traffic originating from the control network). Subsequently, an additional firewall

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

4

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

could be added to the architecture (if possible, a firewall from a different vendor), that would lead to the separation of firewall rules to regulate traffic between the corporate network and the demilitarized zone network, and between the demilitarized zone network and the control network. The ANSI/ISA-99 standards also emphasize network segmentation and define security zones as the “grouping of logical or physical assets that share common security requirements” [30]. The zones provide a means to segment specific areas of a control system. First, control subsystems (security zones) are created by grouping assets (physical or logical) that share common security requirements. Next, communications between the security zones are defined by means of security conduits, denoting the pathways through which data is forwarded within a specific installation between various zones. The security zones and conduits defined by the ANSI/ISA-99 standards may be implemented by network segmentation techniques, industrial firewalls and virtual private networks (VPNs).

3.2.

Implementing defense-in-depth strategies

The integration of multiple security mechanisms in different infrastructure layers is a well-established approach to ensure defense-in-depth for critical assets. The NIST Guide to Industrial Control Systems Security [55] mentions the importance of architecting security based on defense-in-depth principles by identifying possible attack vectors and integrating firewalls, demilitarized zones, intrusion/anomaly detection systems and incident reporting techniques. Procedures and network architectures for defense-indepth-enabled network design in industrial control systems are described by the U.S. Department of Homeland Security and the Idaho National Laboratory in [32] and more recently in [60]. The documents outline the main threats targeting industrial control systems and provide detailed analyses of defensein-depth strategies. The first step is to identify architectural zones and assign security priority numbers to each zone by computing the associated safety and risk levels. Then, for each zone, specific security measures are implemented by adopting different types of firewalls (e.g., packet filtering firewalls, proxy gateway firewalls and host firewalls), demilitarized zones and intrusion detection systems. Following this, policies and procedures must be established to implement a successful defense-in-depth strategy; these include log and event management, security policies, patch management, security training, and incident response and forensics. Redundancy also plays a major role in the implementation of defense-in-depth strategies. The deployment of different backup layers – to achieve backup-in-depth [53] properties – can help a system recover from security incidents.

3.3.

Using firewall configurations

The use of network segregation and demilitarized zones may place heavily accessed services (e.g., data historians) in different networks, which could reduce firewall throughput. Therefore, network firewall configuration, service distribution and security zone creation should be carefully planned. NIST's Guide to Industrial Control System Security [55] points out several issues regarding the configuration of

industrial control system firewalls. Since a data historian could be placed in a separate demilitarized zone protected by firewalls, protocols for accessing such services (e.g., Modbus/TCP or DCOM) must be permitted through the firewalls. However, this could diminish firewall throughput, especially in the case of heavy access from the corporate network. A solution suggested in [55] is to place a mirror service on the corporate network, which is time-synchronized with the historian located in the demilitarized zone. Another issue highlighted in [55] is the remote access of network services. The recommendations include using virtual private networks and a strong token-based authentication scheme to connect to the corporate network and a second authentication at the control network firewall. Yet another issue related to multicast traffic involves the deployment of multicast-enabled routers in each network and the configuration of multicast groups. This requires new firewall rules to be added and maintained. The Network Access Policy Tool (NetAPT) [43] provides a means to perform a comprehensive analysis of security policy configurations, including firewall rules. NetAPT takes as input a network topology, the rules implemented in security devices and a set of global access constraints. It uses this information to perform an exhaustive analysis and identify possible violations of global access constraints. NetAPT also supports statistical analysis for large networks and uses sampling methods as well as mathematical models to identify possible policy violations. The tool is currently transitioning towards commercialization as NP-View.

3.4. Implementing intrusion and anomaly detection systems Intrusion detection is a well-established field of research. Therefore, the deployment of complex detection engines together with malware signature databases have received significant attention from the scientific community. However, an emerging trend in industrial control system security is the deployment of anomaly detection systems. Unlike traditional intrusion detection systems, which require the presence of an up-to-date signature database, anomaly detection systems focus on the identification of abnormal behavior. Recent research has highlighted the applicability of anomaly detection techniques to industrial control systems [23,48,62]. In fact, anomaly-based intrusion detection is ideal for scenarios where the encountered behavior is narrow enough to allow meaningful detection of deviations from the “normal.” Unfortunately, the unpredictable behavior of traffic in traditional information and communications networks raises significant challenges with regard to provisioning anomaly detection systems. On the other hand, the regulated and predictable behavior of traffic in industrial control networks [5,18] make anomaly detection systems an attractive option in industrial control environments. For example, Garitano et al. [18] have developed an algorithmic approach that can effectively predict industrial control network traffic based on applicationspecific parameters such as the number of variables, types of variables and traffic periodicity. Goldenberg and Wool [23] have leveraged deterministic finite automata theory to construct a detailed model of the Modbus/TCP protocol; the model captures details such as source and destination IP addresses,

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

Modbus master and slave identifiers and message types. Zhao et al. [62] have employed dynamic-time warping and the adaptive fuzzy c-means algorithm to detect anomalies in industrial control network traffic; the effectiveness of this approach was validated using data from a real steel plant. The deterministic and long-lasting behavior of communications flows between control equipment can be used to train anomaly detection systems that are highly sensitive to cyber attacks that inject just a single packet into a network. This characteristic is explored in the work of Barbosa et al. [4], where periodic traffic bursts are associated with independent traffic flows and are carefully monitored. The approach incorporates a learning phase for monitoring and tuning the parameters of the anomaly detection system. The short-time Fourier transform is used to construct a traffic spectrogram, which is then used for anomaly detection. In more recent work [5], Barbosa et al. have proposed an approach for identifying connection patterns between different hosts and capturing source, destination, protocol type and server port information for each traffic flow. The approach incorporates a phase during which traffic flows between hosts are learned; the resulting knowledge is used in the detection phase. The main concern is the duration of the learning phase because some connections do not occur regularly and the resulting longer learning phase could lead to misconfigurations as a result of ongoing attacks. Genge et al. [20] have proposed the SPEAR methodology and tool suite that relies on similar assumptions as the methodologies mentioned above. SPEAR provides a formal language based on ns-2 and a graphical interface to model an industrial control system topology and its communications flows. The resulting model is processed by SPEAR to generate Snort anomaly detection rules. The presence of the physical dimension in an industrial control environments enables model prediction techniques to be applied in detecting anomalies during the operation of physical processes. Svendsen and Wolthusen [56,57] have employed physical process models along with feedback control theory to predict future states and ultimately to detect physical anomalies. A similar approach has been proposed by Cardenas et al. [11], who employed a chemical plant model with feedback control loops to predict the state of the physical process and to detect attacks against an industrial control system. Although these and other similar approaches may accurately detect anomalies, they require complete and highly detailed models of physical processes, which may not always be available. Nai Fovino et al. [10,41] address this issue by assuming that every control system attack will ultimately lead to a transition of the system from a secure state to a critical state. In their approach, critical state descriptions created by process engineers are input to a detection engine that estimates process state. The final set of approaches focuses on the cyber and physical dimensions of industrial control systems. Raciti and Nadjm-Tehrani [47] have proposed an alert aggregation technique that collects data from cyber and physical sensors. Levorato and Mitra [37] have created a unified view of several smart grid dimensions (e.g., energy markets and weather conditions) using sparse approximation and wavelets. Their approach is well-suited to the high-level analysis of the smart

5

grid, but it can lose its effectiveness when dealing with lowlevel aspects (e.g., detecting SYN attacks). Genge et al. [22] have proposed data fusion techniques to detect anomalies in data originating from sensors in the cyber and physical dimensions of an industrial control system. In their approach, each sensor monitors, detects and reports its own perspective (belief) of the observed cyber and/or physical attributes. The beliefs of several sensors are then fused to provide a unified view of the system state. While the approaches discussed above are useful for detecting intrusions, their deployments in real industrial control systems, especially large-scale critical installations, present several challenges. Bellovin et al. [6,7] emphasize that intrusion detection and prevention systems used in critical infrastructure assets must take into account the complexity of information collection, the complexity of telecommunications networks and the complexity of specific physical installations such as the power grid. In large-scale infrastructures, the collection of information from millions of sensors and the secure storage and processing of data across different domains (e.g., multiple companies) can raise serious technical and privacy issues. At the same time, the hardware limitations of intrusion detection systems limit their applicability in large infrastructures where data from millions of sensors must be analyzed and decisions made in near real-time. The use of cryptography to secure communications can also limit the effectiveness of intrusion detection systems due to the overhead involved in encryption and decryption and in ensuring that data protection laws are not violated.

3.5.

Leveraging advanced IP networking solutions

Several companies specializing in traditional IP networking as well as companies in the various industry sectors are working on improving and/or replacing older communications infrastructures based on frame relay, power line carrier and asynchronous transfer mode (ATM) technologies with state-of-the-art networking solutions such as Internet Protocol/Multiprotocol Label Switching (IP/MPLS). One example is the partnership between Cisco and IBM to implement a reliable and secure networking infrastructure for Terna, an Italian electrical transmission and distribution company [27]. The Italian blackouts of 2003 and the loss of communications signaling experienced by Italian operators as a result of damage to a high-power pylon in Switzerland demonstrated the urgent need to improve the Italian communications infrastructure. After considerable analysis, Cisco and IBM selected MPLS technology to implement a manageable and controllable solution for the Italian electric grid. The advantages of MPLS over generic IP networks include efficient routing capabilities, built-in support for virtual private networks and traffic engineering. However, as shown in [21], MPLS-enabled networks are still vulnerable to cyber attacks (e.g., denial-of-service attacks) that may have a profound impact on the quality of service (QoS) of industrial traffic. Therefore, traditional QoS-specific protective measures must be implemented in MPLS-enabled networks in order to isolate critical traffic. An emerging paradigm in traditional IP networks is the replacement of local router based decision solutions with

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

6

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

global routing decision software. A prominent enabler is OpenFlow, a protocol designed to ensure remote access to the forwarding pane of a network switch [38]. This approach separates control from forwarding, enabling more complex traffic management techniques to be implemented. OpenFlow also bridges the gaps between network switch/router providers because the same protocol can be used to program a wide range of OpenFlow-enabled hardware devices. Another important advancement is software-defined networking [44], which provides the means to create virtual networking services and to implement global networking decisions. Software-defined networking relies on OpenFlow to enable communications with remote devices and has recently been categorized as the “next big technology” [1] that will revolutionize the way decisions are implemented in switches and routers. Software-defined networking provides a directly programmable network that is managed centrally, but can be monitored and configured remotely; moreover, it is based on open standards. However, it should be noted that software-defined networking is still an emerging paradigm and its adoption in industrial control systems requires extensive research to understand the opportunities and challenges involved in creating softwaredefined industrial control networks. While several studies have revealed that software-defined networking can enhance the quality of service in industrial control systems, especially largescale electric grids [24,39], software-defined networking and its applications to industrial control systems are still in an embryonic state. In fact, from a security point of view, the centralized software-defined networking architecture is susceptible to compromised controllers, which can reconfigure the network and expose it to denial-of-service attacks with potentially catastrophic consequences [14]. Furthermore, since security is not a part of the initial design of a software-defined network, a wide variety of attack vectors can target the different layers of the network [2]. Indeed, the lack of a security-oriented design is considered to be the main impediment to the growth of software-defined networking. Therefore, systematic analyses and extensive experimental validation are required to fully understand the advantages of software-defined networking and the profound security risks that this emerging technology may introduce to modern industrial control systems. Apart from software-defined networking, virtualization is also gaining momentum, especially the concept of network functions virtualization (NFV) [16]. Network functions virtualization leverages standard virtualization technologies to embody networking equipment in virtual environments. This concept leads to the deployment of virtual routers, virtual switches and high volume servers in one high-power physical machine. In real production systems, and possibly industrial control systems as well, network functions virtualization can significantly reduce costs, power consumption and the time required for products to reach consumers. Also, it can enable highly flexible network configurations. Significant progress has also been reported in firewallbased protection. The next generation firewall (NGFW) [51] enhances the capabilities of a traditional firewall with indepth packet inspection at various levels. For example, a next generation firewall can allow Modbus protocol write operations to close/open valves only for specific hosts. Also, it can

limit monitoring capabilities of process variables to requests that originate from within a preconfigured set of networks. Additionally, its packet inspection capabilities can enable the

Fig. 1 – Local industrial control network topology. (a) Without network segmentation and (b) with network segmentation.

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

dynamic routing. The first architecture used in the experiments was the local industrial control network shown in Fig. 1(a) and (b). It incorporated programmable logic controllers, human– machine interfaces, servers, workstations, firewalls and a real physical process. The physical process comprised a steam-based power generator that was recreated in a laboratory environment; the control tasks were to maintain the level and flow. The process control network deployed Allen-Bradley and Panasonic programmable logic controllers. All the workstations ran Windows XP with NetBios and file sharing activated. An HP Tipping Point S8005 Next Generation Firewall with intrusion prevention capabilities was also incorporated. The second architecture was a large-scale networking infrastructure recreated by the Mininet [26] emulation software (Fig. 2). A Floodlight [46] software-defined network controller was executed on top of the infrastructure to monitor and dynamically change the routing tables of the virtual switches. In the scenario, data originated from a sensor network consisting of ten WaspMote sensor nodes equipped with XBee Pro communications modules. Each WaspMote node transmitted the measured temperature and battery level to a Meshlium sensor gateway. Meshlium acted as data aggregator, collecting the data from WaspMote nodes and storing them in a local MySQL database. From here, the data was periodically read by the sensor data server, which exposed an XMPP interface to enable

detection of known vulnerabilities such as Heartbleed [15] and POODLE [40]. Thus, a next generation firewall has some intrusion prevention capabilities, such as blocking packets that match specific signatures. However, a next generation firewall is not intended to be a full-featured intrusion prevention system. An intrusion prevention system, on the other hand, enhances a traditional intrusion detection system with traffic blocking capabilities. This makes an intrusion prevention system an active and valuable security component that can independently and effectively block malware and traffic based on rules and signatures.

4.

Experimental assessment

The experimental assessment of the security measures discussed above considered various characteristics of real industrial control system topologies, including networks and protocols, hosts, programmable logic controllers and physical processes. Experiments were conducted with real malware (i.e., Stuxnet) along with synthetic attacks in order to identify the level of detail that an attacker needs to ensure a successful outcome. The experiments focused on two architectures: (i) a local industrial control system that included two networks separated by a firewall; and (ii) a larger networked infrastructure with

Legend: Virtual switch Virtual host

Sensor network (WaspMote XBee Pro) ...

Regular traffic Malicious traffic

Attacker script (Python + iperf)

Gateway (Meshlium)

...

XMPP ...

... Sensor data server (Jabber/XMPP)

... Emulated network (Mininet)

Sensor network (WaspMote XBee Pro)

Legend: Virtual switch Virtual host

Sensor network monitoring application (Python)

Regular traffic Malicious traffic

... SDN controller (Floodlight)

Attacker script (Python + iperf) Gateway (Meshlium) ...

XMPP ... Sensor data server (Jabber/XMPP)

7

... Emulated network (Mininet)

...

Sensor network monitoring application (Python)

Fig. 2 – Large-scale industrial control network topology. (a) Without SDN controller and (b) with SDN controller. Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

8

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

Table 1 – Hardware used in the experiments. Hardware

Description

12 workstations with Intel Dual Core 2.0 GHz CPUs and 2 GB RAM

The workstations were equipped with the Windows XP operating system The network switches were used to interconnect experimental components The industrial firewall was used to test policies and capture traffic between hosts The programmable logic controller was used to implement the level control loop The programmable logic controller was used to implement the flow control loop The nodes were used to create a wireless sensor network

3 Cisco 2950 Switches 1 HP Tipping Point S8005 Next Generation Firewall 1 Allen-Bradley 1756 l61 programmable logic controller 1 Panasonic FP0R programmable logic controller 10 WaspMote nodes equipped with temperature sensors and XBee communication modules 1 Meshlium gateway equipped with XBee and WiFi

communications with client applications. The sensor data server provisioned a mirror service for the data aggregator, which is discussed in detail in [19]. Its main task was to implement basic communications and architectural features defined by Sensei/ IoT* [45], an emerging standard for smart grid communications. Sensei/IoT*, also known as the ISO/IEC/IEEE P21451-1-4 XMPP Interface Standard, is the result of a joint effort between the ISO, IEC and IEEE to design a semantic web 3.0 sensor standard for sensor networks, machine to machine (M2M) technologies and the Internet of Things (IoT). The main goal of the Sensei/IoT* standard is to provide interoperability, scalability and security using the XMPP protocol. Details about the emerging standard and the sensor data server can be found in [19,45]. The experimental architecture is shown in Fig. 2 and a summary of hardware used in the experiments is provided in Table 1. The Stuxnet malware was employed to evaluate the effectiveness of various security countermeasures in the presence of real malware. Given that several versions of Stuxnet are available “in the wild,” a version that includes most of Stuxnet's documented behavior was selected. Although the laboratory installation did not replicate the exact conditions for which Stuxnet was constructed, the goal was to measure Stuxnet's behavior from the perspective of network communications and replication. As confirmed by previous reports [17] and by the experiments described in this paper, Stuxnet's replication/infection sequence is activated and launched even in the absence of the target Siemens configuration. Thus, Stuxnet could be used to perform realistic tests of the network-level defense mechanisms discussed in this paper. The second attack involved a denial-of-service attack launched using the Iperf software [52]. While the Stuxnet malware provided the ability to study the effects of different configurations on the outcome of a real cyber attack, the second attack assisted with the evaluation of network topologies and routing algorithms.

5.

Experimental results

This section presents the experimental results. The experiments began with a baseline architecture without network segmentation. The architecture was gradually upgraded to incorporate several defense-in-depth strategies, including

The gateway was used as a data aggregator for the WaspMote sensor network

network segmentation, industrial firewalls, intrusion prevention systems, anomaly detection systems and industrial control networks based on software-defined networking.

5.1.

Baseline architecture

The baseline architecture without network segmentation provides a reference for comparing the results obtained with the various security measures applied in the following subsections. In the first experiment, all the hosts were located in the same network – the network was not segregated and critical components were not placed on a separate, protected network segment (see Fig. 1(a)). Although the outcome of this experiment is highly intuitive, it demonstrates how Stuxnet propagates to other hosts located in a network. Network traffic was observed between the hosts that monitored the physical process and programmable logic controllers, along with the regular Windows NetBios. While NetBios was not particularly relevant to the scenario, it was enabled in all the experiments because of its widespread use in older operating systems (e.g., Windows XP and earlier). Fig. 3(a) shows the network traffic. The larger spikes are due to regular NetBios traffic, which did not exceed 12 KBps. In the next experiment, one of the monitoring Windows stations was deliberately infected with Stuxnet using a USB flash drive. As documented by several researchers [17], immediately after the USB drive was plugged into the system, Stuxnet used a zero-day vulnerability in the Windows Explorer Shell shortcut parsing code sequence (MS10-046) to infect the system. Although the experimental setup did not incorporate Siemens hardware and software, Stuxnet successfully infected the target host. After a few hours, network traffic captures revealed significant activity involving Stuxnet's replication routines. In particular, Stuxnet exploited a zero-day Windows network sharing and RPC vulnerability (MS08-067), which allowed it to create a temporary file on the remote host and to copy itself over the network. Table 2 shows a sequence of Server Message Block (SMB) protocol packets, which demonstrates Stuxnet's replication behavior. Stuxnet's infection and replication processes are clearly visible in the network traffic in Fig. 3(b). The experiment revealed that the time when the infection occurs depends on

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

Stuxnet's internal state and the visibility of Windows hosts. As such, the experiment demonstrated that Stuxnet might infect one station at a time (at two- to eight-hour intervals) or several stations in just a few seconds. Although this work concentrates on network design and network traffic (mostly the cyber dimension of industrial control systems), previous studies have shown that the impact of targeted attacks on industrial control systems can be dramatic [31]. Such attacks can lead to profound changes in the behavior of physical processes as well as in malfunctions of critical equipment. In some cases, these may trigger alarms that, in turn, could lead to complete process shutdown [31]. Additional experiments and results concerning the impact of cyber attacks on the normal functioning of simulated physical processes are documented in [8,11,21]. Taking into account the best practices concerning security planning (e.g., disabling unused services), the file and printer sharing features were disabled under the assumption that they were not required by the installation. Because Stuxnet exploits network sharing to replicate its code to other hosts, the deactivation of this feature halted the replication of

12

Throughput (KBytes/s)

10

8

Stuxnet. Fig. 4(a) shows this behavior – the attack is not executing and the traffic only consists of regular packets. This experiment demonstrates the benefits of best practices, but it shows once again that Stuxnet's self-replication routines are executed even if the infected machine is not its designated target. This is explained by the fact that Stuxnet continuously executes its propagation routines in the search for victims. The infected hosts then contact a command and control server on the Internet to receive instructions and updates. This particular behavior is highlighted by experiments presented in the remaining of this section. The previous experiments illustrated Stuxnet's ability to self-replicate using two zero-day exploits. The first exploit, which enables Stuxnet to be injected via removable drives, leverages the Microsoft Windows shortcut LNK/PIF files automatic file execution vulnerability (MS10-046). The second exploit leverages Windows network sharing and RPC vulnerability (MS08-067) to copy Stuxnet's code over the network to a remote host. While these are only two of several selfreplicating mechanisms used by Stuxnet [9,17,33], they provide opportunities to demonstrate the possible impact of network design decisions on the propagation of complex malware. Furthermore, since previous studies on Stuxnet provide adequate details of its capabilities, this work focuses on systematically testing established network design methodologies in the presence of Stuxnet. Indeed, using Stuxnet as a test case can provide valuable insights into developing and refining industrial control system protection mechanisms.

6

5.2.

4

Following the NIST recommendations in [55], two network segments (shown in Fig. 1(b)) were created by placing the corporate and process networks on two different layer 2 segments. In this configuration, network traffic from clientmonitoring applications running on the workstations was filtered by an industrial HP Tipping Point S8005 Next Generation Firewall. The firewall was configured with a default deny-all policy and only allowed Windows domain traffic (including file and printer sharing traffic) and Ethernet Industrial Protocol (EtherNet/IP) to pass. The experiment revealed that, although the firewall was activated, Stuxnet was able to successfully run its replication routine and spread from the corporate network to the process network, infecting two human–machine interface stations. Fig. 4(b) shows this behavior – two workstations were infected in less than 30 s. This effect is expected because the firewall was configured to allow regular file and printer sharing protocols, which are exploited by Stuxnet to replicate its code in other network segments. NIST's recommendations in [55] were also followed to provision a demilitarized zone network between the corporate and process networks, following which Stuxnet's replication routines were tested. The firewall rules were configured to allow connections from the corporate network to the demilitarized zone network and from the demilitarized zone network to the process network. By relying on the same vulnerability and the fact that the two firewalls were configured to permit legitimate traffic, Stuxnet was able to self-replicate from the

2

0 10

15

20

25

30

35

40

45

50

Time (minutes)

700 600 Throughput (KBytes/s)

9

500 400 300 200 100 0

0

2

4

6

8

10

Time (minutes)

Fig. 3 – Traffic without defense mechanisms. (a) Normal traffic without Stuxnet infection and (b) traffic with Stuxnet infection.

Network segmentation with firewall

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

10

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

Table 2 – Network packet capture showing part of Stuxnet's replication sequence. Time

Source

Destination

Information

4964:783 4964:783 4964:784 4964:786 4964:791 4964:799 4964:8

10:1:150:1 10:1:150:4 10:1:150:1 10:1:150:4 10:1:150:1 10:1:150:4 10:1:150:1

10:1:150:4 10:1:150:1 10:1:150:4 10:1:150:1 10:1:150:4 10:1:150:1 10:1:150:4

Trans2 Request; QUERY_FILE_INFO; FID : 0x4001; Query File Internal Info Trans2 Response; FID : 0  4001; QUERY_FILE_INFO Trans2 Request; SET_FILE_INFO; FID : 0x4001 Trans2 Response; FID : 0  4001; SET_FILE_INFO Close Request; FID : 0  4001 Close Response; FID : 0  4001 NT Create AndX Request; FID : 0  4002; Path : ⧹Documents and Settings⧹DEFRAG24681:TMP

4964:801

10:1:150:4

NT Create AndX Response; FID : 0  4002

4964:802 4964:94 4964:94

10:1:150:1 10:1:150:1 10:1:150:1

10:1:150:1 … … … 10:1:150:4 10:1:150:4 10:1:150:4

½TCP segment of a reassembled PDU ½TCP segment of a reassembled PDU ½TCP segment of a reassembled PDU

corporate network to the demilitarized zone and then to the process network. The results described above provide experimental confirmation of the risks posed by protocol vulnerability exploitation. The experiments were specifically designed to permit the execution of vulnerable file sharing protocols – a prerequisite for ensuring the successful execution of Stuxnet's replication routines. Similarly, it is possible to extrapolate and record similar occurrences for other protocols and attacks as well. For example, targeted attacks that exploit unprotected industrial protocols could send specially crafted packets to programmable logic controllers to trigger physical events with dramatic outcomes [41]. Therefore, the results in this section confirm previous experiences and point to similar outcomes for other protocols and configurations.

5.3. Network segmentation, firewall and intrusion prevention system The next experiment took the previous configuration one step further. In addition to network segmentation and firewall configuration, intrusion prevention was activated within the firewall. The intrusion prevention system (IPS) represents an additional component in the defense-in-depth security strategy because it filters packets that match malware signatures. In the experiment, the intrusion prevention system was configured to trigger an alert and log each packet that exploited the file and printer sharing vulnerability. Furthermore, the packets were dropped by the firewall, allowing only regular traffic to pass through. Fig. 4(c) shows this behavior – Stuxnet was unable to successfully run its replication routine and only regular Windows domain traffic (spikes in the figure) and industrial protocol packets were allowed to pass through the firewall. This protection mechanism would be ineffective for attack signatures that were not included in the intrusion prevention system database. Although an up-to-date database coupled with heuristics for identifying malicious packets might prove to be effective in most scenarios, complex attacks that send common industrial control protocol packets to critical assets (e.g., programmable logic controllers) may not be detected. In these situations, additional security mechanisms, such as the advanced intrusion detection and anomaly detection

mechanisms discussed in the previous sections, would be required. The importance of using such detection mechanisms is demonstrated in the following subsection.

5.4. Network segmentation, firewall and anomaly detection system The next experiment employed an architecture with network segmentation and a firewall, without the intrusion prevention system, but with an anomaly detection system (ADS). SPEAR [20], a systematic approach for modeling industrial control network topologies and automatically generating Snort rules, was also employed. SPEAR relies on the predictive behavior of connections between different industrial control system hosts in order to identify abnormal packet exchanges. Essentially, the detection rules generated by SPEAR whitelist known/allowed traffic flows between specific hosts and generate alarms for packets that violate the modeled communications patterns. Snort was launched with the rules generated by SPEAR in the process control network to analyze incoming and outgoing traffic at the firewall. In this configuration, Stuxnet used the same exploit to replicate itself and was able to pass through the firewall (shown in Fig. 4(d)) and the anomaly detection system undetected. This is not surprising because the normal Windows domain traffic was also whitelisted by SPEAR. After the infection succeeded, Stuxnet attempted to connect to a http://www.windowsupdate.com and http://www.msn.com in order to reach its command and control servers. Since direct Internet connections to these external addresses were not whitelisted, Snort issued an alarm for each packet (Fig. 5). Note that, in this particular experiment, the firewall played an important role in blocking connection attempts made by Stuxnet to the aforementioned Internet addresses. However, it was the anomaly detection system that actually signaled the anomaly in the network traffic. This is a key result when designing defense-in-depth strategies, where, as confirmed by this experiment, in addition to network design choices, special attention must be given to firewall configuration policies, intrusion prevention systems and anomaly detection systems. It is also important to be aware of the limitations of simple anomaly detection systems such as SPEAR that rely on

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

11

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

8

Throughput (KBytes/s)

7 6 5 4 3 2 1 0

0

5

10

15

20

25

30

35

40

Time (minutes)

Throughput (KBytes/s)

1200 1000 800 600

engineers' expertise to whitelist industrial control network traffic. However, given the variety of protocols and network traffic present in typical industrial facilities, such configurations are not trivial and may yield a high number of false positive alerts. In the experiment, Windows updates were deliberately disabled and legitimate software applications installed on hosts were not allowed to communicate with outside (i.e., Internet) hosts. As a result, traffic flows to outside networks were not whitelisted and Stuxnet's attempts to contact Internet hosts immediately triggered Snort alarms. Nevertheless, like Stuxnet, other malware may employ whitelisted traffic to self-propagate to other hosts in a network; these attacks would not be detected by elementary anomaly detection systems. Therefore, industrial control installations need to adopt advanced detection systems that integrate semantic packet inspection that infer the validity of specific operations and also fuse data from the cyber and the physical dimensions to deal with the state of complex physical processes. Interested readers are referred to [12,41] for more information about these issues.

400

5.5.

MPLS, dynamic routing and SDN controllers

200 0

0

1

2

3

4

5

6

30

35

7

Time (minutes)

Throughput (KBytes/s)

30 25 20 15 10 5 0

0

5

10

15

20

25

40

Time (minutes)

In large-scale industrial control facilities, hardware is spread across large geographical areas ranging from a city to an entire country or even multiple countries. In such a scenario, physical processes are monitored remotely over complex network topologies, possibly hosting more than one communications flow (e.g., industrial communications flow and monitoring communications flow). Consequently, denial-ofservice attacks launched from logically separated communications flows can cause severe telecommunications service degradation that can propagate and severely affect the operation of the physical processes. Previous work [21] has shown the impact of disruptive cyber attacks on MPLS-enabled communications and ultimately on the functioning of physical processes. The telecommunications infrastructure in question involved an MPLS network with four MPLS-enabled Cisco routers. MPLS virtual 3

2.5

1000

Number of Snort alerts

Throughput (KBytes/s)

1200

800 600 400 200 0

0

2

4

6

8

10

12

14

16

2

1.5

1

0.5

Time (minutes)

Fig. 4 – Effects of various countermeasures on network traffic. (a) Disabled unused services (file and printer sharing), (b) network segmentation with firewall activated, (c) network segmentation with firewall and IPS activated and (d) network segmentation with firewall and ADS activated.

0

0

2

4

6

8

10

12

14

16

Time (minutes)

Fig. 5 – Alarms raised when Stuxnet tests for Internet connectivity.

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

12

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

circuits were used to isolate industrial control traffic from public traffic. The results showed that the virtual circuits did not prevent denial-of-service attacks and additional measures had to be taken (e.g., enable quality of service) in order to limit the effectiveness of the attacks. Conversely, this study engaged the architecture shown in Fig. 2, which incorporated a sensor network monitored by a mesh-like switch/router topology. As in real-world infrastructures, a data aggregator implemented with an XMPP/Jabber server was employed. Network throughput was monitored by a controller based on software-defined networking, which could install static flows in the underlying switch/router topology to optimize data transfer and alleviate denial-ofservice attacks. Compared with the previous work [21], this architecture demonstrates the potential applicability of controllers based on software-defined networking to implement dynamic closed-loop network traffic control strategies in large-scale industrial control facilities. Without the intervention of the software-defined networking controller, the malicious hosts flooded the network with

140 Normal traffic DoS traffic

Throughput (KBytes/s)

120 100 80 60 40 20 0

0

1

2

3

4

5

6

Time (minutes)

120

UDP packets, an operation that had a dramatic impact on regular network traffic. As shown in Fig. 6(a), the first phase of the attack was executed for ten seconds, followed by a second phase for two minutes. In the first phase, the impact is barely visible, however, the two-minute second phase causes significant fluctuations in normal TCP traffic, which may have severe repercussions on time-sensitive control loops. For example, according to the IEEE 1646-2004 standard for communications delays in substation automation [29], high-speed messages must be delivered within 2 ms to 10 ms. Therefore, the loss of network packets triggered by the attack significantly alters the shape of the time-critical control system traffic, especially in the case of high-speed message delivery requirements. To further illustrate the disruptive impact of the implemented cyber attack, the round trip time (RTT) of TCP-ACK packets was measured. The results in Fig. 7 show that the attack causes the RTT values to increase up to six seconds, which results in a severe degradation of the quality of service of industrial control system communications. The software-defined network controller mitigated the denial-of-service attack by re-routing the attack traffic to protect the highly sensitive sensor monitoring traffic. The traffic throughput was sampled every second; if a specific throughput level was exceeded, then the traffic re-routing routine was executed. Fig. 6(b) shows the effects of this intervention. The denial-of-service traffic burst is effectively eliminated after it is detected, while the regular monitoring traffic is protected from the disruptive traffic. The detection engine coupled with the software-defined network controller is a simple proof-of-concept throughput measurement module; however, it is important to note that more complex detection engines can be realized. For instance, intrusion detection or anomaly detection engines that identify specific malicious flows could be configured to send alerts to the software-defined network controller to block or re-route malicious flows. On the other hand, malicious traffic may be re-routed for further inspection by security hardware. Such approaches can close an important loop in cyber security mitigation strategies, especially in the case of large-scale

Normal traffic DoS traffic

1

10

80

TCP−ACK round trip time (s)

Throughput (KBytes/s)

100

60

40

20

0

0

0.5

1

1.5

0

10

−1

10

−2

10

2

Time (seconds)

Fig. 6 – Ability of SDN-controller-supervised networks to mitigate denial-of-service attacks. (a) Network traffic without SDN controller intervention and (b) network traffic with SDN controller intervention.

−3

10

0

1

2

3

4

5

6

Time (minutes)

Fig. 7 – Effect of a disruptive cyber attack on TCP packet roundtrip time.

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

networks where an attack could be isolated and stopped at the edge routers. Nevertheless, it is important to note once again that software-defined networking is an emerging technology, which needs to be thoroughly examined and validated before it can be embraced by the traditional IP networking community, and possibly by the industrial control system community as well. Therefore, while the experiment demonstrates certain advantages that software-defined networking can bring to large-scale industrial control systems, the technology is still in an embryonic state and its applicability even in the traditional IP networking domain has yet to be fully explored. Consequently, significant research is needed to redesign the software-defined networking architecture to incorporate security mechanisms and also to understand the opportunities that software-defined networking can provide to enhance industrial control system security.

6.

Conclusions

This paper has presented a systematic experimental analysis of the effects of network design choices on the outcome of cyber attacks targeting industrial control systems. Several design methodologies applied to industrial control systems were discussed in detail and security evaluations were performed using two attacks, one involving the actual Stuxnet malware and the other a synthetic denial-of-service attack. The experimental results demonstrate that network segmentation can isolate specific hosts and critical services, and when accompanied by firewalls, can block arbitrary packets originating from attackers. However, network segmentation and firewalls must be integrated in a defense-in-depth strategy that incorporates additional detection components. In particular, intrusion prevention systems, intrusion detection systems and anomaly detection systems deployed in different network segments and operating at different layers can significantly enhance the security of industrial control systems. A key contribution of this paper is the evaluation of state-ofthe-art IP networking technologies that could be applied to secure industrial control systems. One technology is MPLS networks, which are already being used to provide a reliable communications infrastructure for the Italian electric grid. Other emerging technologies include OpenFlow and softwaredefined networking that can enable dynamic closed-loop network traffic reconfiguration decisions. However, these two technologies are still in the embryonic state and a new round of research is needed in order to fully understand the security requirements of software-defined networking, to integrate security prerequisites into the core software-defined networking architecture and also to identify the security risks of software-defined industrial control networks. The main novelty of this paper, however, lies in the experimental assessment of various industrial control system security solutions under attacks by the Stuxnet malware. A typical industrial control architecture was recreated in a laboratory environment and its hosts were deliberately infected with Stuxnet to test various security solutions. The experiments confirm that the best solution to combating advanced malware is to employ a defense-in-depth strategy, where network segmentation is supplemented by firewalls

13

and intrusion detection and anomaly detection systems. The experiments also reveal that firewalls and network segmentation alone are ineffective against malware that exploits vulnerabilities in whitelisted protocols. Therefore, network segmentation and firewall configuration must be accompanied by detection systems and possibly by new technologies such as next generation firewalls. Future research will further explore the capabilities of OpenFlow and software-defined industrial control networks. Specifically, the research will evaluate traffic engineering algorithms that are triggered by detection engines to re-route malicious traffic and protect sensitive communications flows. Another line of research concerns the deployment of multiple hierarchical software-defined-networking-based controllers for which the implementation of protective mechanisms is a strong requirement.

Acknowledgments This research was supported by a Marie Curie FP7 Integration Grant under the 7th European Union Framework Programme (Grant no. PCIG14-GA-2013-631128). The research of Béla Genge was also supported by the TÁMOP-4.2.2.C-11/1/ KONV-2012-0001 Project of the European Union and cofinanced by the European Social Fund. This paper does not reflect the official opinion of the European Union. The authors are entirely responsible for the information and views expressed in the paper.

r e f e r e nc e s

[1] R. Ackerman, Software-defined networking looms as next big technology, Signal, May 12, 2014. [2] A. Akhunzada, E. Ahmed, A. Gani, M. Khan, M. Imran and S. Guizani, Securing software-defined networks: Taxonomy, requirements and open issues, IEEE Communications, vol. 53 (4), pp. 36–44, 2015. [3] R. Anderson and R. Hundley, The Implications of COTS Vulnerabilities for the DoD and Critical U.S. Infrastructures: What Can/Should the DoD Do? RAND/P-8031, RAND Corporation, Santa Monica, California, 1998. [4] R. Barbosa, R. Sadre and A. Pras, Towards periodicity based anomaly detection in SCADA networks, Proceedings of the Seventeenth International Conference on Emerging Technologies and Factory Automation, 2012. [5] R. Barbosa, R. Sadre and A. Pras, Flow whitelisting in SCADA networks, International Journal of Critical Infrastructure Protection, vol. 6(3–4), pp. 150–158, 2013. [6] S. Bellovin, S. Bradner, W. Diffie, S. Landau and J. Rexford, As simple as possible – But not more so, Communications of the ACM, vol. 54(8), pp. 30–33, 2011. [7] S. Bellovin, S. Bradner, W. Diffie, S. Landau and J. Rexford, Can it really work? Problems with extending EINSTEIN 3 to the critical infrastructure, Harvard National Security Journal, vol. 3(1), 2011. [8] E. Bilis, W. Kroger and C. Nan, Performance of electric power systems under physical malicious attacks, IEEE Systems Journal, vol. 7(4), pp. 854–865, 2013. [9] E. Byres, A. Ginter and J. Langill, How Stuxnet Spreads – A Study of Infection Paths in Best Practice Systems, White Paper, Tofino Security, Lantzville, Canada, 2011.

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

14

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

[10] A. Carcano, A. Coletta, M. Guglielmi, M. Masera, I. Nai Fovino and A. Trombetta, A multidimensional critical state analysis for detecting intrusions in SCADA systems, IEEE Transactions on Industrial Informatics, vol. 7(2), pp. 179–186, 2011. [11] A. Cardenas, S. Amin, Z. Lin, Y. Huang, C. Huang and S. Sastry, Attacks against process control systems: Risk assessment, detection and response, Proceedings of the Sixth ACM Symposium on Information, Computer and Communications Security, pp. 355–366, 2011. [12] M. Caselli, E. Zambon and F. Kargl, Sequence-aware intrusion detection in industrial control systems, Proceedings of the First ACM Workshop on Cyber-Physical System Security, pp. 13–24, 2015. [13] T. Chen and S. Abu-Nimeh, Lessons from Stuxnet, IEEE Computer, vol. 44(4), pp. 91–93, 2011. [14] X. Dong, H. Lin, R. Tan, R. Iyer and Z. Kalbarczyk, Softwaredefined networking for smart grid resilience: Opportunities and challenges, Proceedings of the First ACM Workshop on Cyber-Physical System Security, pp. 61–68, 2015. [15] Z. Durumeric, J. Kasten, D. Adrian, J. Halderman, M. Bailey, F. Li, N. Weaver, J. Amann, J. Beekman, M. Payer and V. Paxson, The matter of Heartbleed, Proceedings of the Internet Measurement Conference, pp. 475–488, 2014. [16] European Telecommunications Standards Institute, Network Functions Virtualization: An Introductory, Benefits, Enablers, Challenges and Call for Action, ETSI White Paper, SophiaAntipolis Cedex, France, 2012. [17] N. Falliere, L. O’Murchu and E. Chien, W32.Stuxnet Dossier, version 1.4, Symantec, Mountain View, California, 2010. [18] I. Garitano, C. Siaterlis, B. Genge, R. Uribeetxeberria and U. Zurutuza, A method to construct network traffic models for process control systems, Proceedings of the Seventeenth International Conference on Emerging Technologies and Factory Automation, 2012. [19] B. Genge, P. Haller, A. Gligor and A. Beres, An approach for cyber security experimentation supporting Sensei/IoT for smart grid, Proceedings of the Second International Symposium on Digital Forensics and Security, pp. 37–42, 2014. [20] B. Genge, D. Rusu and P. Haller, A connection pattern-based approach to detect network traffic anomalies in critical infrastructures, Proceedings of the Seventh European Workshop on System Security, article no. 1, 2014. [21] B. Genge and C. Siaterlis, Analysis of the effects of distributed denial-of-service attacks on MPLS networks, International Journal of Critical Infrastructure Protection, vol. 6(2), pp. 87–95, 2013. [22] B. Genge, C. Siaterlis and G. Karopoulos, Data fusion based anomaly detection in networked critical infrastructures, Proceedings of the Forty-Third Annual IEEE/IFIP Dependable Systems and Networks Workshop, pp. 1–8, 2013. [23] N. Goldenberg and A. Wool, Accurate modeling of Modbus/ TCP for intrusion detection in SCADA systems, International Journal of Critical Infrastructure Protection, vol. 6(2), pp. 63–75, 2013. [24] A. Goodney, S. Kumar, A. Ravi and Y. Cho, Efficient PMU networking with software-defined networks, Proceedings of the Fourth IEEE International Conference on Smart Grid Communications, pp. 378–383, 2013. [25] M. Hagerott, Stuxnet and the vital role of critical infrastructure operators and engineers, International Journal of Critical Infrastructure Protection, vol. 7(4), pp. 244–246, 2014. [26] N. Handigol, B. Heller, V. Jeyakumar, B. Lantz and N. McKeown, Reproducible network experiments using container-based emulation, Proceedings of the Eighth International Conference on Emerging Networking Experiments and Technologies, pp. 253–264, 2012. [27] IBM, Cisco and IBM provide high-voltage grid operator with increased reliability and manageability of its

[28]

[29]

[30]

[31]

[32]

[33]

[34] [35]

[36]

[37]

[38]

[39]

[40]

[41]

[42]

[43]

[44]

[45]

[46]

telecommunications infrastructure, Armonk, New York, November 6, 2007. Industrial Control Systems Cyber Emergency Response Team, Cyber Security Evaluation Tool (CSET), version 6.2, U.S. Department of Homeland Security, Washington, DC, 2014. Institute of Electrical and Electronics Engineers, IEEE 16462004 Standard: Communication Delivery Time Performance Requirements for Electric Power Substation Automation, New York, 2005. International Society of Automation, Security for Industrial Automation and Control Systems, Terminology, Concepts and Models, ISA-62443-1-1 (99.01.01), Research Triangle Park, North Carolina, 2013. M. Krotofil, A. Cardenas, J. Larsen and D. Gollmann, Vulnerabilities of cyber-physical systems to stale data – Determining the optimal time to launch attacks, International Journal of Critical Infrastructure Protection, vol. 7(4), pp. 213–232, 2014. D. Kuipers and M. Fabro, Control Systems Cyber Security: Defense-in-Depth Strategies, INL/EXT-06-11478, Idaho National Laboratory, Idaho Falls, Idaho, 2006. R. Langner, To Kill a Centrifuge: A Technical Analysis of What Stuxnet’s Creators Tried to Achieve, The Langner Group, Arlington, Virginia, 2013. R. Leszczyna, Approaching secure industrial control systems, IET Information Security, vol. 9(1), pp. 81–89, 2015. R. Leszczyna, I. Nai Fovino and M. Masera, Simulating malware with MAlSim, Journal in Computer Virology, vol. 6(1), pp. 65–75, 2008. R. Leszczyna, I. Nai Fovino and M. Masera, Approach to security assessment of critical infrastructure information systems, IET Information Security, vol. 5(3), pp. 135–144, 2011. M. Levorato and U. Mitra, Fast anomaly detection in smart grids via sparse approximation theory, Proceedings of the Seventh Sensor Array and Multichannel Signal Processing Workshop, pp. 5–8, 2012. N. McKeown, T. Anderson, G. Parulkar, L. Peterson, J. Rexford, S. Shenker and J. Turner, OpenFlow: Enabling innovation in campus networks, ACM SIGCOMM Computer Communication Review, vol. 38(2), pp. 69–74, 2008. E. Molina, E. Jacob, J. Matias, N. Moreira and A. Astarloa, Using software-defined networking to manage and control IEC 61850 based systems, Computers and Electrical Engineering, vol. 43, pp. 142–154, 2015. B. Moller, T. Duong and K. Kotowicz, This POODLE Bites: Exploiting the SSL 3.0 Fallback, Google, Mountain View, California, 2014. I. Nai Fovino, A. Coletta, A. Carcano and M. Masera, Critical state-based filtering system for securing SCADA network protocols, IEEE Transactions on Industrial Electronics, vol. 59(10), pp. 3943–3950, 2012. I. Nai Fovino and M. Masera, InSAW – Industrial Security Assessment Workbench, Proceedings of the First International Conference on Infrastructure Systems and Services: Building Networks for a Brighter Future, 2008. D. Nicol, W. Sanders, S. Singh and M. Seri, Usable global network access policy for process control systems, IEEE Security and Privacy, vol. 6(6), pp. 30–36, 2008. Open Networking Foundation, Software-Defined Networking (SDN) Definition, Palo Alto, California (www. opennetworking.org/sdn-resources/sdn-definition), 2015. M. Presser, P. Barnaghi, M. Eurich and C. Villalonga, The SENSEI Project: Integrating the physical world with the digital world of the network of the future, IEEE Communications, vol. 47(4), pp. 1–4, 2009. Project Floodlight, Project Floodlight (www.projectfloodlight. org), 2015.

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005

international journal of critical infrastructure protection ] (] ] ] ]) ] ] ] –] ] ]

[47] M. Raciti and S. Nadjm-Tehrani, Embedded cyber-physical anomaly detection in smart meters, Proceedings of the Seventh International Conference on Critical Information Infrastructures Security, pp. 34–45, 2012. [48] F. Schuster, A. Paul and H. Konig, Towards learning normality for anomaly detection in industrial control networks, Proceedings of the Seventh IFIP International Conference on Autonomous Infrastructure, Management and Security, pp. 61–72, 2013. [49] C. Siaterlis and B. Genge, Cyber-physical testbeds, Communications of the ACM, vol. 57(6), pp. 64–73, 2014. [50] sKyWIper Analysis Team, sKyWIper (a.k.a. Flame a.k.a. Flamer): A Complex Malware for Targeted Attacks, v1.05, Technical Report, Laboratory of Cryptography and System Security (CrySyS Lab), Department of Telecommunications, Budapest University of Technology and Economics, Budapest, Hungary, 2012. [51] J. Snyder, What is a next generation firewall? Network World, August 22, 2011. [52] SourceForge, Iperf (sourceforge.net/projects/iperf), 2013. [53] K. Stouffer, J. Falco and K. Scarfone, Guide to Industrial Control Systems (ICS) Security, NIST Special Publication 80082, National Institute of Standards and Technology, Gaithersburg, Maryland, 2011. [54] K. Stouffer, J. Falco and K. Scarfone, Guide to Industrial Control Systems (ICS) Security, Revision 1, NIST Special Publication 800-82, National Institute of Standards and Technology, Gaithersburg, Maryland, 2013.

15

[55] K. Stouffer, S. Lightman, V. Pillitteri, M. Abrams and A. Hahn, Guide to Industrial Control Systems (ICS) Security, NIST Special Publication 800-82, Revision 2, Final Public Draft, National Institute of Standards and Technology, Gaithersburg, Maryland, 2015. [56] N. Svendsen and S. Wolthusen, Modeling and detecting anomalies in SCADA systems, in Critical Infrastructure Protection II, M. Papa and S. Shenoi (Eds.), Springer, Boston, Massachusetts, pp. 101–113, 2008. [57] N. Svendsen and S. Wolthusen, Using physical models for anomaly detection in control systems, in Critical Infrastructure Protection III, C. Palmer and S. Shenoi (Eds.), Springer, Heidelberg, Germany, pp. 139–149, 2009. [58] Symantec, W32.Duqu: The Precursor to the Next Stuxnet, Mountain View, California, 2011. [59] Symantec, Dragonfly: Cyberespionage Attacks Against Energy Suppliers, version 1.21, Mountain View, California, 2014. [60] U.S. Department of Homeland Security, Recommended Practice: Improving Industrial Control Systems Cybersecurity with Defense-in-Depth Strategies, Washington, DC, 2009. [61] U.S. Department of Homeland Security, Cyber Security Assessments of Industrial Control Systems, Washington, DC, 2010. [62] J. Zhao, K. Liu, W. Wang and Y. Liu, Adaptive fuzzy clustering based anomaly data detection in energy systems of the steel industry, Information Sciences, vol. 259, pp. 335–345, 2014.

Please cite this article as: B. Genge, et al., Experimental assessment of network design approaches for protecting industrial control systems, International Journal of Critical Infrastructure Protection (2015), http://dx.doi.org/10.1016/j.ijcip.2015.07.005