- Email: [email protected]
Software defined network communications: The likely standard for smart grids
The Electricity Journal 32 (2019) 106639
Contents lists available at ScienceDirect
The Electricity Journal journal homepage: www.elsevier.com/locate/tej
Software defined network communications: The likely standard for smart grids
T
⁎
Victor Glassa, , Eric Woychikb, Ephram Glassac a
Rutgers Business School, 1 Washington Park, Newark, NJ 07102, United States Willdan, Oakland, California, United States c Smart Grid Consulting, Littleton, Colorado, United States b
A R T I C LE I N FO
A B S T R A C T
Keywords: Smart grid Software defined networks Network function virtualization
A smart grid generates massive amounts of data with many data streams transmitting at different speeds. A critical decision for an electric utility is what type of communication technology will best serve the needs of a smart grid. The objective of this paper is to show that a version of a Software Defined Network has the potential to accommodate the communication needs of the grid operator and other customer/supplier groups that connect to the grid.
1. Introduction Software Defined Networks can Accommodate Smart Grid Communication Needs In one respect, electric utilities are like cable companies in the 1990s: they are facing a major transformation of their networks from one-way to two-way delivery of service. But the similarity ends there. Residential customers may grouse when their cable service is out but are irate when they have no power. Keeping the lights on is increasingly a challenge because electric utilities are opening their networks to accommodate new energy sources and new customer uses that create network instabilities. Solar, wind and other renewables are replacing reliable fossil fuel energy sources. Microgrids are growing. Electric vehicles are beginning to plug into the grid. Monitoring the energy use of appliances, which is at the heart of demand response, may be a gateway for offering Internet of Things services. The evolving grid will require a communications network with many challenging requirements: It must transmit massive amounts of data with data streams transmitting at different speeds, some “bursty,” others periodic. “Event” notifications require priority handling while others such as meter reading will be at a much lower priority level. A critical decision for an electric utility is what type of communication technology will best serve the needs of a smart grid. We believe the likely winner in the broadband evolution is a version of a Software Defined Network (SDN) because it has the potential to accommodate the communication needs of the grid operator and other customer/supplier groups that connect to the grid. SDN is capable of
⁎
Corresponding author. E-mail address: [email protected] (V. Glass).
https://doi.org/10.1016/j.tej.2019.106639
1040-6190/ © 2019 Elsevier Inc. All rights reserved.
implementing quality of service (QoS) levels, testing new service categories, and assuring private and secure connections. It also promises scalability to grow with the large number of new devices that will be connected to the grid, and the interoperability with legacy communication systems. As we will show, SDN is also cost effective because it employs less sophisticated computers (switches and routers) than the Internet. Of course, there are hazards with a new approach to the communications network, but they are being worked on very actively by broadband and equipment companies. Perhaps the best endorsement for SDN is that broadband providers are rapidly transitioning to SDN (Robuck, 2019). The objective of this paper is to define SDN and contrast it with other types of broadband solutions for transmitting and receiving information. Specifically, SDN will be compared to basic best-effort communication that dominates the Internet and overlays to broadband networks used to introduce QoS. Section 2 briefly reviews smart grid requirements with a focus on new digital equipment. Section 3 describes the limitations of Internet Protocol and other packet labeling transmission techniques to accommodate a smart grid. Section 4 describes how basic SDN in combination with virtual machines configured by hypervisors can meet the evolving communication needs of the next generation smart grid. Section 5 summarizes SDN areas of research to understand its current limitations. Section 6 describes the authors’ proposal to develop two types of virtual communications networks: one for network monitoring and operations, one for customer communications. Section 7 concludes the paper.
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
2. Smart grid requirements
interfaces that display network performance in real time will be needed by systems operators (Fang et al., 2012, p. 946). Some data may be analyzed offline to perform real time stability studies to prevent system collapses (U.S. Department of Energy, 2015, p. 3.2). These management systems must be flexible enough to adapt to radical changes in grid design caused by a variety of technological changes and changes in customer behavior, including the growth in distributed energy resources (DER), power aggregation, use of renewable energy, customer sensitivity to “green” issues, increased number of microgrids, demand for electric vehicles (EVs), and other power-related services. Three examples are taken from this list of model changers to highlight the necessity of a sophisticated communication system to maintain high levels of reliability and resiliency: 1) power aggregation, 2) growth in microgrids, and 3) the expanding market for electric vehicles. The growth of DER increases the likelihood that localized grouping of generation will occur (Fang et al., 2012, p. 950). DER and demand response, or interruptible load, is increasingly aggregated into Virtual Power Plants (VPPs) to improve dispatchability and increase access to power markets (Wikipedia Virtual Power Plant, 2019). These aggregations have the potential to replace a conventional power plant and deliver extra benefits such as higher efficiency and more flexibility (Fang et al., 2012, p. 950). However a VPP is a complex system that requires a central controller to control, secure, and optimize performance (Fang et al., 2012, p. 950). More complicated integration of generation and transmission will result from VPPs. Increased formation of microgrids is a second game changer. A Microgrid is a localized grouping of generators, storage facilities, and loads to assure energy quality. Under normal operations, it is connected to the traditional power grid. Closed switches allow power to enter the microgrid from the traditional grid. When a power outage occurs, the switches are opened, and like raising a drawbridge, the microgrid is separated from the main grid and operates using its own power sources. Growth in the number of microgrids is a market signal that customers are willing to pay for quality of service (QoS). In other words, increased reliability and resilience are marketable products to new classes of customers. Marketing QoS tiers requires sophisticated network monitoring to assure that customer classes willing to invest in microgrids or other backup generation can receive the same services from the electric utility company. The third game-changer, Grid-to-Vehicle (G2V) and Vehicle-to-Grid (V2G), poses significant service quality challenges. The growth of EVs will be a significant new grid load source potentially driving up peak demands beyond current generation capacities. Power grid distribution connections will grow as charging stations are opened to make EV charging convenient. Uncoordinated charging could significantly degrade power system performance and efficiency (Fang et al., 2012, p. 951). Looking further ahead, millions of new connection nodes are not unreasonable. The critical issue is how to incorporate them all into the SG and meet customer needs and expectations. And even if this is done well, there are still two threats that must be addressed: the need for privacy and security. Criminals will have rich new data sources with high payoff potential. Compromising smart meters to manipulate energy costs and fabricate meter readings may become a fertile market (Fang et al., 2012, p. 969). And the meter itself is a rich source of information that can reveal individual habits, behaviors, preferences, and even beliefs – and whether someone is home (Fang et al., 2012, p. 970). These are issues that a smart communication system must address. Preventing network cyberattacks requires distinguishing between network failures and malicious intrusions. Adversaries can inject misinformation into the system by tapping into sensing and measuring devices, or they can attack central controllers (Fang et al., 2012, p. 969). They can also manipulate state data to trigger alarms (Fang et al., 2012, p. 970). Or they may launch Denial of Service attacks to overload information transmission.
A smart grid (SG) is a network with built in intelligence derived from a two way flow of data among network devices. The genesis of a SG is the new array of digital and computerized equipment embedded in the grid used to automate and manage an increasingly complex power delivery system. Like any computer network that delivers services – in this case, power – a SG has operating systems that manage physical devices memory, transmission links that monitor, shape, deliver power, and application programming interfaces that serve as a conduit for exposing the functionality of one software program to another. The network must also use cyber-secure communication technologies and computational intelligence to integrate electric power generation, transmission, and distribution (Fang, 2012, p. 944). A SG has multiple objectives starting with the broad objective of enhancing the capacity and efficiency of the existing network (Fang et al., 2012, p. 945). The U.S. Department of Commerce (What is a Smart Grid?) lists benefits associated with the Smart Grid:
• More efficient [and reliable] transmission of electricity. • Quicker restoration of electricity after power disturbances. • Integration and optimization of distributed energy resources (customer-side-of-the-meter). • Reduced utility operations and management costs to lower power costs for consumers. • Managed peak demand to lower electricity costs and provide ramping capacity. • Increased integration of all renewable energy sources. • Better integration of customer- side-of-the-meter resources, including energy efficiency. • Improved security. We focus mainly on the first two stated benefits because the cost of blackouts, brown-outs, short service interruption has made improving network reliability and resiliency top priorities. Grid reliability depends on failure prediction and prevention, while resiliency, quick recovery from failures, depends on fast failure identification, diagnosis, and recovery (Fang et al., 2012, p. 967). Both improved reliability and resiliency depend crucially on realizing the potential of new sensors and advanced metering systems that are being widely deployed. The U.S. Department of Energy (2015, p. iii) predicted that the United States would have 65 million smart meters nationwide by 2015. Synchrophasor equipment used to monitor network voltage and current would increase from 1000 to eventually 50,000.1 Most of the synchrophasors will be within the transmission segment of the grid but will eventually penetrate the distribution segment. The speed and latency requirements of sensing/communicating devices varies dramatically. Phasor Measurement Units (PMUs) measure voltage and current as often as 30–60 times per second, roughly the equivalent of streaming video (U.S. Department of Energy, 2015, p. 3.1). Residential usage data (meter reading) can range from seconds to 30 days (U.S. Department of Energy, 2015, p. 4.3). The location of sensing/communicating devices depends on their use. PMU (Phasor Measurement Unit) readings are typically from widely dispersed locations to measure network performance (Dept. of Energy, 2015, p. 4.3). Use of inverters in the grid to convert DC current to AC and flow control devices are proliferating (U.S. Department of Energy, 2015, p. 3.2) within the power distribution network (Wikipedia Power Inverter, 2019). New control management systems will be needed to integrate huge data flows of varying speeds clustered in different locations. Graphical
1 Phasors measure the magnitude and angle of sine waves found in electricity and when measurements are at the same time are called synchrophasor.
2
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
3. Limitation of traditional internet transmission
video over the Internet that mainly work with packet delays and buffering to provide acceptable voice and video services (Kurose and Ross, 2012, chapter 7). A popular alternative for transmitting data is to carve out transmission tunnels, that is, virtual circuits with their own dedicated routes and bandwidth to provide high-quality service. Multiple Protocol Label Switching (MPLS) is one such method. Special packet labels are used to bypass traditional Internet Protocol transmission, which is a common channel for all traffic. However, MPLS has a series of drawbacks: it was designed for point-to-point communications. Now, almost 50% of wide area network traffic accesses third-party data centers, such as Amazon’s. These services are defined as Cloud services. MPLS prices are also high and customers complain of long delays in provisioning the service (Ruparel, 2018). The Internet also has the distinct cost disadvantage of being at its core hardware focused. Largely autonomous switches and routers transmit data packets using TCP/IP protocols. Strategically placed devices have been developed to overcome other traffic management shortcomings:
The game-changing examples highlight that a successful SG requires a fast, flexible, reconfigurable, secure, and reliable communications network. Based on the U.S. Department of Energy (2015, pp. 3.1-3.5) below is a list of key requirements for SG operations:
• Increasing data volumes from new instrumentation. • Faster system dynamics. With increasing presence of solar and wind • • • • •
generation, and power electronics for inverters, active time scales are moving down to sub-seconds or even milliseconds. Responsive loads in the distribution network are adding a layer of complexity in grid control and management that didn’t exist before. Example: Demand Side Management and Demand Response are becoming automated at the commercial building level with the use of controls and behind-the-meter storage. Reactive loads and generation on the distribution system that are hidden from utilities and have the potential to cause operational issues. Associated with non-utility actors (customers or Independent System Operators) that can cause operational violations or blackouts. Increased renewable, variable generation is making load balancing more difficult. Bifurcation of the generation model from central stations connected to transmission to a mix with distributed generation connected to the distribution grid introduces multi-way real power flows. Change in fuel mix require joint gas and electric system planning.
• Gateways (allow entry and user authentication). • Encryption devices. • Deep Packet Inspection devices to manage packet streams (Kreutz et al., 2014, p. 4).
A basic take-away is that a “best effort” network originally designed to transmit datasets from university research centers has morphed to accommodate an online economy, but it has done so with hardware and labeling patches.
The SG communication system must have the capability to transmit data to and from many different devices, each with different transmission needs, and each located primarily in a specific grid segment. Data will be generated by residential meters, line sensors, substation transformers and remote terminal units, transmission phasor measurements in the distribution and transmission segments of the grid, and DER gateways (U.S. Department of Energy, 2015, pp. 4.7-4.8). The data streams, themselves, will have very different characteristics, from burst “event” messages to streaming PMU (U.S. Department of Energy, 2015, p. 4.13). And according to the U.S. Department of Energy (2015, p. 5.4) leading edge communication systems must seamlessly interconnect with legacy communication systems. Utilities have traditionally used multiple non-converged communication networks for telemetry, meter data, voice, digital field data, protection and control, inter-control center communication, video, and enterprise networking. Some utilities have had as many as nine separate communications networks, often owned and managed by different departments within a single utility. Some are as simple as twisted wire pair hub and spoke links from control centers to substations. Others make use of telephone leased lines; still others use fiber, various forms of wireless links, satellite links, and even power line communication (where the physical medium is the actual power line). A popular candidate for a standardized communications network is the basic network design used to carry Internet traffic. The Internet has an amazing track record. It allows many types of devices to communicate and transport data packets. It has expanded to meet fast-increasing volumes. It does carry voice, data, and video over the same network. And TCP/IP are the standard transmission protocols for highspeed wireless communication. In May 2018, Hawaiian Electric teamed with Verizon to install smart sensors in its grid that communicate using Verizon’s 4 G LTE network (Wu, 2018). However, the Internet network design has a fatal flaw for a SG communications network: Transmission is “best effort.” The basic Internet service model does not guarantee packet delivery, or bounded delay for delivered packets, or bandwidth for a particular services, or timing and ordering of delivered packets (Kurose and Ross, 2012, pp. 336-338). Workarounds have been developed to stream audio and
4. Software defined networks, a primer for smart grids Problems using an Internet transmission strategy first became apparent in data centers. In response to rapidly growing data and bandwidth capacity requirements, companies such as Google decided that instead of purchasing more physical computers and physically reconfiguring their networks to connect equipment to meet their needs, they could replace hardware with software. The original strategy was to separate computer routing strategies and instructions (the control plane) from the routing tables used to transmit data (the data plane). By making this separation, the controlling function could be centralized, either in one central controller for the entire network or a distributed network of controllers that work together. In either case, the controller function is concentrated, meaning that an SDN needs fewer sophisticated computers for transporting traffic. The controller function includes an Application Programming Interface (API), basically a set of subroutines that can be used to develop new software defined applications such as a gateway service, or a deep packet inspection service, or special routing. The routing, for example, can be programmed to accommodate basic IP/TCP, or special packet labeling such as MPLS for end-to-end defined routing (Goransson et al., 2017, p. 64). Administrators of the network can define other policies pertaining to access of the network’s assets, including quarantining misbehaving users (Goransson et al., 2017, p. 51). The most notable API is the open source OpenFlow protocol (Kreutz et al., 2014, p.2). An OpenFlow controller uses the OpenFlow protocol to communicate with the data-plane switches. The protocol consists of a set of messages exchanged between the controller and data plane switches. The messages can modify the data plane’s routing table (Goransson et al., 2017, p.93). Messages can also map traffic to port switches that have different packet queuing strategies. Ports with short packet waiting lines can deliver higher QoS (Goransson et al., 2017, p. 95). In effect, the original SDN could reconfigure traffic flow by software updates. A complementary development was the rethinking of network function applications that could be accomplished using software. This 3
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
view of network performance. Resilience is easily improved by using SDN to redirect data traffic flows from broken wired links to wireless links, and when a link fails SDNs can be programmed to redirecting traffic quickly. SDN technology is not vendor specific; therefore, different communication devices (including legacy) can work together.
led to the concepts of Network Function Virtualization (NFV) and with it the idea of a Virtual Machine (VM) providing Virtual Functions (VF) over a Virtual Network (VN). A VM is a software designed carve-out of a physical computer into separate virtual computers that can operate independently as switches, routers, load-balancers, and traffic shapers (Kreutz et al., 2014, p. 2). State of the art servers can conceivably host twenty VMs. In a data center that can accommodate 120,000 physical servers, the data center could interconnect 2,400,000 hosts (Goransson et al., 2017, p.6). Virtual functions could include data storage, real-time state of the network monitoring, and networking resources that would be combined into services by a virtual manager known as an orchestrator. A hypervisor is hardware or software that enables a physical computer to host several virtual machines. A hypervisor-based overlay network keeps the physical network configuration as it is but virtualizes the network (Goransson et al., 2017, p.84). For example, virtualized ports and paths can be used for building virtual tunnels of a specific design through the physical network. These tunnels can be defined and quickly reconfigured based on customer needs. By contrast, ordering new services from a traditional network could take weeks to provision because the network would need to be physically reconfigured (Kreutz et al., p. 5, for other examples of reconfiguration difficulties). SDN together with NFV development has reached the point where supporters such as Aydeger (2016, p. 15) cite the following potential advantages of systems based on the two related technologies:
5. SDN implementation challenges If the description of SDN seems too good to be true, you are right. SDN is still in its development stage with SDN developers trying many different approaches for improving the performance of SDN. A more fundamental problem with SDN is controller vulnerability. If a network has only one or even a few controllers, they become inviting targets for malicious attackers. By contrast, the intelligence of the Internet is widely distributed making it much less vulnerable to a system-wide shut down. Here we describe in some detail SDN developmental challenges. For starters, network equipment manufacturers and their customers have large embedded investments in equipment. It is uneconomic for them to simply divest themselves of legacy equipment. Instead, they will look for patches such as APIs that will work on legacy switches to perform some of the functions of an SDN network (Goransson et al., 2017, p.330). Because of the many objectives of a next generation communication system, no single design will fit all customer needs. The development of hypervisors is instructive. One way to virtualize an SDN is to either centralize the hypervisor function, which in effect means one central controller, or use distributed hypervisors for a network with more than one controller but all linked. Some hypervisors run on general purpose servers, some on specialized network equipment to accommodate specialized hypervisor functionalities (Blenk et al., 2016, p. 661). Hypervisors are being developed to address different network virtualization needs. Beginning with Flow Visor, the seminal hypervisor used as a controller, additional hypervisors were developed such as AdVisor to abstract and coordinate physical resources that were not possible with Flow Visor. There are many other hypervisors being tested for special networks and improved use of applications (Blenk et al., 2016, pp. 664670). Another example directly related to SG communications is instructive. In 2007–2009, the U.S. Department of Energy, the North American Electric Reliability Corporation, and North American electric utilities, consultants, federal and private researchers, and academics collaborated “to develop an ‘industrial grade’ secure, standardized, distributed, and expandable data communications infrastructure to support synchrophasor applications in North America” (Myrda and Koellner, 2019). The project was called the North American Synchrophasor Initiative (NASPI) and the proposed network design was called NASPInet. As with combined SDN and Network Function Virtualization, the basic strategy was to use APIs (software) to control this new network. The first specification of NASPInet was released in 2009 (Hu, 2009). In 2018, The U.S. Department of Energy released an assessment report. The report concludes that “many of designs concepts remain useful but implementation has fallen short.” A NASPInet 2.0 effort is underway. The assessment report recommends that NASPInet 2.0 should pay more attention to
• A global end-to-end view of the network. • Hardware virtualization that reduces hardware expenses by either • • • • •
reducing the amount of hardware or swapping cheaper hardware that can be software-configured for hardware with intelligence built into it. An open protocol standard that invites innovation. Rapid network reconfiguration as opposed to manual redesign. Enable better management and control of bandwidth. As a result, facilitates bandwidth on demand as opposed to fixed bandwidth specifications. QoS instead of best effort. Dedicated pathways to particular customers using routing rules instead of shared bandwidth.
SDN’s growing popularity among broadband carriers is evident. AT &T, for example, announced that it wants 75% of its network virtualized by 2020 (Robuck, 2019). The Metro Ethernet Forum (MEF) is actively developing technical standards for interconnecting broadband networks to provide SDN service across large service areas (MEF, SDWAN, 2019) and offering professional certifications for SDN (MEF, Professional Certification, 2019). SDN networks are projected to enable future wireless technologies such as 5 G by isolating bandwidth slices for voice, data, and video (Blenk et al., 2016, p. 656). The value of SDN for SG implementation is similar to that for other broadband communications networks. APIs are already available that allow for
• Rapid network reconfiguration. • Offering QoS levels. • Assuring security and privacy. • Enabling service orchestration.
• network logical level structure; • data management architecture including storage tiers and sharing of tiered stored data; • comprehensive cyber security; • distributed registry structure; • performance monitoring/performance management measures (Taft,
Building a test bed for simulating of new services (Rehmani et al., 2018, p. 8). Specific SDN advantages for SG include isolating different traffic types and applications. Separate virtual networks can be designed for real time, low latency PMU data and lower capacity, high latency smart meter data. SDN can even adapt a PMU’s measurement data traffic according to the capabilities of the receiving device. Traffic prioritization is a feature of SDN. SDN also has the distinct advantage of a global
2018 p. 15).
The really critical issue for SDN is network security, which is a must for a SG. In general, security threats are associated with accidents or 4
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
support data used to manage the grid on existing dedicated lines while enabling expanded use of an increasing amount of sensor data. A second VN can support meter data and enable access to power markets for each customer (to supply generation, demand response, ancillary services, energy efficiency, etc.).
attacks on a centralized controller, compromises of controllers through API abuse leading to re-routing sensitive traffic, increased risk of malware due to SDN’s use of simple, inexpensive network switches, lack of authentication and encryption protocols, and simply because SDN is a less mature technology than traditional network technology. Aydeger identifies four threat categories for a SG (2016, pp. 22-23):
7. Conclusion 1 2 3 4
Method-specific. Target-specific. Software-specific. Identity-specific.
Smart grids require sophisticated communication systems. Many electric utilities are transitioning towards IP-based networks. The key issue is whether to invest mainly in hardware-based networks with controller intelligence embedded in each device or to invest in networks that rely of more centralized software packages to control network services and performance. Broadband companies have placed their bets on SDN. Their investment “vote” makes sense because SDN reduces hardware costs, increases service offering flexibility. Broadband providers believe well-designed network software is customer empowering. A SDN enables customers to select from a menu of QoS levels and a host of security services that are available on short notice. SDNs also give the network operator much more control over network monitoring and reconfigurations. This type of control is critical for electric utilities. A great deal of network intelligence is necessary to control a two-way power grid with unreliable power sources from renewables. SDN promises to meet six critical SG needs: (1) transmit large volumes of data at (2) low latency, (3) control data streams to have virtually 100% reliability, (4) accommodate markets with many participants, and (5) facilitate cloud based data analysis. A sixth crucial need is to assure a secure network. Here the debate will likely center on answering a basic question: Is SDN as secure as networks without central controllers? Apparently, AT&T believes it is. We recommend a basic strategy of separating customer communications from grid management communications using a VN approach. The grid operations VN would provide utilities with capacity for large increases in sensor data and cloud-based analytics. The customer data VN would give customers greater access to power markets and would give utilities and ISOs greater capacity to utilize meter data to improve grid efficiency and reliability.
Method-specific is either passive eavesdropping on SG communications to gather data or actively inserting fake messages or malformed packets into the SDN data stream. Target-specific threats are potential attacks on digital devices – intelligent electronic devices, which are controllers of network equipment and Phasor Measurement Units. Software-specific threats target the protocols that send, transmit and receive messages between the controller and the rest of the network. Identity-specific threats are either insiders or outsiders who seek to undermine network performance. SDN and other computer networks must address all these threats, but because SDN’s intelligence is more centralized than the Internet, the security risks are higher. 6. Proposed strategic design using SDN Currently, utilities have multiple communication systems that, for the most part, are operated independently from one another. For example, a utility may have one communication system for dispatching field personnel, a multitude of independent communication lines for grid protection, several networks for remotely monitoring and operating switches, and another communications system for meter readings. In general, existing utility communications can be divided into two groups: data communications used to manually or automatically manage the grid and communications between customer meters and the utility. Communications used to manage the grid include: Supervisory Control and Data Acquisition (SCADA) systems that monitor power flows and voltage and allow System Operators to remotely operate switches, protection systems that control breaker operations for faults and special protection systems that perform switching to protect against frequency or voltage collapse, and sensor data sent from devices in the field that is used for real-time stability control, protection against equipment failure, and outage restoration. Communications between customer meters and the utility include: power consumption for billing purposes, controls to remotely connect or disconnect meters (on later smart meter models), power factor and voltage readings, meter pinging to identify outage locations, and power generation (if separately metered). Due to security concerns and data criticality, most communications currently used to manage the grid are sent via dedicated circuits owned by telecommunications companies or via utility owned wireless and fiber networks. On the other hand, customer meter data are sent via cellular networks, radio networks, power lines, or some combination of communication systems and typically gathered at substations before being sent to the central utility data management systems by yet another communication method – usually dedicated lines. The data stream used to manage the grid is the most important to guard against possible cyberattacks. In fact, the typical method utilities use to secure critical cyber assets is simply to isolate them from the internet. In contrast, utilities view data provided from customer meters as significantly less critical. For utilities to embrace a new communications architecture, the less secure communications with customer meters must remain isolated from the data networks managing the grid operations. This security concern and the existing separation of communication infrastructure lends itself to a two VN solution: One VN can
References Aydeger, Abdullah, 2016. Software Defined Networking for Smart Grid Communications. Available at. Florida International University Thesis. https://digitalcommons.fiu. edu/etd/2580/. Blenk, Andreas, Basta, Arsany, Reisslein, Martin, Keller, Wolfgang, 2016. Survey on network virtualization hypervisors for software defined networking. Ieee Commun. Surv. Tutor. 18 (1) First Quarter 2016. Available at. http://mre.faculty.asu.edu/ SDNHVsurv.pdf. Fang, Xi, Misra, Satyajayant, Xue, Guoliang, Yang, Dejun, 2012. Samrt grid – the new and improved power grid. IEEE Communications Survey and Tutorials 14 (4) Fourth Quarter 2012. Available at. https://ieeexplore.ieee.org/document/6099519. Goransson, Paul, Black, Chuck, Culver, Timothy, 2017. Software Defined Networks: A Comprehensive Approach, 2nd ed. Elsevier, New York. Hu, Yi, 2009. Phasor Gateway Technical Specifications for North American Synchrophasor Initiative (NASPInet). project manager. U.S. Department of Energy Available at file:///C:/Users/vglass.BUSINESS/Documents/Documents/rbr/rbr/Grid %20Communication/naspinet_phasor_gateway__final_spec_20090529.pdf. Kreutz, Diego, Ramos, Fernando, Verissimo, Paulo, Rothenberg, Christian, Azodolmolky, Siamak, Uhlig, Steve, 2014. Software –Defined Networking: A Comprehensive Survey. Available at. IEEE. https://arxiv.org/pdf/1406.0440.pdf. Metro Ethernet Forum, SD-WAN from technology to service standard. Available at https://www.mef.net/mef-3-0-sd-wan. Metro Ethernet Forum. MEF Professional Certification. Available at https://www.mefprocert.com/. Myrda, Paul, Koellner, Kris, 2019. NASPInet –the Internet for Synchrophasors. Available at. IEEE Xplore Digital Library. https://ieeexplore.ieee.org/document/5428363. Rehmani, Mubashir, Davy, Alan, Jennings, Brendan, Assi, Chadi, 2018. Software Defined Networks Based Smart Grid Communication: a Comprehensive Survey. Available at. https://arxiv.org/abs/1801.04613. Ruparel, Jig, 2018. How SD-WAN As a Service Solves MPLS Limitations. Available at. sdx central. https://www.sdxcentral.com/articles/contributed/sd-wan-as-a-servicesolves-mpls-limitations/2018/10/. Robuck, Mike, 2019. AT&T On the Homestretch of Virtualization Goal. Available at. FierceTelecom. https://www.fiercetelecom.com/telecom/at-t-homestretchvirtualization-goal. Taft, J.D., 2018. Assessment of Existing Synchrophasor Networks. Available at. U. S.
5
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
service reform, and new access services. He is the lead author of many business and academic studies. Dr. Glass earned his MBA in marketing and finance, and Ph.D. in economics from Columbia University.
Department of Energy; Prepared by Pacific Northwest National Laboratory. https:// gridarchitecture.pnnl.gov/media/white-papers/Synchrophasor_net_assessment_final. pdf. U. S. Department of Energy, 2015. The Emerging Interdependence of the Electric Power Grid & Information and Communication Technology. Available at. Prepared by Pacific Northwest National Laboratory. https://www.pnnl.gov/main/publications/ external/technical_reports/PNNL-24643.pdf. Wikipedia, Power Inverter. Available at https://en.wikipedia.org/wiki/Power_inverter. Wikipedia, Virtual Power Plant. Available at https://en.wikipedia.org/wiki/Virtual_ power_plant. Wu, Nina, 2018. Verizon and Hawaiian Electric Partner to Install Smart Sensors on the Grid. Star Advertiser, July 17, 2018. Available at. https://www.staradvertiser.com/ 2018/07/17/business/business-breaking/verizon-heco-partner-to-install-smartsensors-on-grid/.
Dr. Eric Woychik brings more than 40 years of regulatory policy expertise in the cleanenergy and utilities arena, and has worked on behalf of a broad range of organizations including the California PUC and the New York PSC. He has provided research and analysis in topical areas such as smart grid integration, rate design and revenue allocation, as well as in DER project development and financing. Eric holds a Doctorate (Energy Market Game Theory) from Case Western Reserve University; an M.A. (Economics) from New Mexico State University; and a B.S. (Environmental Policy Analysis and Planning) from the University of California, Davis. Ephram Glass is an independent utility consultant specializing in smart grid services and enterprise asset management. In his thirteen years in the industry, he has worked on load growth planning, renewable energy and demand response regulatory affairs, generator interconnections, compliance, cyber security, transmission and distribution automation, reliability, analytics, asset accounting, resource management, and project management. He earned a Bachelor of Science degree in Electrical Engineering from Princeton University.
Victor Glass is Director, CRRI Scholar, and Professor of Professional Practice - Finance and Economics, Rutgers Business School - Newark and New Brunswick, Rutgers University. Prior to joining Rutgers, Dr. Glass was Director of Demand Forecasting and Rate Development at the National Exchange Carrier Association. For almost thirty years, he was responsible for forecasting demand and setting switched and special rates for more than 1100 telephone companies. He was heavily involved in access restructure, universal
6
Contents lists available at ScienceDirect
The Electricity Journal journal homepage: www.elsevier.com/locate/tej
Software defined network communications: The likely standard for smart grids
T
⁎
Victor Glassa, , Eric Woychikb, Ephram Glassac a
Rutgers Business School, 1 Washington Park, Newark, NJ 07102, United States Willdan, Oakland, California, United States c Smart Grid Consulting, Littleton, Colorado, United States b
A R T I C LE I N FO
A B S T R A C T
Keywords: Smart grid Software defined networks Network function virtualization
A smart grid generates massive amounts of data with many data streams transmitting at different speeds. A critical decision for an electric utility is what type of communication technology will best serve the needs of a smart grid. The objective of this paper is to show that a version of a Software Defined Network has the potential to accommodate the communication needs of the grid operator and other customer/supplier groups that connect to the grid.
1. Introduction Software Defined Networks can Accommodate Smart Grid Communication Needs In one respect, electric utilities are like cable companies in the 1990s: they are facing a major transformation of their networks from one-way to two-way delivery of service. But the similarity ends there. Residential customers may grouse when their cable service is out but are irate when they have no power. Keeping the lights on is increasingly a challenge because electric utilities are opening their networks to accommodate new energy sources and new customer uses that create network instabilities. Solar, wind and other renewables are replacing reliable fossil fuel energy sources. Microgrids are growing. Electric vehicles are beginning to plug into the grid. Monitoring the energy use of appliances, which is at the heart of demand response, may be a gateway for offering Internet of Things services. The evolving grid will require a communications network with many challenging requirements: It must transmit massive amounts of data with data streams transmitting at different speeds, some “bursty,” others periodic. “Event” notifications require priority handling while others such as meter reading will be at a much lower priority level. A critical decision for an electric utility is what type of communication technology will best serve the needs of a smart grid. We believe the likely winner in the broadband evolution is a version of a Software Defined Network (SDN) because it has the potential to accommodate the communication needs of the grid operator and other customer/supplier groups that connect to the grid. SDN is capable of
⁎
Corresponding author. E-mail address: [email protected] (V. Glass).
https://doi.org/10.1016/j.tej.2019.106639
1040-6190/ © 2019 Elsevier Inc. All rights reserved.
implementing quality of service (QoS) levels, testing new service categories, and assuring private and secure connections. It also promises scalability to grow with the large number of new devices that will be connected to the grid, and the interoperability with legacy communication systems. As we will show, SDN is also cost effective because it employs less sophisticated computers (switches and routers) than the Internet. Of course, there are hazards with a new approach to the communications network, but they are being worked on very actively by broadband and equipment companies. Perhaps the best endorsement for SDN is that broadband providers are rapidly transitioning to SDN (Robuck, 2019). The objective of this paper is to define SDN and contrast it with other types of broadband solutions for transmitting and receiving information. Specifically, SDN will be compared to basic best-effort communication that dominates the Internet and overlays to broadband networks used to introduce QoS. Section 2 briefly reviews smart grid requirements with a focus on new digital equipment. Section 3 describes the limitations of Internet Protocol and other packet labeling transmission techniques to accommodate a smart grid. Section 4 describes how basic SDN in combination with virtual machines configured by hypervisors can meet the evolving communication needs of the next generation smart grid. Section 5 summarizes SDN areas of research to understand its current limitations. Section 6 describes the authors’ proposal to develop two types of virtual communications networks: one for network monitoring and operations, one for customer communications. Section 7 concludes the paper.
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
2. Smart grid requirements
interfaces that display network performance in real time will be needed by systems operators (Fang et al., 2012, p. 946). Some data may be analyzed offline to perform real time stability studies to prevent system collapses (U.S. Department of Energy, 2015, p. 3.2). These management systems must be flexible enough to adapt to radical changes in grid design caused by a variety of technological changes and changes in customer behavior, including the growth in distributed energy resources (DER), power aggregation, use of renewable energy, customer sensitivity to “green” issues, increased number of microgrids, demand for electric vehicles (EVs), and other power-related services. Three examples are taken from this list of model changers to highlight the necessity of a sophisticated communication system to maintain high levels of reliability and resiliency: 1) power aggregation, 2) growth in microgrids, and 3) the expanding market for electric vehicles. The growth of DER increases the likelihood that localized grouping of generation will occur (Fang et al., 2012, p. 950). DER and demand response, or interruptible load, is increasingly aggregated into Virtual Power Plants (VPPs) to improve dispatchability and increase access to power markets (Wikipedia Virtual Power Plant, 2019). These aggregations have the potential to replace a conventional power plant and deliver extra benefits such as higher efficiency and more flexibility (Fang et al., 2012, p. 950). However a VPP is a complex system that requires a central controller to control, secure, and optimize performance (Fang et al., 2012, p. 950). More complicated integration of generation and transmission will result from VPPs. Increased formation of microgrids is a second game changer. A Microgrid is a localized grouping of generators, storage facilities, and loads to assure energy quality. Under normal operations, it is connected to the traditional power grid. Closed switches allow power to enter the microgrid from the traditional grid. When a power outage occurs, the switches are opened, and like raising a drawbridge, the microgrid is separated from the main grid and operates using its own power sources. Growth in the number of microgrids is a market signal that customers are willing to pay for quality of service (QoS). In other words, increased reliability and resilience are marketable products to new classes of customers. Marketing QoS tiers requires sophisticated network monitoring to assure that customer classes willing to invest in microgrids or other backup generation can receive the same services from the electric utility company. The third game-changer, Grid-to-Vehicle (G2V) and Vehicle-to-Grid (V2G), poses significant service quality challenges. The growth of EVs will be a significant new grid load source potentially driving up peak demands beyond current generation capacities. Power grid distribution connections will grow as charging stations are opened to make EV charging convenient. Uncoordinated charging could significantly degrade power system performance and efficiency (Fang et al., 2012, p. 951). Looking further ahead, millions of new connection nodes are not unreasonable. The critical issue is how to incorporate them all into the SG and meet customer needs and expectations. And even if this is done well, there are still two threats that must be addressed: the need for privacy and security. Criminals will have rich new data sources with high payoff potential. Compromising smart meters to manipulate energy costs and fabricate meter readings may become a fertile market (Fang et al., 2012, p. 969). And the meter itself is a rich source of information that can reveal individual habits, behaviors, preferences, and even beliefs – and whether someone is home (Fang et al., 2012, p. 970). These are issues that a smart communication system must address. Preventing network cyberattacks requires distinguishing between network failures and malicious intrusions. Adversaries can inject misinformation into the system by tapping into sensing and measuring devices, or they can attack central controllers (Fang et al., 2012, p. 969). They can also manipulate state data to trigger alarms (Fang et al., 2012, p. 970). Or they may launch Denial of Service attacks to overload information transmission.
A smart grid (SG) is a network with built in intelligence derived from a two way flow of data among network devices. The genesis of a SG is the new array of digital and computerized equipment embedded in the grid used to automate and manage an increasingly complex power delivery system. Like any computer network that delivers services – in this case, power – a SG has operating systems that manage physical devices memory, transmission links that monitor, shape, deliver power, and application programming interfaces that serve as a conduit for exposing the functionality of one software program to another. The network must also use cyber-secure communication technologies and computational intelligence to integrate electric power generation, transmission, and distribution (Fang, 2012, p. 944). A SG has multiple objectives starting with the broad objective of enhancing the capacity and efficiency of the existing network (Fang et al., 2012, p. 945). The U.S. Department of Commerce (What is a Smart Grid?) lists benefits associated with the Smart Grid:
• More efficient [and reliable] transmission of electricity. • Quicker restoration of electricity after power disturbances. • Integration and optimization of distributed energy resources (customer-side-of-the-meter). • Reduced utility operations and management costs to lower power costs for consumers. • Managed peak demand to lower electricity costs and provide ramping capacity. • Increased integration of all renewable energy sources. • Better integration of customer- side-of-the-meter resources, including energy efficiency. • Improved security. We focus mainly on the first two stated benefits because the cost of blackouts, brown-outs, short service interruption has made improving network reliability and resiliency top priorities. Grid reliability depends on failure prediction and prevention, while resiliency, quick recovery from failures, depends on fast failure identification, diagnosis, and recovery (Fang et al., 2012, p. 967). Both improved reliability and resiliency depend crucially on realizing the potential of new sensors and advanced metering systems that are being widely deployed. The U.S. Department of Energy (2015, p. iii) predicted that the United States would have 65 million smart meters nationwide by 2015. Synchrophasor equipment used to monitor network voltage and current would increase from 1000 to eventually 50,000.1 Most of the synchrophasors will be within the transmission segment of the grid but will eventually penetrate the distribution segment. The speed and latency requirements of sensing/communicating devices varies dramatically. Phasor Measurement Units (PMUs) measure voltage and current as often as 30–60 times per second, roughly the equivalent of streaming video (U.S. Department of Energy, 2015, p. 3.1). Residential usage data (meter reading) can range from seconds to 30 days (U.S. Department of Energy, 2015, p. 4.3). The location of sensing/communicating devices depends on their use. PMU (Phasor Measurement Unit) readings are typically from widely dispersed locations to measure network performance (Dept. of Energy, 2015, p. 4.3). Use of inverters in the grid to convert DC current to AC and flow control devices are proliferating (U.S. Department of Energy, 2015, p. 3.2) within the power distribution network (Wikipedia Power Inverter, 2019). New control management systems will be needed to integrate huge data flows of varying speeds clustered in different locations. Graphical
1 Phasors measure the magnitude and angle of sine waves found in electricity and when measurements are at the same time are called synchrophasor.
2
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
3. Limitation of traditional internet transmission
video over the Internet that mainly work with packet delays and buffering to provide acceptable voice and video services (Kurose and Ross, 2012, chapter 7). A popular alternative for transmitting data is to carve out transmission tunnels, that is, virtual circuits with their own dedicated routes and bandwidth to provide high-quality service. Multiple Protocol Label Switching (MPLS) is one such method. Special packet labels are used to bypass traditional Internet Protocol transmission, which is a common channel for all traffic. However, MPLS has a series of drawbacks: it was designed for point-to-point communications. Now, almost 50% of wide area network traffic accesses third-party data centers, such as Amazon’s. These services are defined as Cloud services. MPLS prices are also high and customers complain of long delays in provisioning the service (Ruparel, 2018). The Internet also has the distinct cost disadvantage of being at its core hardware focused. Largely autonomous switches and routers transmit data packets using TCP/IP protocols. Strategically placed devices have been developed to overcome other traffic management shortcomings:
The game-changing examples highlight that a successful SG requires a fast, flexible, reconfigurable, secure, and reliable communications network. Based on the U.S. Department of Energy (2015, pp. 3.1-3.5) below is a list of key requirements for SG operations:
• Increasing data volumes from new instrumentation. • Faster system dynamics. With increasing presence of solar and wind • • • • •
generation, and power electronics for inverters, active time scales are moving down to sub-seconds or even milliseconds. Responsive loads in the distribution network are adding a layer of complexity in grid control and management that didn’t exist before. Example: Demand Side Management and Demand Response are becoming automated at the commercial building level with the use of controls and behind-the-meter storage. Reactive loads and generation on the distribution system that are hidden from utilities and have the potential to cause operational issues. Associated with non-utility actors (customers or Independent System Operators) that can cause operational violations or blackouts. Increased renewable, variable generation is making load balancing more difficult. Bifurcation of the generation model from central stations connected to transmission to a mix with distributed generation connected to the distribution grid introduces multi-way real power flows. Change in fuel mix require joint gas and electric system planning.
• Gateways (allow entry and user authentication). • Encryption devices. • Deep Packet Inspection devices to manage packet streams (Kreutz et al., 2014, p. 4).
A basic take-away is that a “best effort” network originally designed to transmit datasets from university research centers has morphed to accommodate an online economy, but it has done so with hardware and labeling patches.
The SG communication system must have the capability to transmit data to and from many different devices, each with different transmission needs, and each located primarily in a specific grid segment. Data will be generated by residential meters, line sensors, substation transformers and remote terminal units, transmission phasor measurements in the distribution and transmission segments of the grid, and DER gateways (U.S. Department of Energy, 2015, pp. 4.7-4.8). The data streams, themselves, will have very different characteristics, from burst “event” messages to streaming PMU (U.S. Department of Energy, 2015, p. 4.13). And according to the U.S. Department of Energy (2015, p. 5.4) leading edge communication systems must seamlessly interconnect with legacy communication systems. Utilities have traditionally used multiple non-converged communication networks for telemetry, meter data, voice, digital field data, protection and control, inter-control center communication, video, and enterprise networking. Some utilities have had as many as nine separate communications networks, often owned and managed by different departments within a single utility. Some are as simple as twisted wire pair hub and spoke links from control centers to substations. Others make use of telephone leased lines; still others use fiber, various forms of wireless links, satellite links, and even power line communication (where the physical medium is the actual power line). A popular candidate for a standardized communications network is the basic network design used to carry Internet traffic. The Internet has an amazing track record. It allows many types of devices to communicate and transport data packets. It has expanded to meet fast-increasing volumes. It does carry voice, data, and video over the same network. And TCP/IP are the standard transmission protocols for highspeed wireless communication. In May 2018, Hawaiian Electric teamed with Verizon to install smart sensors in its grid that communicate using Verizon’s 4 G LTE network (Wu, 2018). However, the Internet network design has a fatal flaw for a SG communications network: Transmission is “best effort.” The basic Internet service model does not guarantee packet delivery, or bounded delay for delivered packets, or bandwidth for a particular services, or timing and ordering of delivered packets (Kurose and Ross, 2012, pp. 336-338). Workarounds have been developed to stream audio and
4. Software defined networks, a primer for smart grids Problems using an Internet transmission strategy first became apparent in data centers. In response to rapidly growing data and bandwidth capacity requirements, companies such as Google decided that instead of purchasing more physical computers and physically reconfiguring their networks to connect equipment to meet their needs, they could replace hardware with software. The original strategy was to separate computer routing strategies and instructions (the control plane) from the routing tables used to transmit data (the data plane). By making this separation, the controlling function could be centralized, either in one central controller for the entire network or a distributed network of controllers that work together. In either case, the controller function is concentrated, meaning that an SDN needs fewer sophisticated computers for transporting traffic. The controller function includes an Application Programming Interface (API), basically a set of subroutines that can be used to develop new software defined applications such as a gateway service, or a deep packet inspection service, or special routing. The routing, for example, can be programmed to accommodate basic IP/TCP, or special packet labeling such as MPLS for end-to-end defined routing (Goransson et al., 2017, p. 64). Administrators of the network can define other policies pertaining to access of the network’s assets, including quarantining misbehaving users (Goransson et al., 2017, p. 51). The most notable API is the open source OpenFlow protocol (Kreutz et al., 2014, p.2). An OpenFlow controller uses the OpenFlow protocol to communicate with the data-plane switches. The protocol consists of a set of messages exchanged between the controller and data plane switches. The messages can modify the data plane’s routing table (Goransson et al., 2017, p.93). Messages can also map traffic to port switches that have different packet queuing strategies. Ports with short packet waiting lines can deliver higher QoS (Goransson et al., 2017, p. 95). In effect, the original SDN could reconfigure traffic flow by software updates. A complementary development was the rethinking of network function applications that could be accomplished using software. This 3
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
view of network performance. Resilience is easily improved by using SDN to redirect data traffic flows from broken wired links to wireless links, and when a link fails SDNs can be programmed to redirecting traffic quickly. SDN technology is not vendor specific; therefore, different communication devices (including legacy) can work together.
led to the concepts of Network Function Virtualization (NFV) and with it the idea of a Virtual Machine (VM) providing Virtual Functions (VF) over a Virtual Network (VN). A VM is a software designed carve-out of a physical computer into separate virtual computers that can operate independently as switches, routers, load-balancers, and traffic shapers (Kreutz et al., 2014, p. 2). State of the art servers can conceivably host twenty VMs. In a data center that can accommodate 120,000 physical servers, the data center could interconnect 2,400,000 hosts (Goransson et al., 2017, p.6). Virtual functions could include data storage, real-time state of the network monitoring, and networking resources that would be combined into services by a virtual manager known as an orchestrator. A hypervisor is hardware or software that enables a physical computer to host several virtual machines. A hypervisor-based overlay network keeps the physical network configuration as it is but virtualizes the network (Goransson et al., 2017, p.84). For example, virtualized ports and paths can be used for building virtual tunnels of a specific design through the physical network. These tunnels can be defined and quickly reconfigured based on customer needs. By contrast, ordering new services from a traditional network could take weeks to provision because the network would need to be physically reconfigured (Kreutz et al., p. 5, for other examples of reconfiguration difficulties). SDN together with NFV development has reached the point where supporters such as Aydeger (2016, p. 15) cite the following potential advantages of systems based on the two related technologies:
5. SDN implementation challenges If the description of SDN seems too good to be true, you are right. SDN is still in its development stage with SDN developers trying many different approaches for improving the performance of SDN. A more fundamental problem with SDN is controller vulnerability. If a network has only one or even a few controllers, they become inviting targets for malicious attackers. By contrast, the intelligence of the Internet is widely distributed making it much less vulnerable to a system-wide shut down. Here we describe in some detail SDN developmental challenges. For starters, network equipment manufacturers and their customers have large embedded investments in equipment. It is uneconomic for them to simply divest themselves of legacy equipment. Instead, they will look for patches such as APIs that will work on legacy switches to perform some of the functions of an SDN network (Goransson et al., 2017, p.330). Because of the many objectives of a next generation communication system, no single design will fit all customer needs. The development of hypervisors is instructive. One way to virtualize an SDN is to either centralize the hypervisor function, which in effect means one central controller, or use distributed hypervisors for a network with more than one controller but all linked. Some hypervisors run on general purpose servers, some on specialized network equipment to accommodate specialized hypervisor functionalities (Blenk et al., 2016, p. 661). Hypervisors are being developed to address different network virtualization needs. Beginning with Flow Visor, the seminal hypervisor used as a controller, additional hypervisors were developed such as AdVisor to abstract and coordinate physical resources that were not possible with Flow Visor. There are many other hypervisors being tested for special networks and improved use of applications (Blenk et al., 2016, pp. 664670). Another example directly related to SG communications is instructive. In 2007–2009, the U.S. Department of Energy, the North American Electric Reliability Corporation, and North American electric utilities, consultants, federal and private researchers, and academics collaborated “to develop an ‘industrial grade’ secure, standardized, distributed, and expandable data communications infrastructure to support synchrophasor applications in North America” (Myrda and Koellner, 2019). The project was called the North American Synchrophasor Initiative (NASPI) and the proposed network design was called NASPInet. As with combined SDN and Network Function Virtualization, the basic strategy was to use APIs (software) to control this new network. The first specification of NASPInet was released in 2009 (Hu, 2009). In 2018, The U.S. Department of Energy released an assessment report. The report concludes that “many of designs concepts remain useful but implementation has fallen short.” A NASPInet 2.0 effort is underway. The assessment report recommends that NASPInet 2.0 should pay more attention to
• A global end-to-end view of the network. • Hardware virtualization that reduces hardware expenses by either • • • • •
reducing the amount of hardware or swapping cheaper hardware that can be software-configured for hardware with intelligence built into it. An open protocol standard that invites innovation. Rapid network reconfiguration as opposed to manual redesign. Enable better management and control of bandwidth. As a result, facilitates bandwidth on demand as opposed to fixed bandwidth specifications. QoS instead of best effort. Dedicated pathways to particular customers using routing rules instead of shared bandwidth.
SDN’s growing popularity among broadband carriers is evident. AT &T, for example, announced that it wants 75% of its network virtualized by 2020 (Robuck, 2019). The Metro Ethernet Forum (MEF) is actively developing technical standards for interconnecting broadband networks to provide SDN service across large service areas (MEF, SDWAN, 2019) and offering professional certifications for SDN (MEF, Professional Certification, 2019). SDN networks are projected to enable future wireless technologies such as 5 G by isolating bandwidth slices for voice, data, and video (Blenk et al., 2016, p. 656). The value of SDN for SG implementation is similar to that for other broadband communications networks. APIs are already available that allow for
• Rapid network reconfiguration. • Offering QoS levels. • Assuring security and privacy. • Enabling service orchestration.
• network logical level structure; • data management architecture including storage tiers and sharing of tiered stored data; • comprehensive cyber security; • distributed registry structure; • performance monitoring/performance management measures (Taft,
Building a test bed for simulating of new services (Rehmani et al., 2018, p. 8). Specific SDN advantages for SG include isolating different traffic types and applications. Separate virtual networks can be designed for real time, low latency PMU data and lower capacity, high latency smart meter data. SDN can even adapt a PMU’s measurement data traffic according to the capabilities of the receiving device. Traffic prioritization is a feature of SDN. SDN also has the distinct advantage of a global
2018 p. 15).
The really critical issue for SDN is network security, which is a must for a SG. In general, security threats are associated with accidents or 4
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
support data used to manage the grid on existing dedicated lines while enabling expanded use of an increasing amount of sensor data. A second VN can support meter data and enable access to power markets for each customer (to supply generation, demand response, ancillary services, energy efficiency, etc.).
attacks on a centralized controller, compromises of controllers through API abuse leading to re-routing sensitive traffic, increased risk of malware due to SDN’s use of simple, inexpensive network switches, lack of authentication and encryption protocols, and simply because SDN is a less mature technology than traditional network technology. Aydeger identifies four threat categories for a SG (2016, pp. 22-23):
7. Conclusion 1 2 3 4
Method-specific. Target-specific. Software-specific. Identity-specific.
Smart grids require sophisticated communication systems. Many electric utilities are transitioning towards IP-based networks. The key issue is whether to invest mainly in hardware-based networks with controller intelligence embedded in each device or to invest in networks that rely of more centralized software packages to control network services and performance. Broadband companies have placed their bets on SDN. Their investment “vote” makes sense because SDN reduces hardware costs, increases service offering flexibility. Broadband providers believe well-designed network software is customer empowering. A SDN enables customers to select from a menu of QoS levels and a host of security services that are available on short notice. SDNs also give the network operator much more control over network monitoring and reconfigurations. This type of control is critical for electric utilities. A great deal of network intelligence is necessary to control a two-way power grid with unreliable power sources from renewables. SDN promises to meet six critical SG needs: (1) transmit large volumes of data at (2) low latency, (3) control data streams to have virtually 100% reliability, (4) accommodate markets with many participants, and (5) facilitate cloud based data analysis. A sixth crucial need is to assure a secure network. Here the debate will likely center on answering a basic question: Is SDN as secure as networks without central controllers? Apparently, AT&T believes it is. We recommend a basic strategy of separating customer communications from grid management communications using a VN approach. The grid operations VN would provide utilities with capacity for large increases in sensor data and cloud-based analytics. The customer data VN would give customers greater access to power markets and would give utilities and ISOs greater capacity to utilize meter data to improve grid efficiency and reliability.
Method-specific is either passive eavesdropping on SG communications to gather data or actively inserting fake messages or malformed packets into the SDN data stream. Target-specific threats are potential attacks on digital devices – intelligent electronic devices, which are controllers of network equipment and Phasor Measurement Units. Software-specific threats target the protocols that send, transmit and receive messages between the controller and the rest of the network. Identity-specific threats are either insiders or outsiders who seek to undermine network performance. SDN and other computer networks must address all these threats, but because SDN’s intelligence is more centralized than the Internet, the security risks are higher. 6. Proposed strategic design using SDN Currently, utilities have multiple communication systems that, for the most part, are operated independently from one another. For example, a utility may have one communication system for dispatching field personnel, a multitude of independent communication lines for grid protection, several networks for remotely monitoring and operating switches, and another communications system for meter readings. In general, existing utility communications can be divided into two groups: data communications used to manually or automatically manage the grid and communications between customer meters and the utility. Communications used to manage the grid include: Supervisory Control and Data Acquisition (SCADA) systems that monitor power flows and voltage and allow System Operators to remotely operate switches, protection systems that control breaker operations for faults and special protection systems that perform switching to protect against frequency or voltage collapse, and sensor data sent from devices in the field that is used for real-time stability control, protection against equipment failure, and outage restoration. Communications between customer meters and the utility include: power consumption for billing purposes, controls to remotely connect or disconnect meters (on later smart meter models), power factor and voltage readings, meter pinging to identify outage locations, and power generation (if separately metered). Due to security concerns and data criticality, most communications currently used to manage the grid are sent via dedicated circuits owned by telecommunications companies or via utility owned wireless and fiber networks. On the other hand, customer meter data are sent via cellular networks, radio networks, power lines, or some combination of communication systems and typically gathered at substations before being sent to the central utility data management systems by yet another communication method – usually dedicated lines. The data stream used to manage the grid is the most important to guard against possible cyberattacks. In fact, the typical method utilities use to secure critical cyber assets is simply to isolate them from the internet. In contrast, utilities view data provided from customer meters as significantly less critical. For utilities to embrace a new communications architecture, the less secure communications with customer meters must remain isolated from the data networks managing the grid operations. This security concern and the existing separation of communication infrastructure lends itself to a two VN solution: One VN can
References Aydeger, Abdullah, 2016. Software Defined Networking for Smart Grid Communications. Available at. Florida International University Thesis. https://digitalcommons.fiu. edu/etd/2580/. Blenk, Andreas, Basta, Arsany, Reisslein, Martin, Keller, Wolfgang, 2016. Survey on network virtualization hypervisors for software defined networking. Ieee Commun. Surv. Tutor. 18 (1) First Quarter 2016. Available at. http://mre.faculty.asu.edu/ SDNHVsurv.pdf. Fang, Xi, Misra, Satyajayant, Xue, Guoliang, Yang, Dejun, 2012. Samrt grid – the new and improved power grid. IEEE Communications Survey and Tutorials 14 (4) Fourth Quarter 2012. Available at. https://ieeexplore.ieee.org/document/6099519. Goransson, Paul, Black, Chuck, Culver, Timothy, 2017. Software Defined Networks: A Comprehensive Approach, 2nd ed. Elsevier, New York. Hu, Yi, 2009. Phasor Gateway Technical Specifications for North American Synchrophasor Initiative (NASPInet). project manager. U.S. Department of Energy Available at file:///C:/Users/vglass.BUSINESS/Documents/Documents/rbr/rbr/Grid %20Communication/naspinet_phasor_gateway__final_spec_20090529.pdf. Kreutz, Diego, Ramos, Fernando, Verissimo, Paulo, Rothenberg, Christian, Azodolmolky, Siamak, Uhlig, Steve, 2014. Software –Defined Networking: A Comprehensive Survey. Available at. IEEE. https://arxiv.org/pdf/1406.0440.pdf. Metro Ethernet Forum, SD-WAN from technology to service standard. Available at https://www.mef.net/mef-3-0-sd-wan. Metro Ethernet Forum. MEF Professional Certification. Available at https://www.mefprocert.com/. Myrda, Paul, Koellner, Kris, 2019. NASPInet –the Internet for Synchrophasors. Available at. IEEE Xplore Digital Library. https://ieeexplore.ieee.org/document/5428363. Rehmani, Mubashir, Davy, Alan, Jennings, Brendan, Assi, Chadi, 2018. Software Defined Networks Based Smart Grid Communication: a Comprehensive Survey. Available at. https://arxiv.org/abs/1801.04613. Ruparel, Jig, 2018. How SD-WAN As a Service Solves MPLS Limitations. Available at. sdx central. https://www.sdxcentral.com/articles/contributed/sd-wan-as-a-servicesolves-mpls-limitations/2018/10/. Robuck, Mike, 2019. AT&T On the Homestretch of Virtualization Goal. Available at. FierceTelecom. https://www.fiercetelecom.com/telecom/at-t-homestretchvirtualization-goal. Taft, J.D., 2018. Assessment of Existing Synchrophasor Networks. Available at. U. S.
5
The Electricity Journal 32 (2019) 106639
V. Glass, et al.
service reform, and new access services. He is the lead author of many business and academic studies. Dr. Glass earned his MBA in marketing and finance, and Ph.D. in economics from Columbia University.
Department of Energy; Prepared by Pacific Northwest National Laboratory. https:// gridarchitecture.pnnl.gov/media/white-papers/Synchrophasor_net_assessment_final. pdf. U. S. Department of Energy, 2015. The Emerging Interdependence of the Electric Power Grid & Information and Communication Technology. Available at. Prepared by Pacific Northwest National Laboratory. https://www.pnnl.gov/main/publications/ external/technical_reports/PNNL-24643.pdf. Wikipedia, Power Inverter. Available at https://en.wikipedia.org/wiki/Power_inverter. Wikipedia, Virtual Power Plant. Available at https://en.wikipedia.org/wiki/Virtual_ power_plant. Wu, Nina, 2018. Verizon and Hawaiian Electric Partner to Install Smart Sensors on the Grid. Star Advertiser, July 17, 2018. Available at. https://www.staradvertiser.com/ 2018/07/17/business/business-breaking/verizon-heco-partner-to-install-smartsensors-on-grid/.
Dr. Eric Woychik brings more than 40 years of regulatory policy expertise in the cleanenergy and utilities arena, and has worked on behalf of a broad range of organizations including the California PUC and the New York PSC. He has provided research and analysis in topical areas such as smart grid integration, rate design and revenue allocation, as well as in DER project development and financing. Eric holds a Doctorate (Energy Market Game Theory) from Case Western Reserve University; an M.A. (Economics) from New Mexico State University; and a B.S. (Environmental Policy Analysis and Planning) from the University of California, Davis. Ephram Glass is an independent utility consultant specializing in smart grid services and enterprise asset management. In his thirteen years in the industry, he has worked on load growth planning, renewable energy and demand response regulatory affairs, generator interconnections, compliance, cyber security, transmission and distribution automation, reliability, analytics, asset accounting, resource management, and project management. He earned a Bachelor of Science degree in Electrical Engineering from Princeton University.
Victor Glass is Director, CRRI Scholar, and Professor of Professional Practice - Finance and Economics, Rutgers Business School - Newark and New Brunswick, Rutgers University. Prior to joining Rutgers, Dr. Glass was Director of Demand Forecasting and Rate Development at the National Exchange Carrier Association. For almost thirty years, he was responsible for forecasting demand and setting switched and special rates for more than 1100 telephone companies. He was heavily involved in access restructure, universal
6