- Email: [email protected]
Distribution Architecture
CHAPTE R 8
Distribution Architecture What You Will Learn • What constitutes distribution architecture • Some distribution architecture patterns
8.1 Overview The distribution architecture focuses on how objects are allocated to different processors and the means by which they collaborate. These means include not only the underlying communications protocols, but also the policies and patterns that govern their use. As with all such design matters, different design patterns offer different technical solutions with different pros and cons. Distribution patterns come in two basic flavors: asymmetric and symmetric. Asymmetric architectures are distinguished in that the allocation of objects to processing nodes is done at design-time. This enables a number of simplifying assumptions to be made for the collaboration, since client objects can reliably know where their servers are and how to talk with them. Symmetric architectures are characterized by runtime allocation of objects to processor nodes. This organization is more flexible than asymmetric but comes at a cost of increased complexity, required infrastructure support, and possibly runtime overhead. Another benefit of symmetric architectures can be improved reliability because objects can be relocated when a processor node fails. Asymmetric architectures can be supported by simple infrastructures such as networks (e.g., Ethernet), buses (e.g., CAN or 1553 bus), serial links (e.g., USB or RS-232), or shared memory (e.g., dual-ported RAM). In all these cases, it is most common to have a general-purpose communication protocol (i.e., one that is not application-specific) and use this protocol to exchange application data and request services. A communication protocol is generally a logically multitiered structure in which low-level bit or byte exchange facilitates the logical exchange of application messages at a higher level of abstraction in the protocol. For example, Ethernet can provide the underlying data link and transport facilities for TCP datagrams, but a system can implement the higher-level exchange mechanism of Remote Procedure Calls (RPCs) using these lower-level facilities. The exchange of messages between remote applications takes place at the application layer in the protocol Real-Time UML Workshop for Embedded Systems. http://dx.doi.org/10.1016/B978-0-12-407781-2.00008-8 Copyright © 2014 Elsevier Inc. All rights reserved.
219
220 Chapter 8 stack, but this is ultimately realized by lower-level mechanisms in the transport, data link, or physical protocol layers. A good system design isolates the communication details away from the application semantics as much as possible. This not only simplifies the application but also allows the application to more easily run over different communication protocols. Distribution patterns appear at or just above the application layer in the protocol stack. As with all patterns, distribution patterns optimize some aspects of the system design at the expense of others. By far, asymmetric architectures are more usual in embedded systems. They are simpler and tend to be more time-efficient than symmetric architectures, which fits in well with the highly constrained environments in which many embedded systems must reside. However, the simple approaches often don’t scale well to larger scale systems. The most common infrastructure for use in symmetric systems is to use an ORB (Object Request Broker) architecture pattern, and the most common ORB standard is CORBA (Common Object Request Broker Architecture).1 Problem 8.1 Roadrunner Distribution Architecture The Roadrunner Traffic Light Control System is a moderately distributed system. Sensors and actuators are scattered about the intersection and may have small processors that allow them to communicate with the intersection controller. Certainly a connection must exist between the front panel display and the intersection controller as well. The system must also connect to the remote service and management system via a wired Ethernet connection and also must support directional IR communication from high-priority and emergency vehicles. For this problem, create class models (and diagrams) for the communication of the intersection controller subsystem to things with which it will communicate. Further, model the distribution architecture for the above-surface IR vehicle detector. Include necessary communications for the software elements, such as protocol stacks, message types, and distribution pattern–specific classes and add them to the collaboration. For this problem, make the following assumptions: • The distribution architecture is asymmetric. • Each light assembly and sensor assembly will be connected to the intersection controller and each other via a serial multidrop RS-422 link. Simple messages shall be exchanged using synchronous RPCs (remote procedure calls) over the serial link. • The multidrop RS-422 link is managed as a token ring, each node getting the master token for a brief period (not to exceed 20ms) to send messages. This ensures that the entire ring is cycled in no less than 1/5 second. • The connection to the front panel display and remote service and management system shall be a wired Ethernet link. This is to allow reviewing and management of the configuration status and modes of the system. See Pattern-Oriented Software Architecture, Volume 2, Patterns for Concurrent and Networked Objects by Doug Schmidt, Michael Stal, Hans Rohnert, and Frank Buschmann (John Wiley and Sons, 2000) for a detailed description of a number of different CORBA-related patterns. 1
Distribution Architecture 221 • The above-surface vehicle detectors support both wired (multidrop RS-422 above) and wireless (802.11b) communications. The above-surface vehicle detectors additionally house the transceivers used to detect high-priority and emergency vehicles; these additional sensors will use the vehicle detector assembly communication apparatus to communicate with the intersection controller. • The transceiver shall support a simple, custom IR protocol that is focused and directional, allowing the reception (only) of messages to indicate the arrival and presence of high-priority and emergency vehicles. • There shall be four IR transceivers (one for each through direction) that transmit the emergency and priority vehicle messages to the intersection controller.
Problem 8.2 Coyote UAV Distribution Architecture For the Coyote UAV, we’ll focus on adding the distribution architecture to the system as a whole and the aircraft. At the system level, the ground system and the aircraft communicate via two telemetry links to the ground using different frequencies. The low-speed data link is encrypted and secure with a bandwidth of 100 bits per second (bps). The high-speed data link is unencrypted and is used for high-bandwidth video and image streaming. It must support 640x480 resolution images at 30 fps. If uncompressed, that means that it must support a bandwidth of 640x480x10x30 = 92,160,000 bps. However, lossless compression of at least 30% on average is possible, leading to a required minimum bandwidth of 64,512,000 bps. Since we need some margin for error, 100 Mbps should be adequate to meet our needs. That means that the internal network must be able to handle this bandwidth in addition to other data necessary to fly and maneuver the aircraft; a minimum of 200 Mbps of real bandwidth should be more than adequate. If we use a CDMA (Collision Detect Multiple Access) protocol, such as Ethernet, then such a network saturates at about 30% of available bandwidth. Thus we need a 1 Gbps Ethernet internal bus to ensure we have no significant delay in the delivery of the data due to collisions on the network if everything operates on the same network.2 How best to model this? As always, there are multiple approaches to solving the problem. The simplest, with Rhapsody, is to simply stereotype the interfaces that specify the contracts on the ports «CORBAInterface». Rhapsody then “CORBA-enables” these ports so that it generates the interface definition language (IDL) for the proxies that marshal and unmarshal the services invoked across those ports. But what if we don’t want to use CORBA? In some sense the most ideal solution would be to create a class called TCPPortClass that has the structure, relations (to the protocol stack, for example), and behavior to marshal and unmarshal service requests and responses and then send them via datagrams, with sessions and sockets, as appropriate. Then use instances of this class to replace the existing ports on the subsystem and other structured classes. This, however, is not easily supported in modeling tools since in most tools ports are not easily extensible, requiring you to write significant amounts of code to implement the solution. Another solution is to refactor the architecture, deleting the old “semantic” ports that are specified in terms of application data and services, and replacing them with TCP/IP ports that are specified in terms of their ability to receive (or require) requests for sessions and sockets, and
222 Chapter 8
Figure 8.1 Port proxy pattern problem. that can send and receive datagrams. This approach is less satisfactory from a model standpoint because it disrupts the continuity from the original architecture model – which is based on data and services in the system – and modifies it to be based on the design infrastructure rather than the application semantics. In order to make the best choice, we need to ask ourselves ”What do we want from our solution?” An ideal solution should • Provide the connections to the other systems over the desired network • Require minimal modification (in the best case, no modification) to the existing collaborations and to the existing architecture • Be easy to reapply to different distribution architectures, networks, and data links One more approach, and the one recommended here, is to create proxies using a variation of the proxy pattern called the port proxy pattern. A proxy is a “stand-in” for another element and in this case understands how to marshal and unmarshal the service requests and responses into TCP/IP datagrams, set up connections and sessions, and so on. The port proxy pattern is specifically designed to address the issue of connections across ports that are to be implemented using a networking infrastructure. The port proxy pattern problem is shown in Figure 8.1. The solution – adding in proxies to manage the communication over the networking infrastructure – is shown in Figure 8.2. It is a “simple” matter of creating proxies that manage the networking communications. These proxies support the required application semantic interfaces and then create the necessary network data structures (such as datagrams) and execute the necessary network behaviors (such as creating sessions and sending and receiving network packets) to achieve the communication goals. The primary advantage of this pattern is that the original objects (in this case, the subsystems) don’t need to change at all to support the network infrastructure. The change is isolated to the links among the objects – now they are connected via proxies instead of directly. The original architecture still executes
Distribution Architecture 223
Figure 8.2 Port proxy pattern structure. because the interfaces haven’t changed and the new architecture with the proxies added works as well. Note that the proxies are connected via a pair of ports, each of which is a conjugate3 of the other. This is required because you cannot both offer and require the same set of services with the same port. Using paired conjugates makes the connections symmetric, simplifying the design. Your task for the CUAV system is two-fold. First, draw the distribution architecture that connects the ground system and aircraft. Second, apply this port proxy pattern to the internal aircraft architecture for subsystem-subsystem links. Draw the related subsystems and connect them via proxies. If the 1 Gbps network is unpalatable, then you might decide to either run multiple networks (one for command and control and another for high-speed data), or reduce the fidelity of the video. 3 In this context, by conjugate, I mean that what one port offers, the other requires, and vice versa. 2
8.2 Summary This brief chapter focused on identifying and modeling the distribution architecture. For the simpler exercise, we defined the distribution architecture for the intersection control system. We defined the three networks (RS-422 multidrop token ring), wired (Ethernet) and wireless (802.11b) networks. Additionally, we detailed the serial connection to the various distributed sensors and lights.
224 Chapter 8 The CUAV is a much larger problem. This system supports both high-speed and low-speed connections between the ground system and the aircraft. We defined that structure, although not the details. Within the aircraft, we applied the port proxy design pattern to connect the subsystems together in such as way that we needn’t modify any software in the subsystems to support the communication, and so that it is easy to change and adapt. The next chapter focuses on the concurrency and resource architecture. As with all architectural viewpoints, the solutions in this space are governed by design patterns. After that, we’ll drill down in the optimization at a more detailed level to finish out the book.
Distribution Architecture What You Will Learn • What constitutes distribution architecture • Some distribution architecture patterns
8.1 Overview The distribution architecture focuses on how objects are allocated to different processors and the means by which they collaborate. These means include not only the underlying communications protocols, but also the policies and patterns that govern their use. As with all such design matters, different design patterns offer different technical solutions with different pros and cons. Distribution patterns come in two basic flavors: asymmetric and symmetric. Asymmetric architectures are distinguished in that the allocation of objects to processing nodes is done at design-time. This enables a number of simplifying assumptions to be made for the collaboration, since client objects can reliably know where their servers are and how to talk with them. Symmetric architectures are characterized by runtime allocation of objects to processor nodes. This organization is more flexible than asymmetric but comes at a cost of increased complexity, required infrastructure support, and possibly runtime overhead. Another benefit of symmetric architectures can be improved reliability because objects can be relocated when a processor node fails. Asymmetric architectures can be supported by simple infrastructures such as networks (e.g., Ethernet), buses (e.g., CAN or 1553 bus), serial links (e.g., USB or RS-232), or shared memory (e.g., dual-ported RAM). In all these cases, it is most common to have a general-purpose communication protocol (i.e., one that is not application-specific) and use this protocol to exchange application data and request services. A communication protocol is generally a logically multitiered structure in which low-level bit or byte exchange facilitates the logical exchange of application messages at a higher level of abstraction in the protocol. For example, Ethernet can provide the underlying data link and transport facilities for TCP datagrams, but a system can implement the higher-level exchange mechanism of Remote Procedure Calls (RPCs) using these lower-level facilities. The exchange of messages between remote applications takes place at the application layer in the protocol Real-Time UML Workshop for Embedded Systems. http://dx.doi.org/10.1016/B978-0-12-407781-2.00008-8 Copyright © 2014 Elsevier Inc. All rights reserved.
219
220 Chapter 8 stack, but this is ultimately realized by lower-level mechanisms in the transport, data link, or physical protocol layers. A good system design isolates the communication details away from the application semantics as much as possible. This not only simplifies the application but also allows the application to more easily run over different communication protocols. Distribution patterns appear at or just above the application layer in the protocol stack. As with all patterns, distribution patterns optimize some aspects of the system design at the expense of others. By far, asymmetric architectures are more usual in embedded systems. They are simpler and tend to be more time-efficient than symmetric architectures, which fits in well with the highly constrained environments in which many embedded systems must reside. However, the simple approaches often don’t scale well to larger scale systems. The most common infrastructure for use in symmetric systems is to use an ORB (Object Request Broker) architecture pattern, and the most common ORB standard is CORBA (Common Object Request Broker Architecture).1 Problem 8.1 Roadrunner Distribution Architecture The Roadrunner Traffic Light Control System is a moderately distributed system. Sensors and actuators are scattered about the intersection and may have small processors that allow them to communicate with the intersection controller. Certainly a connection must exist between the front panel display and the intersection controller as well. The system must also connect to the remote service and management system via a wired Ethernet connection and also must support directional IR communication from high-priority and emergency vehicles. For this problem, create class models (and diagrams) for the communication of the intersection controller subsystem to things with which it will communicate. Further, model the distribution architecture for the above-surface IR vehicle detector. Include necessary communications for the software elements, such as protocol stacks, message types, and distribution pattern–specific classes and add them to the collaboration. For this problem, make the following assumptions: • The distribution architecture is asymmetric. • Each light assembly and sensor assembly will be connected to the intersection controller and each other via a serial multidrop RS-422 link. Simple messages shall be exchanged using synchronous RPCs (remote procedure calls) over the serial link. • The multidrop RS-422 link is managed as a token ring, each node getting the master token for a brief period (not to exceed 20ms) to send messages. This ensures that the entire ring is cycled in no less than 1/5 second. • The connection to the front panel display and remote service and management system shall be a wired Ethernet link. This is to allow reviewing and management of the configuration status and modes of the system. See Pattern-Oriented Software Architecture, Volume 2, Patterns for Concurrent and Networked Objects by Doug Schmidt, Michael Stal, Hans Rohnert, and Frank Buschmann (John Wiley and Sons, 2000) for a detailed description of a number of different CORBA-related patterns. 1
Distribution Architecture 221 • The above-surface vehicle detectors support both wired (multidrop RS-422 above) and wireless (802.11b) communications. The above-surface vehicle detectors additionally house the transceivers used to detect high-priority and emergency vehicles; these additional sensors will use the vehicle detector assembly communication apparatus to communicate with the intersection controller. • The transceiver shall support a simple, custom IR protocol that is focused and directional, allowing the reception (only) of messages to indicate the arrival and presence of high-priority and emergency vehicles. • There shall be four IR transceivers (one for each through direction) that transmit the emergency and priority vehicle messages to the intersection controller.
Problem 8.2 Coyote UAV Distribution Architecture For the Coyote UAV, we’ll focus on adding the distribution architecture to the system as a whole and the aircraft. At the system level, the ground system and the aircraft communicate via two telemetry links to the ground using different frequencies. The low-speed data link is encrypted and secure with a bandwidth of 100 bits per second (bps). The high-speed data link is unencrypted and is used for high-bandwidth video and image streaming. It must support 640x480 resolution images at 30 fps. If uncompressed, that means that it must support a bandwidth of 640x480x10x30 = 92,160,000 bps. However, lossless compression of at least 30% on average is possible, leading to a required minimum bandwidth of 64,512,000 bps. Since we need some margin for error, 100 Mbps should be adequate to meet our needs. That means that the internal network must be able to handle this bandwidth in addition to other data necessary to fly and maneuver the aircraft; a minimum of 200 Mbps of real bandwidth should be more than adequate. If we use a CDMA (Collision Detect Multiple Access) protocol, such as Ethernet, then such a network saturates at about 30% of available bandwidth. Thus we need a 1 Gbps Ethernet internal bus to ensure we have no significant delay in the delivery of the data due to collisions on the network if everything operates on the same network.2 How best to model this? As always, there are multiple approaches to solving the problem. The simplest, with Rhapsody, is to simply stereotype the interfaces that specify the contracts on the ports «CORBAInterface». Rhapsody then “CORBA-enables” these ports so that it generates the interface definition language (IDL) for the proxies that marshal and unmarshal the services invoked across those ports. But what if we don’t want to use CORBA? In some sense the most ideal solution would be to create a class called TCPPortClass that has the structure, relations (to the protocol stack, for example), and behavior to marshal and unmarshal service requests and responses and then send them via datagrams, with sessions and sockets, as appropriate. Then use instances of this class to replace the existing ports on the subsystem and other structured classes. This, however, is not easily supported in modeling tools since in most tools ports are not easily extensible, requiring you to write significant amounts of code to implement the solution. Another solution is to refactor the architecture, deleting the old “semantic” ports that are specified in terms of application data and services, and replacing them with TCP/IP ports that are specified in terms of their ability to receive (or require) requests for sessions and sockets, and
222 Chapter 8
Figure 8.1 Port proxy pattern problem. that can send and receive datagrams. This approach is less satisfactory from a model standpoint because it disrupts the continuity from the original architecture model – which is based on data and services in the system – and modifies it to be based on the design infrastructure rather than the application semantics. In order to make the best choice, we need to ask ourselves ”What do we want from our solution?” An ideal solution should • Provide the connections to the other systems over the desired network • Require minimal modification (in the best case, no modification) to the existing collaborations and to the existing architecture • Be easy to reapply to different distribution architectures, networks, and data links One more approach, and the one recommended here, is to create proxies using a variation of the proxy pattern called the port proxy pattern. A proxy is a “stand-in” for another element and in this case understands how to marshal and unmarshal the service requests and responses into TCP/IP datagrams, set up connections and sessions, and so on. The port proxy pattern is specifically designed to address the issue of connections across ports that are to be implemented using a networking infrastructure. The port proxy pattern problem is shown in Figure 8.1. The solution – adding in proxies to manage the communication over the networking infrastructure – is shown in Figure 8.2. It is a “simple” matter of creating proxies that manage the networking communications. These proxies support the required application semantic interfaces and then create the necessary network data structures (such as datagrams) and execute the necessary network behaviors (such as creating sessions and sending and receiving network packets) to achieve the communication goals. The primary advantage of this pattern is that the original objects (in this case, the subsystems) don’t need to change at all to support the network infrastructure. The change is isolated to the links among the objects – now they are connected via proxies instead of directly. The original architecture still executes
Distribution Architecture 223
Figure 8.2 Port proxy pattern structure. because the interfaces haven’t changed and the new architecture with the proxies added works as well. Note that the proxies are connected via a pair of ports, each of which is a conjugate3 of the other. This is required because you cannot both offer and require the same set of services with the same port. Using paired conjugates makes the connections symmetric, simplifying the design. Your task for the CUAV system is two-fold. First, draw the distribution architecture that connects the ground system and aircraft. Second, apply this port proxy pattern to the internal aircraft architecture for subsystem-subsystem links. Draw the related subsystems and connect them via proxies. If the 1 Gbps network is unpalatable, then you might decide to either run multiple networks (one for command and control and another for high-speed data), or reduce the fidelity of the video. 3 In this context, by conjugate, I mean that what one port offers, the other requires, and vice versa. 2
8.2 Summary This brief chapter focused on identifying and modeling the distribution architecture. For the simpler exercise, we defined the distribution architecture for the intersection control system. We defined the three networks (RS-422 multidrop token ring), wired (Ethernet) and wireless (802.11b) networks. Additionally, we detailed the serial connection to the various distributed sensors and lights.
224 Chapter 8 The CUAV is a much larger problem. This system supports both high-speed and low-speed connections between the ground system and the aircraft. We defined that structure, although not the details. Within the aircraft, we applied the port proxy design pattern to connect the subsystems together in such as way that we needn’t modify any software in the subsystems to support the communication, and so that it is easy to change and adapt. The next chapter focuses on the concurrency and resource architecture. As with all architectural viewpoints, the solutions in this space are governed by design patterns. After that, we’ll drill down in the optimization at a more detailed level to finish out the book.