SONET utilization from FDDI sources

Computers Elect. EngngVol. 18, No. 5, pp. 373-388, 1992 Printed in Great Britain. All rights reserved

SONET

UTILIZATION

0045-7906/92 $5.00+ 0.00 Copyright © 1992 Pergamon Press Ltd

FROM

FDDI

SOURCES

PAUL D. STIGALLt a n d CATHERINE D. BIAGIOLI2 tDepartment of Electrical Engineering, University of Missouri-Rolla, Rolla, MO 65401 and EAT&T, 7600 Interstate 30, Little Rock, AR 72209, U.S.A. (Received 11 October 1990; accepted in final revisedform 26 August 1991) Al~traet--This paper explores the performance of the Synchronous Optical Network (SONET) protocol when it is receiving traffic from a Fiber Distributed Data Interface (FDDI) network. The method of exploration is by simulation using a simple model that was developed with the NETWORK I1.5 simulation program. Three functional areas are simulated, first a traffic generator entity for producing frames, second an extended bridge and third an entity for representing the operation of an interface to SONET communication lines. Using the simulation results, measures of the adequacy of the model and of the performance of the SONET interface are discussed.

1. I N T R O D U C T I O N Recently the Synchronous Optical Network protocol was standardized [1]. The Synchronous Optical Network (SONET) is a fiber optics based network for use by Telephone Operating Companies. It is capable of transporting a wide variety of data stream types regardless of the data formats or rates. As more internetworking is done, and services such as broadband Integrated Services Data Network (ISDN) are commercialized, more non-telephony signals will be carried by the long distance networks. This paper looks at one possibility for a future traffic source to the new long distance network protocol: traffic from a local area network based on the Fiber Distributed Data Interface (FDDI). Simulation is used to examine the requirements of bridging between the networks, the utilization of a SONET by traffic from an FDDI Local Area Network (LAN), and the possible buffering needed at such a SONET interface. The simple models were developed on the program NETWORK II.5, and its run results are presented and examined. This paper combines and extends the results of previous work found in the literature and will be helpful to anyone considering bridging between SONET and FDDI LAN or similar networks. As optical networks become more widely used as LANs this type of bridging will need to be addressed by an increasing number of network designers and users. Additional background information on SONET [2-10] and FDDI LAN [11-20] is readily available in the literature. 2. F D D I

TO S O N E T

CONVERSION,

THE EXTENDED

BRIDGE

Since there are significant differences between optical systems, frames must be converted from one format to another. However, a variety of architectures are available for such use. The approach used here is a variation of one proposed for a LAN to ISDN bridge [21]. Referred to as an extended bridge, it defines a bridge which provides MAC, PHY and PMD. The advantage of this approach is that it allows the entire bridging operation to be divided into functionally and physically standard and nonstandard sections. The extended bridge has two bridges and a cross connect function. In this instance the bridges are a LAN bridge and a SONET interface, and the cross-connectivity function whose nature will be determined by the choice of conversion method. The connections of the functions are shown in Fig. 1. The standard sections are the IEEE 802.1 bridge and SONET interface. All the non-standard functions are in the cross-connectivity which has been divided into pseudo LAN, ashadow stations and pseudo telephony functions. In general the bridging functions must be defined for traffic going in both directions. However, the interest here is in loading of the SONET bound traffic. Since the SONET lines are separate 373

374

PAUL D. STIGALL and CATHERINED. BIAGIOL!

Cross-Connectivity Sh&dow Stations

Function

IEEE Pseudo

FDDI LAN (

802

Lines

Bridge

IN

Pseudo

¢~

SONET

SONET (

)

LAN

Telephony

Interface Lines

Fig. 1. Extended bridge architecture.

for each direction, LAN bound traffic only affects the utilization of the LAN. The SONET bound functions will be defined across the extended bridge. The LAN bound traffic will be defined only where it appears from the bridge. 2.1. IEEE 802 M A C bridge The 802.1 bridge, 802.5 token bridge, and network backbone bridge under development, have been standardized by IEEE to assist the orderly interconnection of different 802 standard LANs [22]. They provide MAC and lower layer services only in order to maintain high throughputs. Hence, the functions performed are frame filtering, frame forwarding, address learning and frame checking [23]. 2.2. S O N E T interface 2.2.1. Choices. In order to define the SONET interface the format of the signal and frame expected must be defined. In choosing this format there are at least four ideals which we would like to meet: (1) (2) (3) (4)

It makes efficient use of available SONET bandwidth. It does not create a bottleneck to traffic flow. It is not overdesigned vs bandwidth needs. The hardware needed is readily available.

Some possible choices which meet the first ideal are: • • • •

FDDI frames to FDDI frames to FDDI frames to or FDDI frames

DS1 1.54 Mb/s frames to STS-1 frames, DS3 51.4 Mb/s frames to STS-1 frames, SPEs to STS-1 frames, to SPEs to STS-N, or STS-Nc frames.

The first two choices certainly will also meet the fourth ideal since SONET must be able to carry DSN signals, the frequencies are already well matched, and the first interfaces designed will be for DSNs. Even part of the cross-connectivity function may be available. The latter two choices have the advantages of not involving the intermediate conversion to DSN format and its overhead. Hence, it would be easier to meet the second ideal, and judicious choice of the STS level should allow meeting the third ideal. 2.2.2. Development. For a first case, a model of the FDDI frame to DS3 to STS-1 frame choice is developed and simulated. Even though DS1 lines/interfaces are presently more common, using DS3s would involve much less fragmenting and multiplexing for the same SONET data rate. 2.3. Cross-connectivity function Now that the exterior bridges/interfaces have been defined, the conversion tasks can be specified, and distributed between the cross-connectivity function components. Some of the tasks which must be accomplished are the reconciling of frame sizes, producing frame overhead and routing ~ o d s . 2.3.1. Pseudo LAN. The functions of the pseudo LAN are the identification of the proper shadow station to receive the frame from its destination and source addresses, and the fragmentation of the frame.

SONET utilization from FDDI sources

375

2.3.2 Shadow stations. The shadow stations initiate the connection management functions of each active telephone address. 2.3.3. Pseudo telephony. This component handles the connection signaling, and produces the SONET interface framing overhead [24]. 3. S I M U L A T I O N

OVERVIEW

Like most expensive computer systems, before installing a system or leasing its service it is desirable to have a good idea of its capacity, and behavior vs the needs of the users. Simulations or modeling techniques are often used for this. The advantages of discrete event simulations are that they are much more flexible about what kinds of systems can be modeled, and more readily yield transient behavior as well as averages. In selecting a simulation program different degrees of flexibility and depths of detail are available. The tools which can be used range from simulation programming languages to specialized packages for particular systems. These tools trade off the flexibility of the languages for the ease of use of the packages. NETWORK II.5 [25, 26] is a simulation package which is designed for a middle position between these extremes. N E T W O R K 11.5 provides a set of simulation entities, hardware, software and interconnections. It also provides utilities for the orderly creation, verification and display of these entities. The inherent difficulty with N E T W O R K II.5 is that it still may not have the particular characteristics needed for an application, or require considerable fussing with the available entities to achieve an application. However, since much of the system simulated here does not have strict, detailed specifications, and only a simple approximation of the system is desired, NETWORK II.5 should be quite adequate for this purpose. 3.1. The simulator program N E T W O R K II.5

The simulator program N E T W O R K II.5 was written to simulate systems of processors and interconnections. By keeping characteristics very general these processors and interconnections may express detailed, intra-computer operations, or express approximate inter-computer operations. The package is written into functionally separate programs which communicate via their input and output files. The three main programs are NETIN, NETWORK and NETPLOT. There are also some other utility programs available. 3.2. Model definitions

Each section of the networks and extended bridge have been modeled for simulation as separable groups of N E T W O R K 11.5 entities. The following describes roughly the entities used in each section, and explains the manner by which numerical parameters were specified for them. 3.2.1. FDDI L A N model. The connection from the LAN to the bridge is modeled as a processing element and a transfer device. As shown in Fig. 2 the processing element, FDDI LAN traffic source, provides the frame generation instructions, while the transfer device, FDDI LAN lines, provides the data rate and the physical connection to the bridge. (a) FDDI L A N traj~c source. Since it is undesirable to model the entire ring to produce the frame traffic seen by the bridge station, one processing element is defined to generate all the frame traffic. This processing element, the FDDI LAN traffic source, transmits frames, by performing message instructions, as from each theoretical station, in a probabilistic manner, taking into account the size parameters of the ring. The software module flow used to do this is shown in Fig. 3. The traffic patterns of the LAN are dependent on a number of variables, some of them quite random. These variables are: the arrival rate of frames at individual stations, the cycle time of the LAN, the length of frames being sent, the percentage of the frames destined for nonlocal stations, and the number of stations active on the LAN. Values for these variables have been calculated or measured in studies on existing LANs, H U B N E T and another fiber HSLAN [27], and also large Ethernets [28]. The values used for this simulation are taken or derived from these studies [29]. The length of frames distribution used here is derived from H U B N E T [17] and File Transfer [27] studies. The resulting distribution is shown in Fig. 4.

376

PAULD. STIGALLand CATHERINED. BIAGIOLI

PE: F D D I L A N Traffic S o u r c e Modules: S t a r t Traffic

Find N u m b e r o f F r i m e s t o S e n d Send and Sink Frame Send Frame G o t o N e x t Station for F r a m e s

Wait for Bridge Sink D a t a F r a m e

TD: FDDI LAN

Fig. 2. FDDI LAN simulation entities.

(b) FDDI L A N lines. Connecting the FDDI LAN traffic source to the bridge are the FDDI LAN optical lines. They are modeled by a transfer device entity. This transfer device was specified as serial, one bit per cycle, with a cycle time of 0.01 # s. The protocol used for the transfer device access is first come first served. Since the access to the LAN in this model is completely determined, no contention or scheduling problems will occur. There are also no transmission overhead times specified. The transfer device during simulation is busy only while messages are being transmitted by the connected processor. So the effect of propagation delay on traffic transmission was included in the processor instructions, and not in the transfer device specification. 3.2.2. IEEE 802 bridge model. The IEEE 802 bridge was modeled by a processing element, IEEE 802 MAC bridge, connected to the FDDI LAN, and Channel 1 transfer devices. The simulation entities of the bridge are shown in Fig. 5. The software modules are involved in three tasks. These tasks are: repeating on LAN frames, processing and forwarding off LAN frames, and transmitting [ S t a r t L A N Traffic I Station C o u n t , l Token Token

f

T

)

Module: Find N u m b e r of F r a m e s to S e n d

i

Token

Number i of Frames

Send all frames, Modules: S e n d a n d Sink F r a m e s , Send, Frame

Station sends token T Station Count

i Decrement ' station count,

Module: Go to

Wait for B ~ g e

i

to send token

I

Station Count T

S.,d 4 No

I Is t h e n e x t station a bridge?

Yu

J

Set station count, [

' Module: "Wait for I

Next Station Fig. 3. Flow chart of traffic generation modules on FDDI LAN traffic source.

SONET utilization from FDDI sources

377

% 50-

14_ 12_ I0_ 8_

~6-4_

I

2

I

0 0

606886

100

126

160

i...]

218

800

.oo

7s104

_ 106

by~e8

Fig. 4. Frame length probability density function.

returning frames from the SONET source to the LAN. All three tasks are initiated upon receipt of the proper message from the FDDI LAN traffic source. Figure 6 shows the module flow on this processor. 3.2.3. Cross-connectivity function model. As previously described, the cross-connectivity function contains three sections which are readily specified by three independent software modules on three processors connected by buses. The modules and processors have been labeled the pseudo LAN functions on pseudo LAN processor, the shadow station functions on shadow station processor, and the pseudo telephony functions on pseudo telephony processor. The connections of these entities are shown in Fig. 7. The processors are connected serially by 8 bit, parallel, first come first served, transfer devices. The separate transfer devices avoid contention occurring between processors sending on frames. P E : I E E E 802 M A C B r i d g e Modules: I E E E 802 B r i d g e P r o c e s s i n g T a s k s Repeat

LAN

Traffic

Bridge Token Response Find Number of R Frmnes to Send S e n d a n d Sink R Fraane Send R Frmne

TD:

Fig. 5. IEEE 802 bridge simulation entities.

Channel 1

378

PAULD. STIGALLand CATHERINED. BIAGIOLI Data Frame

No Bridge Actions Module: Repeat LAN Tragic

~

Off LAN Frame

~ Token Frame

Module: Find Number of Return Traffic Frames

Process Frame for Forwarding Module: IEEE 802 Bridge Processing Tasks

N u m b e r ~ of Frames

Send all frames, I Modules: Send and I Sink R Frames, Send R Frame

l Send Token To FDDI LAN Traffic Source Module: Bridge Token Response Fig. 6. Flow chart of modules on IEEE 802 bridge. PE: Pseudo LAN Processor Module: Pseudo LAN Functions

TD: Internal Cross-Connectivity Bus 1 PE: Shadow Station Processor Module: Shadow Station Functions

TD: Internal Cross- Connectivity Bus 2 PE: Pseudo Telephony Processor Module: Pseudo Telephony Functions

TD: C h n n e l

Fig. 7. Cross-connectivity function simulation entities.

SONET utilization from FDDI sources

379

PE: S O N E T Interface P r o c e s s o r Module: DS8 Processing

T D : Interface B u s PE: S T S F n u n e P r o c e s s o r Module: SOH Processing

T D : STS B u s PE: S O N E T E / O T r a n s m i t t e r Modules: Transmit SONET Frame S O N E T F r a m e Sink

TD: S O N E T

Lines

Fig. 8. SONET simulation entities. This arrangement does not prevent bottlenecks between sections because the processing delays frame forwarding. The model could easily be improved for better performance of the crossconnectivity function. Indeed, the forwarding of frames between processors would probably not be done in an implementation of the cross-connectivity function. However, in the simulation it allows an easy method of module scheduling. 3.2.4. SONET model. The SONET interface and SONET lines were defined by three processors! the SONET interface processor, the STS frame processor, and the E/O transmitter. These processors have the following tasks: the SONET interface receives from the channel device, and converts a DS3 frame to an SPE. The STS frame processor receives from the interface bus transfer device, and converts the SPE frame to an STS-1. The E/O transmitter represents the bit scrambling and electrical to optical conversion of the STS-1 frame received from the STS bus transfer device. Figure 8 shows the modules of these simulation entities. 4. S I M U L A T I O N

RUN

RESULTS

AND

ANALYSIS

4. I. Model summary and expectations A summary of some parts of the models are provided. Also, brief lists are given of the basic assumptions that were made with this model, and of the general kind of results which were expected. The model assumptions are divided into area of service approach, traffic and physical layout. 4.1.1. Approach assumptions. The approach chosen for modeling this communication service was that it be simple and optimistic. This led to the following assumptions: • There are no representations of error situations on the LAN. • The target token rotation time was sufficiently long to allow exhaustive service by all stations.

380

PAUL D. STIGALLand CATHERINED. 81AGIOLI

• There are no connection establishment delays made by the cross-connectivity function. • Also, the SONET interface was sufficiently synchronized that SPE frames did not span STS frames.

4.1.2. Traffic assumptions. The assumptions made for the nature of the traffic on the FDDI LAN were; • The assumed traffic from each station contained interactive, file transfer, and unrestrictive digital data. • All file arrivals were independent events. • These events were characterized as bursty, exponentially distributed arrivals with a combined 2 of 40.784. • Also, the files for transmission had a bimodal length pdf.

4.1.3. Physical assumptions. The assumptions made for the physical layout of the system were: • The ring contained N active stations of like kind. • The stations were equally spaced at 2 km apart. • The IEEE 802 bridge is the Nth station on the bridge. It is connected to the cross-connectivity function, and thus to the SONET interface.

4.1.4. Precision of results. The approach taken in the development of the model characterizes the nature of the expected results. Firstly, that optimistic assumptions would produce results which indicate upper bounds on performance. Secondly, the simplicity produces results which should be considered only approximate. More than a couple significant digits should not be used for examination except for comparison between marginally different simulation run results. 4.1.5. Utilization expectations. When exploring the results of the simulation, how much and how well SONET accepts FDDI traffic should be discovered. Basically, it is expected to find that SONET can accept less than 50% of the traffic of a fully utilized LAN. This is for the following reasons: • Chiefly, the basic bit rates are 100 Mbps to 51.84 Mbps. • Frame fragmenting, DS3 format conversion, and SONET format conversions add overhead to the basic FDDI frames. • And, since FDDI frames have variable lengths, the SPEs will also be filled out with idle symbols.

4.1.6. Efficiency expectations. As to how well SONET accepts the traffic, it is expected that this bridging scheme is not highly efficient, for these following reasons: • The extended bridge includes the multiple layers of overhead. • The shadow stations of the extended bridge generate the idle symbols with no provisions for minimizing them. • Also, the bridge does not moderate the burstiness of the traffic.

4.2. Model performance The particular values which are checked are the traffic generation parameters of window times and the average number of arrivals produced by the FDDI LAN traffic source. 4.2.1. Traffic generation. When the FDDI LAN was defined, an average value of the window time W was used. Since the FDDI LAN traffic source had no provisions for timing control, or adjustments for changes of LAN size, it was quite likely that different window times than the approximate 50 ms assumed would occur. Indeed, small LAN sizes would be likely to have window times which are too short, and large LAN sizes would be likely to have window times which are too long. In order to assess the validity of the simulation run results, the actual W that occurred needs to be checked against the maximum allowed token rotation time and the averase value used. Since the W is not directly available from the simulation output, it must be found from the performance of several of the entities. Part of a NETWORK output report from a simulation run is shown in Tables l(a) and l(b), All time ~alues in the ouptut reports were in/~s. An average window time is found from the average time e tpsed between receptions of tokens at the IEEE 802

SONET utilization from FDDI sources

381

Table I. (a) Network II.5 completed module statistics (~s) Module name Host PE Completed executions Cancellations due to Iteration period Run until semaphores Message requirements Successor activation Num precondition time Avg precondition time Max precondition time Min precondition time Std dev precond time Avg execution time Max execution time Min execution time Std dev execution time Restarted interrupts Avg time per interrupt Max time interrupted Std dev interrupt time

Find number of R frames to send IEEE 802 MAC bridge

Send and sink R frame IEEE 802 MAC bridge

IEEE 802 MAC bridge

54

54

54

O 0 0 0 54 55.110 279.340 1.850 90.128 0 0 0 0 0 0 0 0

Send R frame

O 0 0 0 54 36.299 200.010 0 61.717 1393.789 t3,667.450 0 3229.132 0 0 0 0

0 0 0 0 54 0 0 0 0 904.375 8000.000 0 2380.141 O 0 0 0

Table 1. (b) NETWORK II.5 completed module statistics (ks) Module name Host PE Completed executions Cancellations due to Iteration period Run until semaphores Message requirements Successor activation Num precondition time Avg precondition time Max precondition time Min precondition time Std dev precond time Avg execution time Max execution time Min execution time Std dev execution time Restarted interrupts Avg time per interrupt Max time interrupted Stddev interrupt time

Sink data frame Sonet frame sink FDDI LAN traffic Sonet E/O source transmitter 523

10,394

0 0 0 523 3808.100 71,939.420 0.500 11,796.902 0 0 0 O

0 0 0 0 10,394 191.688 21,636.699 124.999 827.003 0 0 0 0

0

0

O 0 0

0 0 0

0

bridge. There are three models which have results which can be used. These are the Send and Sink Frames, the Send R Frame, and Sink Data Frame. The Send and Sink R Frames and Send R Frame execution times give the length of time the bridge held the token. The Sink Data Frame precondition times can yield the length of time between the last frame of one bridge token holding and the first of the next. The Sink Data Frame module's purpose was to clear received data frames from the message queue of the FDDI LAN traffic source. Its operation takes no simulation time, and its precondition for activation is the presence of a data frame message in the queue. The number of precondition times is the number of times the precondition was met. The minimum precondition time is the time delay between frame reception while the bridge is transmitting. The maximum precondition time occurs during the rest of the bridge's window. The value needed for calculating an average window time is an average of the inter-token precondition times. To find this value the following formula was used. Avg Precondition Time = {Num Precond. Time
382

PAUL D . STIGALL a n d CATHERINE D . BIAGIOLI Table 2. Window time simulation results N 5 10 15 20 25 30 100 500

Average W (ms) 14 23 28 41 48 54 184 1427

Table 3. N E T W O R K II.5 instruction output report Instruction name

Completed instruction report from 0 to 2 s Count instruction name Count

FDDI LAN traffic source To LAN frame To bridge frame To LAN frame 4500 To LAN frame 2500 NOP Wait to sink frame 2 Wait to sink frame 4 Reset next station count Remove arrival from queue

1362 Send token 564 Send token to bridge 3592 To bridge frame 4500 79 To bridge frame 2500 409 Wait to sink frame I 97 Wait to sink frame 3 1095 Set next station count 1079 Set arrival count 2195

1037 43 1537 35 5129 67 1100 2196

IEEE 802 M A C bridge Forward frame To LAN frame To LAN frame 1000 Process off LA ~'' frame Wait to sink frame 1 Wait to sink frame 3 Set arrival count

2182 152 7 2182 244 4 165

43 244 6 5142 6 112 165

Send token To LAN frame 4500 To LAN frame 2500 Process on LAN frame Wait to sink frame 2 Wait to sink frame 4 Remove arrival from queue

The resulting Ws found vs the number of stations of a run are shown in Table 2. With these results it can be seen what number of stations support the window time assumptions. For the other numbers, methods of bringing the models and the assumptions into agreement can be suggested. It is found that the window times for a number of stations in the range 20-30 or so supported the assumed time reasonably well. So with this model and the k pdf that was used these sizes of LANs are most valid. As it happens, thi~ is a rcaso'lable size for a backbone network which was one of the applications for FDDI LANs. A number of stations smaller than 20 is the size of a back end network, another FDDI LAN application. For this size, it is seen that the token was traversing the LAN faster than assumed. The only methods of lengthening the token rotation time are by generating more frames per station or longer frames. Since the arrival rate of frames assumed was already rather high, and some of the file lengths were already quite long, it would be better to achieve agreement by changing the assumed window time than by forcing the model to perform to the present one. An iterative development method, using both the simulation runs and prob(k) calculations, could be used to find window times that provide the correct k pdfs for a number of stations in this range. For a number of stations over 35 it is seen that the token was traversing the LAN slower than assumed. Each station added to the LAN increases the propagation delay, hence increases the time needed to transmit a frame, and shortens the amount of time a station may transmit. Increasing the assumed token rotation time would be a simple way to achieve agreement. However, it would be more useful to change the model instead. In this case the assumption of exhaustive service needs to be abandoned. Instead, the token holding time of each station would be represented in the model, and queuing of frames for later transmission would he done. However, for this simulation these corrective steps have not be¢n done. The results from the 100 and 500 stations runs are not used. 4.2.2. Arrivals per second. The assumed value of the arrivals per second, 2, was 40.784. Since the number of files expected to arrive at a station for transmission was dependent on the window time in which the files may arrive, having the actual window time in the simulation different from the assumed one also caused a different average file arrivals per second. In particular having shorter window times than the assumed one would produce more files per second, and longer ones would produce fewer. The average number of film transmitted per second during the simulation could be found from the output report of the number of executions of the Set Arrival Count instruction on the FDDI LAN trafftc source proceuor. Table 3 shows a sample of an instruction report. As the semaphore Arrivals was set once per file arrival, the total number of times it was set divided by the duration of the simulation run yields the average arrivals per second. Table 4 shows the values found from the simulations. These values are not surprising considering the window times previously discussed. These values demonstrate how nonlinear the effect of an incorrect window time was. Notably the 5 and 10 station LANs, whose window times differed the most from 50 ms, have greatly inflated values for the average arrivals per second.

SONET utilization from FDDI sources Table 4. Arrivals per second simulation results

383

Table 5. Comparison of utilizations vs number of stations (%)

N

Average ~

N

N • ~

FDDI

5 10 15 20 25 30

177.2 105.1 72.5 56.4 44.3 38.7

15 20 35 30

1125 1080 1100 I 170

99.46 99.55 99.54 99.60

IEEE 802 bridge 31a 32 33 30

21 b 25 26 26

SONET 53 63 65 65

"The first column includes source and return traffic. bThe second column includes only source traffic.

The net effect here is that with simulation runs, which look at different LAN sizes, there are two variables actually being altered, N and ~.. This does not violate any protocols of the network, so the model is not necessarily invalid. However, it does need to be kept in mind when comparing the different simulation runs. Furthermore, since they vary so greatly the 5 and 10 station run results are also not used for comparisons. 4.3. Run results

Three common measures of performance are utilization, throughput, and queue length. For this simulation the FDDI LAN, IEEE 802 bridge, and the SONET will be examined for their utilization and bit throughputs. The queue lengths occurring at the SONET interface also will be examined. These measures give different means of evaluating how well the needs of the traffic were being met. 4.3.1. Utilization. Examining the utilization of a system reveals the amount of time elements of the system are idle. It is a simple way of measuring the excess capacity available. For instance, with the simulation utilization results could be used to estimate how large LANs may be possible in the different run situations. To do this the utilization of the FDDI LAN, the IEEE 802 bridge, and the SONET lines are found. From the electrical or optical signal point of view, token ring networks are nearly 100% utilized because there is always a frame of some kind being transmitted on the network. Hence, an idle token ring FDDI LAN is indicated by the presence of a free token frame. The three values in the output results, which are used to calculate the utilization of the FDDI LAN are: the percent utilization reported for the FDDI LAN transfer device, the number of token frames instruction, and the number of frame sink instructions executed. Knowing the percent utilization allows the amount of time the LAN was busy to be calculated. This time includes the time when token frames were present, so this amount of time needs to be removed. Also, the total time does not include the time for sinking a data frame when the propagation delay of a frame was longer than the transmission time. Hence, the total amount sink time needs to be added. The utilization of the IEEE 802 bridge with all traffic and processing was taken directly from the output report for the processing element. Utilization from just the LAN source traffic was calculated from the sum of the total times the IEEE 802 bridge processing tasks and the repeat LAN traffic modules were active. The sum was found from multiplying the average execution time by the number of completed executions. Lastly, the SONET utilization was taken directly from that given by the E/O transmitter processing element. Table 5 gives the results from the simulation runs where the number of stations were changed. Table 6 gives the results from the simulation runs where the percent of traffic destined off the LAN was changed. The former table includes the total average traffic, N . 2. Table 6. Comparison of utilizations vs off L A N percentage Off L A N % 5 10 15 27 35 40

IEEE 802 bridge 14a 18 24 32 43 45

5b 10 17 25 33 38

SONET 11 25 42 63 81 93

"The first column includes source and return traffic. bThe second column includes only source traffic. Utilization of F D D I equals 100% for all ca~s.

PAUL D. STIGALLand CATHERINED. BIAGIOLI

384

Looking at the results from the first set of runs, there are three points notable about the FDDI LAN utilization. Firstly, due to the manner in which the traffic was being generated, the total arrivals per second differed little from one size LAN to another. Hence the utilization remained nearly constant. Secondly, this load on the LANs was sufficiently high, and the LANs sufficiently small, that the token passing times had no significant effect on the utilization. The percent utilizations reported directly by the program were all lower than those calculated. Thirdly, it is found that with these arrival rates and file lengths the FDDI LAN was fully utilized, the percentage value being practically 100%. Looking at the results for the IEEE 802 bridge in the same set, it is seen that its total utilization also remained nearly the same. This was for the same reasons as for the LAN. The utilization by frames from the traffic source, however, decreases with the LAN size, and so was also a decreasing part of the total. The bridge gained the free token more often with fewer stations, hence it spent more total time on that function. This is one demonstration of the effect of the return traffic on the capacity of the bridge and network, because it is the return traffic which was making up the differences in the load on the LAN to produce the same utilizations. While this effect was seen in these runs because the probability of k arrivals was not changed with the LAN size, if the probability had been changed, it might occur with increasing size instead. The increasing amount of return traffic would offset the decreasing availability of a free token. Lastly, looking at the results for the SONET only one measure was really obtained for this set of runs. On the whole the traffic in these runs to the SONET interface varied little so it is simply found that for a fully loaded FDDI LAN and 27% of the traffic destined to the SONET interface, the SONET is about 63% utilized. SONET utilization in general was better demonstrated by the second set of stimulation runs. The simulation results done with changed percentage of off LAN traffic show that the SONET was fully utilized with less than 45%. Indeed, it is found that for each 5% of off LAN traffic 11,6% of the SONET capacity was used. Alternatively, the maximum number of stations SONET could support, which were dedicated to off LAN traffic, can be found. In the simulation the off LAN traffic was generated by twenty stations, each station generating 5% of the total LAN traffic. Thus the SONET can support fully only eight of these stations at one time. 4.3.2. Throughput. Throughput, whether in frames, bytes or bits, gives another measure of the capacity of a system. It also can yield a measure of its efficiency. By looking at the bit throughput at various points in the model, what the percentage of the final bit throughput which was due to the bridging method, and not part of the traffic being forwarded, could be found. The bit throughput of each processing element was calculated from its total number of bits transmitted, divided by the duration of the run. This information was obtained from the instruction output report for their number of executions, and the instruction specifications for their number of bits transmitted. All the message instructions on the FDDI LAN traffic source processor were used for the LAN throughput calculation. All but two of those instructions for both LAN and off LAN traffic produced frames of fixed length. The two other message instructions use the frame length pdf, and an expected number of bits sent was calculated for their contribution. Similarly, the IEEE 802 bridge throughput was found using only the off LAN message instructions. The total number of bits the SONET interface produced was very simply found from the number of executions of the Transmit OC-1 Signal times the bit size of the OC-1 frame. Table 7 shows the resulting bit throughputs vs the off LAN percentage. Table 7. Comparison of throughput vs off LAN percentages (Mbps) % 5 10 15 27 35 40

Bridge 4.0 10.0 17.1 24.9 33.0 38.0

Shdw. station

Pseudo tel.

SONET

5.0 10.8 18.4 26.9 36.1 41.3

5.7 12.4 21.0 30.8 41.3 47.2

6.0 12.9 21.7 32.0 42.2 48.0

Throughput of FDDI equals 9218 for all percentages of off LAN traffic,

S O N E T utilization from F D D I sources Table 8. Overhead throughput vs off LAN percentage %

Total

(Mbps) Ext. bridge

5 10 15 27 35 40

2 2.9 4.6 7.1 9.2 10.0

1.7 2.4 3.9 5.9 8.3 9.2

385

Table 9. Overhead throughput percentages vs off I.AN percentage

Idle bits

DS3 bits

%

Total OH %1

From ext. %1

Bridge %2

1.0 0.8 1.3 2.0 3.1 3.3

0.7 1.6 2.6 3.9 5.2 5.9

5 10 15 27 35 40

33 22 21 22 22 21

28.3 18.6 18.0 18.4 19.7 19.2

85 83 85 83 90 92

Idle bits %1 %3

DS3 bits %J %3

16.7 6.2 6.0 6.2 7.3 6.9

11.7 12.4 12.0 12.2 12.3 12.3

59 33 33 34 37 36

41 67 67 66 63 64

%t = Percentage of total traffic at SONET. %2 = Percentage of total OH. %3 = Percentage of ext. bridge OH.

The necessary overhead bytes in this system are from the FDDI protocol, the SPE and the STS. These are inserted before and after the bridging process. These would be a part of the data stream regardless of how the bridge had been constructed. The overhead which was induced by the bridge structure is from the DS3 frame conversion, the DS3 overhead bits and idle filler bits. By examining the intermediate throughput results the effect of these may be found. The idle bits were included by the shadow stations processor when retransmitting an FDDI frame fragment at DS3 frame size. The DS3 overhead was included by the pseudo telephony processor when transmitting the DS3 frame to the SONET interface processor. Table 8 shows the throughput values injected by these bridging operations, and Table 9 shows the percentage make up of the total traffic and extended bridge overhead. These throughput comparisons can be made because there exists excess capacity across the system. If some portion was at capacity, then its throughput would be at its limit and the effect of added overhead would not be fully seen. A couple of points- that verify that the system was not fully loaded are that the throughputs were increasing with each traffic increase, and that the SONET throughput remained less than its 51.8 Mbps maximum. As would be expected the amount of overhead transmitted increased with increasing data transmitted, but the relative percent make up of the traffic was constant in general. The examination of these results shows that this bridging scheme was about 80% efficient in forwarding the FDDI LAN traffic. The cross=connectivity function contributed about 18-20% of the total traffic. The cross=connectivity function's overhead did comprise most of the overhead of the system being roughly 85% of the overhead. The remaining overhead would be just from the SONET protocol. So comparatively the SONET transmission was fairly efficient. In regard to the cross-connectivity function overhead, it was found that it consisted of about 34% idle bits and 66% DS3 overhead bits. This ratio was pretty consistent. This is not surprising, since only short files and the last frame of a long file produce frames with idle portions, while all frames contain DS3 information. Thus the DS3 overhead is proportional to the number of frames, and the idle bits are proportional to the fraction of the frames which were short. Since it was the DS3 overhead which comprised most of the cross-connectivity overhead, and this overhead cannot be removed for this bridging scheme, the efficiency of the SONET extended bridge can be improved moderately. Reduction of the idle bits inserted, by perhaps concentrating short FDDI frames to like destinations, might improve it by a third to 86% efficient. However, the effort to implement an algorithm to accomplish this in a real system may not be desirable, since speed is one of the virtues of a bridge and this would reduce its speed. Fiber optic networks at present have not been so loaded as to cause concern about bit squeezing. Thus the 2 Mbps, or one more active station on the LAN, which the idle bits represent would not be worth the slowing of the bridge. 4.3.3. Queue lengths. One consequence of accommodating bursty traffic is that there may be temporary overloads of system capacity during bursts. In order not to lose frames, buffers of some kind must be provided. The simulation models had unlimited buffering so there was no loss of frames experienced. However real devices of course have limitations, so it would be of value to estimate the needs here. The queue lengths were found from the message instruction output reports. Table 10 shows a sample of such a report. The queue times give the length of time a message waited in the input queue of the processor. Dividing this time by the transmission time of a frame, 125/~s, yields the

386

PAUL D. STIGALL and CATHERINE D. BIAGIOLI

Table 10. NETWORK II.5 message output report Message statistics from 0 to 3 s Message name SONET STS-1 OC-1 signal FDD! fragment frame for forwarding Receiving PE SONET E/O SONET E/O shadow station transmitter transmitter processor Number queued Number used Avg queue time Max queue time Min queue time Std dev queue time

10,360 10,334 6866.074 30,127.000 0 6161.377

10,333 10,333 0 0 0 0

10,390 10,390 2.377 344.000 0 16.120

Table 11. Number of STS-I frames in queue vs IEEE bridge throughput Average queue Maximum queue Mbps length length 4 l0

27 42

64 241

17 25 33 38 52

49 79 130 328 1124

220 299 593 748 2047

number of frames in the queue. Table 11 gives the results of these calculations. For more generality they are presented vs the bit throughputs of the IEEE 802 bridge given in the last section. To give perspective on these numbers it should be realized that some file arrivals produce many SONET frames. While 88% of the files produce only 1 to 4 SONET frames, the two large file sizes produce 23 and 16 FDDI frames which become 161 and 112 SONET frames respectively. Consequently, long queue lengths could be produced by a small number of files. Since each frame contains 810 bytes these queues represent blocks of memory, 8 Kb to 900 Kb on the average, and 52 Kb to possibly 1.66 Kb depending on the amount of traffic. The last figure is however an upper bound for the useful amount of buffer memory for this model because for the last case the off LAN percentage was 45%; the capacity of the SONET had already been exceeded. Once the capacity has been exceeded the queue will grow indefinitely. Some form of flow control would be a better way to address the congestion, rather than adding more buffering. When implemented, a system would probably be designed to meet the needs of the maximum queues. However, it can be seen that there were considerable differences between the average and maximum queues. So even though adequate memory for the expected worst case situations must be provided, much less than half would typically be in use. 5. C O N C L U S I O N S Two contrasting network protocols were involved in this work: SONET and FDDI. Although both are optical fiber based networks, their differences demonstrate the effects that distances and applications can have on physical specifications of a network. The shorter spans of FDDI networks allow more physical reliability and less protocol overhead, less expensive transmission equipment, and a frame format targeted for a smaller set of applications. The long haul SONET requirements allow for less inherent physical reliability and thus have more overhead for management, more powerful transmission equipment and a frame format that is very flexible towards the application payload. A simple model was developed to simulated traffic from an FDDI LAN being bridged to a SONET interface. This model was created with little difficulty using the NETWORK II,5 simulation program. The traffic source was specified to represent a heavy load of some typical high speed LAN applications. The bridging was done via an extended bridge architecture partitioned into standard IEEE 802 bridge, SONET interface, and a cross-connectivity function. As was described, the standard bridge was for handling LAN to LAN bridging functions, the SONET interface was for handling DS3 to SONET functions, and the cross-connectivity function was for handling the conversion process for the changes in frame formats and connection establishment. It was found that with some care to the size of LAN specified the model performed adequately in regard to the basic assumptions made prior to its development. Thus the model would be a viable base for further studies on this bridge. However, should further work greatly expand the detail in the model, transporting the model to a more powerful computer or to a simulation l a n ~ may be necessary because of the time and memory constraints of NETWORK H,5 found during this work. From the simulation the results showed that an STS-1 SONET communications line could adequately carry a significant portion of the traffic on an FDDI network. With the bursty traffic load of an average 56.4 file arrivals per second at each of twenty stations, the SONET lines were

SONET utilization from FDDI sources

387

fully utilized with a b o u t 4 0 % o f the L A N traffic. This represents a t h r o u g h p u t o f F D D I d a t a o f 45 Mbps. The bridging scheme was f o u n d to be only moderately efficient. T h e total overhead a b o v e that o f the F D D I p r o t o c o l was a b o u t 20% o f the bit rate. M o s t o f this was f r o m the DS3 frame conversion a n d not S O N E T overhead, and only marginal i m p r o v e m e n t looks possible by reducing idle space in the DS3 frames. It was also f o u n d that the S O N E T interface did require a large buffer m e m o r y to a c c o m m o d a t e the long queues o f frames that occurred. The a m o u n t o f buffer was very dependent on the average a m o u n t o f traffic and its burstiness. It was seen that m u c h less than half the m a x i m u m buffer was typically used, and that when the S O N E T was being fully utilized the m a x i m u m useful buffer m e m o r y size was under 1.66 Mb. Thus, no impediments to the F D D I to S O N E T bridging was found. The conversion process was possible with a m i n i m u m o f specialized equipment, the inefficiencies are n o t at present very costly, and the S O N E T capacity is quite adequate for current applications. Consequently, S O N E T using the STS-1 f o r m a t and DS3 p a y l o a d appears to be a viable alternative to present long-haul networking options for F D D I L A N s . REFERENCES 1. Ralph Ballart and Yau-Chau ChinE, SONET: now its the standard optical network. IEEE Communs Mag. 29, 8-15 (1989). 2. William R. Bryne, Broadband ISDN technology and architecture. IEEE Network 3, 23-28 (1989). 3. Kazumitsu Maki et al., Implementation and application of equipment with network node interface. IEEE GLOBECOM 2, 30.1.1. (1988). 4. Koichi Asatani, Network node interface for new synchrono ,us digital networks--concepts and standardization. IEEE GLOBECOM 1, 4.5.1. (1988). 5. Anna Hac and Hasan B. Mutlu, Synchronous optical network and broadband ISDN protocols. IEEE Comput. Mag. 22, 26-34 (1989). 6. N. B. Sandesara, T. H. Jones and A. (3. Edwards, SONET intra-office interconnect signal. IEEEGLOBECOM2, 30.5.1. (1988). 7. Steve Bootman, The integration of SONET into switching equipment: the merger of transmission and switching. IEEE Int. Conf. Communications 1, 14.3.1. (1988). 8. Rodney J. Boehm, The SONET interface and network applications. IEEE GLOBECOM 2, 30.4.1. (1988). 9. T. P. J. Flanagan, B. E. Allen and G. C. Copley, Application benefits of SONET networking. IEEE GLOBECOM 2, 30.3.1. (1988). 10. Ning Zhang, Kuo-Hui Kuo-Hui and E. C. Posner, Reliable payload pointer protocol for synchronous optical network. IEEE Int. Conf. Communications 1, 14.2.1. (1988). 11. Gerd E. Keiser, Local Area Networks. Mc(3raw-Hill, New York. 12. William Stallings, Handbook of Computer-Communications Standards, Vol. 2. Macmillan, New York. 13. Floyd E. Ross, FDDI--a tutorial. IEEE Communs Mag. 24, 10-23 (1986). 14. Larry Green, Planning for FDDI. IEEE 12th Conf. Local Computer Networks, pp. 4--6 (1987). 15. Werner Bux, Token-ring local-area networks and their performance. Proc. IEEE 77, 238-256 (1989). 16. D. Karvelas and A. Leon-Garcia, Performance analysis of the medium access control protocol of the FDDI token ring network. IEEE GLOBECOM 2, 34.6.1. (1988). 17. E. Stewart Lee, Peter I. P. Boulton and Brian W. Thomson, HUBNET performance measurement. IEEE J. Selected Areas in Commun 6, 1025-1032 (1988). 18. Ilan Kolnik and Joseph Garodnick, First FDDI Local Area Network. IEEE 12th Conf. Local Computer Networks, pp. 7-11 (1987). 19. M. J. Johnson, Proof that timing requirements of the FDDI token ring protocol are satisfied. IEEE Trans. Communs 35, 620-625 (1987). 20. Doug Dykeman and Werner Bux, Analysis of tuning of the FDDI media access control protocol. IEEE J. Selected Areas in Communs 6, 997-1010 (1988). 21. Dieter J/ipel and Erich Port, LAN/ISDN interconnect via frame relay. IEEE GLOBECOM 3, 54.2.1. (1988). 22. (3erd E. Keiser, Local Area Networks, p. 259. Mc(3raw-Hill, New York (1989). 23. IEEE Communications Society. IEEE NETWORK, Vol. 2, No. 1 (1988). 24. Bernhard E. Keiser and Eugene Strange, Digital Telephony and Network Integration. Van Nostrand, Princetown, N.J. (1985). 25. CACI Products Company, N E T W O R K 11.5 User's Manual Version 4.0. CACI Products Co., La Julia, Calif. (1988). 26. Shun Cheung, Socrates Dimitriadis and Waiter J. Karplus, Introduction to Simulation Using N E T W O R K 11.5. CACI Products Co., La Jolla, Calif. (1988). 27. Peter Martini, Otto Spaniol and Thomas Weizel, File transfer in high-spqa~d token ring networks: performance evaluation by approximate analysis and simulation. IEEE J. Selected Area~ in Communs 6, 987-996 (1988). 28. John F. Shoch and Jon A. Hupp, Measured performance of an ethernet local network. Communs A C M 23, 711-721 (1980). 29. Mischa Schwartz, Telecommunication Networks: Protocols, Modeling and Analysis. Addison-Wesley, Reading, Mass. (1987).

388

PAUL D. STIGALLand CATHERINED. BIAGIOLI AUTHORS' BIOGRAPHIES

Paul D. Stigall---Dr Sfigall received the B.S. degree in electrical engineering from the University of Missouri-RoUa in t962 and the M.S. and Ph.D. in electrical engineering from the University of Wyoming in 1965 and 1968. He is Professor of Electrical Engineering at the University of Missouri-Rolla, which he joined in 1970. Before joining the univeristy, he worked for McDonnell Douglas, the Navy Electronics Laboratory, and the Collins Radio Company. His research interests include computer architecture and systems, modeling and simulation of computer systems and networks, computer applications, and fault-tolerant computing. He is the author of several technical publications and the principal investigator on several research projects.

Catlleriae l ~ t h e r i n e Biagioli is a test engineer with AT&T Computer Systems in Little Rock, Arkansas. Her research inte~sts include computer communications and distributed systems. Biagioli received a B.S. in electrical and computer engineering from the University of Wisconsin at Madison in 1985 and a M.S. in electrical engineering from the University of Missouri at Rolla in 1990. She is a member of the IEEE and the IEEE Computer Society.