Real-Time UNIX: What Performance can we Expect? Prof. Dr. Helmut Rzehak Universitjl der Bundeswehr MuncbeD Wemer·Heisenberg-Weg 39,0-8014 Neubiberg Tel.-Nr. X-49-89-6004-3397 I Fax-Nr. X-49-89.(,()()4.3S60 E·mail:
[email protected]
Abstract. The paper deals with necessary enhancements of traditional UNIX systems to meet requirements for real-time applications and gives an insight on sources of delays for application processes caused by the operating system. A survey of the real-time extensions for POSIX, the ISO/IEC standard operating system, is given. Some key parameters and basic CODcepts for implementing good real-time performance are discussed. Figures for commonly used metrics are reported. Kevwords. Real-time operating systems; POSIX real-time extensions; real-time performance metrics; reentraDt system calls; concurrency control
In order to tayloring operating systems for applications not
I, INTRODUCTION
requiring the full functionality real-time profiles and their minimum hardware requirements are defined. The actual state of work on POSIX extensions for supporting real-time applications distinguishes between four profiles [PX.13]:
Recently UNIX-systems have become available which claim to have real-time properties, and real-time extensions for POSIX, the ISO/IEC standard operating system based on UNIX [PX.11 are going to be defined. This gives occasion not only to look at the functional requirements in terms of required system calls, but also to ask for real-time performance of such UNIX derivatives. Slogans like 'Fully Preemptable Kernel" are used to underline real-time capabilities and even more serious performance-metrics may be confusing in the case of poor knowledge on basic system concepts. The aim of the following sections is to give a better understanding whereof real-time performance depends OD.
Minimal real-time system: This profile is typically used in embedded systems and somewhere else called an operating system kernel. The programming model is that of a single address space. This corresponds to a single POSIX process. Concurrent activities may be established through more than one PO SIX threads [pX.4aJ. Minimum hardware is • single processor with its memory, but no memory management unit or common 110 devices.
2. FUNCfIONAL REQUIREMENTS
- Real-time controller system: The minimal real-time system is enhanced by • file system interface, character serial I/O and exception handling (POSIX Signals). One or more serial channels (RS-232like) have to be added to minimal hardware but no mass storage device, since the file system may be implemented in memory.
Real-time operating systems are state of the art and have been used since a couple of years for many applications. Specialists highly agree upon functional requirements provided by system calls and differences in functionality mainly result from the scope of applications (e.g. whether an application requires I file system). Using the draft on realtime extensions for POSIX [PX.4) as an example the traditional functionality of UNIX (mUltiprocessing. interprocess communication, file handling. standard input and OUlput) will be enhanced by the following features for realtime applications (listed in the order of the draft):
DediC
Realtime Signals Extensions Synchronized Input and Output - Asynchronous Input and Output Semaphores Process Memory Locking - Memory Mapped Files and Shared Memory Priority Scheduling Oocks and Timers Message Passing - Realtime files
Multi·purpose real-time system This Profile supports full functionality of PO SIX real-time extensions. The hardware model assumes processors with memory management unit, high speed mass storage devices, network support and display devices.
247
Execute further tasks
Task T 1
OS-functions must not be interrupted in case
Entry
of non reentrancy
IEx~ Event
Return
Fig. 1 Calling an event reaction
3. REAL-TIME PERFORMANCE
3.1 Event Reaction Time
10 addition to the defined functionality the draft on real-time extensions for POSIX defines performance metria and it is mandatory to suppon appropriate measurement results for an implementation claiming conformance to the proposed standard. This is necessary to achieve a required level of service in a bounded response time. Furthermore for the user this is a means of comparing different implementations.
An external event is commonly signaled hy an interrupt and we assume that the associated event reaction is pan of a user task. Supposed that the interrupt has high priority to he immediately served and the currently running task has lower priority as the user task containing the event reaction. we define the time berween the occurrence of the external event and the execution of the first statement of the associated event reaction within the user task as the Process Dispatch Latency Time (PDL1). If this fint statement sets • Bit in an externally observable register, the PDLT is easy to measure. For real-time applications we ask for the upper bound of the PDLT.
10 general the metria should serve for calculating the overhead introduced by a service or upper bounds for delays of running programs caused by the operating system.
To give an answer we assume that just before the interrupt occurs the running task has executed I system call. If the called system function is not preemtable, the transfer of control to the task containing the event reaction is delayed until control is returned from the system function to user code. Fig. 1 gives. sketch of this situation.
Some methodologies and related metria for measuring realtime performance have been proposed in the literature [FURaJ. This paper is focused on the event reaction time and related topics, because the timely reaction to (external) events is the main requirement for real-time applications..
In Fig. 2 the transitions of control are marked on tbe time scale. The PDLT is the time between point (1) and (2). Worst case is the dynamically longest system function. A traditional UNIX implementation does not fit for real-time applications because the kernel functions are not preemptable. This leads to an extraordinarily high PDLT. Now real-time UNIX systems claim to have fully preemptable kernel functions and therefore the PDLT is very small. This is only half of the truth. For a better understanding we have to know why the preemption of kernel functions may be a problem.
An often referenced metric is the Process Dispatch Latency Time of the operating system - sometimes misleading shortly called Interrupt Latency Time. But the crucial issue on realtime operating systems is to avoid ill unexpected delays of the running processes. As we will see the Process Dispatch Latency Time is not the only delay to be considered for timely reactions.
248
External Event
1 +
I -
~;;~y::
Process Disp.1lch-----.J uteocy Time
"I
____. - ____
User Code
~ ~
1--+-_ _ _ _0-l2f--_ _ _ _ _ _ _
System Code System Code for T 1 L......".--I for T1
Execute Task T i (event re iated)
-1-
____
_
time
-1~
Continue Task T 1
f Interrupt Service Routine
Fig. 2 Definition of the Process Dispatch Latency Time
Fig. 3 gives an example with three critical regions. It is shown how I preemption with I reentrant function call works.
3.2 Reentrant System Calls Generally kernel functions manipulate entries in data structures of the operating system kernel q,ecessary to manage the various objects and resources of the operating system. Qanges necessary 10 performing I function cannot be done in a single indivisible instruction and therefore during execution the contents may be temporarily inconsistenL If an interruption occu~ in such I situation some or even all kernel functions are not executable because they operate on inconsistent data. A very coarse wlY 10 overcome the problem is to forbid kernel preemptions. A traditional UNIX kernel use this method. It is fairly simple and not so bad except for real-time applications.
It should be noticed, that the construction of such a kernel is not an easy task.. As we will see below, for good real-time performance the critical regions should be (dynamically) IS short as possible. A traditional UNIX kernel =01 be adapted by minor changes.. An other problem is the use of drive~ written for a traditional UNIX system, because their kernel level synchronization does not fit inlO the concept described here. Some compatibility modes may be offered for this purpose on the cost of real-time performance [FURaJ. In the case of I fully preemptable kernel the PDLT does not depend on whether user or system code will be executed if the external event occurs. Therefore measured values Ire reasonable small. But as shown in Fig. 3 an unexpected delay may happen later on, if a system call during the event reaction invokes the interrupted kernel function. This is the time between the points t3 and t6 in Fig. 3. This is necessary to terminate the critical region. Wo~t case is obviously the time necessary to execute the dynamically longest critical region. This leads to the
Allowing preemptions to a ceroin extend it my happen that a kernel function may be called twice until returning from the first call. Therefore we need reentrant system calls. Summarizing. for k.ernel preemptions we have to maintain the following conditions: A sequence of instructions changing data from one consistent state to another consistent one, commonly called a critical region. must not be entered twice until the first entry call has reached the exiL
umclusion: For I fully preemptable kernel of an operating system an arbitrary delay may be imposed into an event reaction, which does not exceed the execution time of the (dynamically) longest critical region of the kernel. We call this the Maximum Kernel Latency Time (MKL1). It is not part of the Process Dispatch Latency Time.
If system data used by each of two critical regions have I common subset these two critical regions must be entered mutual exclusive. Several concepts may be used to meet these conditions. With respect to real·time requirements the most elaborate one is to constructing the kernel functions with critical regions and using semaphores for preventing reentrant execution of critical regions or concurrent execution if these are conflicting. Such a kernel is sometimes called fully preemptable.
Without knowledge of the internal structure of the kernel the MKLT is not easy to determine more to measure. Sometimes reported statistical results may give a fi~t glance but wo~t case values may be much higher than mean values.
249
Call with prio = 32 semal
semal PI
sema2
se mal
Event Causing Function Cau with prio = 27
P2
sema3
j. ____ _
sema3
P3
PI; P2; P3: Mutual Exclusive SectiollS
t
Cau with prio = 27
o
t
1
t
2
t
3
t
4
semI
semal
semaJ
Fig. 3 Preemptive execution of a rcentranl syslem function
3.3 Published Results P=piODOFF
Table 1 shows some measured values of the PDLT obtained from [ruRal. All systems claim to be real-time UNIX systems. Some seem to have fairly good real-time performance.
90 .. ol plIhs thrwgb !be laneI
..ams
99 .. ol pubs thrwgb !be laneI
129ms
3.~ l1li
1127 ms
1~.61111
maL pritbs
Process Dispatch UlCncy
MINps
MAXps
655
1991
Table 2 HP-UX
HP9OOl&S Harris Night
CXJRT
Hawk 3400
R11J
Coocwrclll/ Masscomp 6300
REAlAX
AEG·MODCOMP Tri·D9730 (MC 68030-25)
I
Table I
thrwgb cbe Iu:rocJ
l.~
ms
Hardware
OS
I
P=piODON
90
430
1605
11605
50
Kernel Latency Time for HP-UX
For HP·UX we see that the MKLT is 14.6 ms. whereas the maximum PDLT is 1.991 ms. As described in [DOUa] and [DOUb] the kernel of HP-UX is not fully preemptable because the concept of critical regiollS guarded by semapbores (see section 3.2) is mixed with the concept of preemption points. This leads to a bigher PDLT. Other available data ([KRIEJ. [GRA] and [MODq) underline the general supposition that the Maximum Kernel Latency Time is much higher than the Process Dispatch Latency Time.
360
To comparing real-time UNIX systems with specialized realtime operating system kernels Table 3 and Table 4 show some application oriented characteristics for two operating systems measured on the same hardware (Obtained from [GRA]). The considered UNIX system looks not so bad. Surely the results depend on the considered load (terminal VO in Table 3 and Table 4). We expect not so good results if network applications are included in the load.
Process Dispatch Latency Time for some Systems
In contrast to the values for HP·UX in Table 1 HewlettPackard has published values for the kernel latency time of HP·UX ([DOUa] and [DOUb]). These values are shown in TJble 2.
250
IITQS.UH
b contrast to the kernel the latency on higher levels concerns only user tasks using the same service class. Especially tasks using only kernel services cannot be delayed by locked entical regions on higher level.
REAl./IX
(SjxaaI (ltaI·TIID< Uh'lX) Rtaj·TlID< KantJ)
I
Mu. lnltlT1lll Frtqucocy (\HI)
14.7
6.67
InltlT1lll RcsaiulioolilllC (fJS)
6.6
150.0
SSl 6.3.9
171
S7J
128.8
110
2JI
Pnxxss Dispadl ulClCy lilllC } ( fJS J MiD. wilhwt Load
Mu.
Pnxxss DisJlldl UIaICy TIIIIC } ( fIS J MiD. wimLoad Mu.
The problem of latency times on higher levels is weaker to some extend. but the problem still remains. We have seen Ibat for the kernel latency time it is bard to get some figures by measurements without knowledge of the intemal strucrure of Ibe kernel. This is true too for latency times on higher levels. Wormation from vendors are quite unusual.
1~
5. COSCLUSIONS Hardv.'3rC:
MVME 147 with MC68ro-25 MHz
Table 3
It is fairly good understood what services should be provided by an operating system for real-time applications. and it is time to standardize it. The POSIX real-time extensions Ire going to establish such a standard. Although the standardisation process is not completed real-time UNIX sysu:ms available today are more or less on the line of Ibis standard.
Event Reaction Olaracteristics
RTQS.UH (Spacial
RcaJ·T_lmdl
InlClldcd Cycle T1IlIe Talms]
REAlAX (RcaI·TIIIC UNIX)
2!XXl
2.(ID3
}:
·015 0.00 2000
·1.28 +1.28 20833
DeviatiooofTol'J } :.
- 11.75
+11.75
- 4.96 +4.96 2.0833
Withoot Load: Deviatioo of To I' J
Mean Value Ims] With Load:
Mean Value [msJ
2000
Focussing on the process dispatch latency time and other unexpected delays we have discussed some crucial issues concerning real-time performance of operating sysu:ms and collected some results of measurements for UNIX systems claimiDg to have real-time properties. These results show that some of these UNIX Systems are well suited for I class of real-time Ipplications. Comparing with dedicated small realtime operating system kernels we have to tate into account that UNIX offers I much more extensive functionality which costs some overhead. but simplifies user programs if needed. Disregarding operating economy the use of real-time UNIX systems makes SCDSC in many cases. h is difficult to comparing performance data from different vendors, because mostly the metrics used for measurement are not exact the same or the operating conditions during measurements differ. The definition of performance metrics and its mandatory support for POSIX implementations is • step in the right direction. But our analysis shows, that unexpeaed delays of considerable duration caused by the operating system may happen in user tasks which are not Included in these metrics. Unfortunately these unexpected delays are not easy to measure without inside knowledge of the kernel.
Hardware: MVME 147 with MC 6S)J(}.25 MHz Table 4
Olaracteristics for Cyclic Tasks
4. HIERARCHY OF LEVELS Powerful operating systems are no longer a monolithic entity. For reducing complexity the internal structure consists of a hierarchy of levels of which the lowest is the kernel. It provides only I minimal set of functions from which the rest of the operating system can be constructed. The critical regions of the kernel can be kept short. Some user services may be performed by kernel functions directly. other services by functions on a higher level using kernel functions indirectly. Examples are file access and management servIces or network services.. Similar to the kernel services on higher levels use internal data structures to manage their objects and resources. These internal data structures may be temporarily inconsistent too. Sharing services between several user tasks leads to the same problems as described in section 3.2. These problems are resolved in the same way establishing critical regIOns guarded by kernel semaphores. Oearly we have some kind of latency time on higher levels too. Services on higher levels are not so closely linked together. They form classes with no common critical regions for any two classes.
251
REFERENCES (DOUa]
Doughty. 5.M; S.F. JUry; S.R. Kusmer Ind D.V. UNIX Cor Rullime; Proceedings oC the UniForum 1987. Wasbington D.e.. January 1987. pp. 219-230
[PX.4]
Rultimc Extensions Cor Portablc Openating Systems; IEEE Pl00J.41D12 Feb. 1992
(PX.4a]
Threads Extensions Cor Portable Operating Systems; IEEE Pl00J.4a1D6 Feb. 1992
[PX.4b]
Rcaltime System API Extension; IEEE P1003.4blDl Nov. 1991
[PX.13]
Staadardized Application EnviroDJDellt Profile· POSIX Rca.ltime ApplicatioD Suppon (AEP); IEEE Pl003.131D4 Nov. 1991
[RZEa]
Rzehak.. H.: Ecbtz.eitdateDvcrubcirung - Grundlagcn uDd Mcthoden Ciir die Praxis; E1cktronik. Nr. oS - Nr. 7. 1991; Fruzis. Verlll MiiDchcn
[RZEb]
RzebaL H.: Distributed Systems Cor Reallimc Applications- Using Manufacturing Automatioll U ID Example; Ill: M. Schicbe. S. PCerrer (Eds.). Real-TLlDC Systems, EngiDeering IlId Applications, lCIuwcrVerlag. 1992
[RZEc]
Rzehak. H.: Distributed Real lime Systcms Cor MuufacruriDg Automation; IFAC Symposium 011 IDlCUigent CompoaCIIIS Ind llISuumcnlS for Conaoi Applications (SIQCA '92). Malaga. May 2()'22, 1992
Lt~n:
(DOUb]
DouGhty. S.M.: Adding Real TLlDe Capabilities to the UNIX Operating System; Proceedings oC tbe PEARL 87 Workshop Iiber Rulz.eitsysreme, Boppatd. lA. Dez.. 1987. pp. 151-165
[FOG]
Fogelill. J.: The VlIWorks Real-TLlDe Kuud; Wind River Systems, Alameda
(F'U1U]
Furbt, et al.: Real-time UNIX Systems: Dc:sip aDd Application Guide; lCIuwer Aadcmic Publisbcn, 1991
(fURb]
Furbl, B.: The Reality oC a Rca.I-TLlDe UNIX Operating System; Proceedings oC the coDveDtiOll Ecbtz.eit '91. SiadeJfiDgeD, 11.-13. JIIJIi 1991. pp. 281296. Ludwig Drcbinger GmbH. MUDcben
(GRA]
GnalJa, e.: Quantitativer Verglcicb VOD drci Ecbtzeitbetricbssystcmen CUr die digitale Regelung; Proceedings oC the PEARL 91 Workshop iiber Rc:alz.eilSysteme, Boppard. 28.-29. Nov. 1991. pp. 141·155. InCormatik- Facbbericbte. Springer- Vcrlag
(KRlE]
Adding RcaI-Tune Capabilities 10 I StaDdard UNIX Implcmcatation: Thc AIX VCrsiOD l.l.5 Approacll; PlOCCCdings oC the coDVcDtion Echtzeit '91. SilldclflDgcD, 11.-13. Juni 1991. pp. 63-71. Uidwig Drcbinger GlIlbH. Muncben
[SllE]
Stiegct. JC.: Eine PEAJU,AblauCumgcbUIIg Ullter AlISllulZUllg zweict unterschiedlicbct Betricbssysteme; Pr-oc«dings oC the collvention Ecbtz.eit '91. SindelfiDIlen. 11.-13. Juni 1991. pp. 203212, Uidwig Drcbinger GmbH. Muncbell
[lEV]
l..cvi. S.T.llld AJC. Agrawala: Rca.I-lime System
[TRE)
Trcnnhaus, W.: Du M.ikrocompu\e!$ystem SX mit seinem Ecbtzeit-UNIX·Betriebssystem SORlX; EiD Sysrem Ciir die Realisienmg YOD veneilreD AUlOmatisiClUJlgs-AuCgaben; PEARL 89 Worlcshop abet RealzeilSysteme, BoPt-rd. 6.-& Dez.. 1989. pp. 157-171. InCormatik· Facbbericbte, Springer- Verlag
[ZEL]
:Ultwangct. H.: Trend zu Ecbtz.eit-Unill und Crossentwicklung uater UnilI; E1ektronik 511991 S. 1~19O; FraJIZis- Verlag Miincheu
a
Design; McGnaw- Hill PublisIUng CompaDY. 1990 [MODC)
REAlJSTAR 1000 and 2000 Series Benchmark Report; MODCOMP BenCbllllriCDg Group. Modular Computer Systcms, FL Ltuderd.ale, 1991
(pX.l]
Portable Operating System Interfacc Cor Comput.er EnvironmcnlS; ISOIIEC 9954-1 : 1990 and IEEE Std 1003.1·1990 (POSIX1)
252