A file management system for a magnetic disk used as a buffer to write-once optical storage

A File Management System for a Magnetic Disk Used as a Buffer to Write-Once Optical Storage Daniel A. Cafias Department of Mathematics and Computer Science, Wake Forest University

William G. Bulgren Department of Computer Science, University of Kansas

Write-once optical disks offer a technology that includes high capacity and removability for handling the informationstorage requirements of today’s “information age”. A multiple-write buffer is proposed to integrate write-once storage into existing systems and to provide an interface that is transparent to the user. This paper discusses the design of a multiple-write buffer for integrating WORM optical storage into existing computer systems. A brief discussion of the attributes of optical storage motivates the design of file management systems with a magnetic disk acting as a buffer for a WORM optical disk. The required data structures for the magnetic buffer file management system (MBFMS) and the optical disk file management system (ODFMS) are discussed. The major responsibilities of the MBFMS in acting as a go-between for the existing file management system (FMS) and the ODMS are outlined. Techniques for manipulating MBFMS and ODFMS data structures, responding to FMS directives, and submitting requests to the ODFMS are also presented.

This paper describes a file management system (FMS) for a magnetic disk used as a buffer between an existing operating system and write-once optical storage. Included is a brief discussion of the advantages realized through use of the magnetic disk buffer, as well as the duties and responsibilities of the magnetic buffer file management system (MBFMS) for this design. Techniques for data structure manipulation and requests to the optical disk tile management system (ODFMS) are given. References made in this report to optical disks or optical storage are meant to indicate WORM optical storage technology. The attributes of write-once optical disk technology and a detailed description of this design for integrating optical disk storage into existing systems are found in Refs. 1 and 2. 2. THE MAGNETIC

BUFFERING

SYSTEM

1. INTRODUCTION

As discussed in Ref. 1, WORM optical disk technology presents a vast resource for meeting the accelerating demands for storage media. Through the complementary use of magnetic storage and WORM optical storage, problems of write-once technology can be minimized, resulting in a system that provides the user with the advantages of both types of media. In addition, the use of a magnetic disk as a buffer eliminates the need for major changes to existing operating systems in order to interface with WORM optical storage.

Address correspondence to Daniel A. Caiias, Department of Mathematics and Computer Science Wake Forest University, Box 7311, Reynolda Station, Winston-Salem, NC 27109.

2.1 Justification Several attributes of optical WORM storage justify the use of multiple read/write media as a buffer between the existing operating system and the optical storage device. Since current file systems were designed to interact with multiple read/write media, they are based on the assumption that once written, an area in storage may be rewritten. An examination of the techniques for handling tile maps and directories clearly illustrates this point [3, 41. Blocks written on WORM optical media may be written once. A multiple read/write buffer eliminates the need for changes to existing file management systems due to this write-once characteristic of secondary storage by intercepting commands from the file management system and handling all direct communication with the ODFMS. Thus, the existing operating system file

15 The Journal of Systems and Sofhvare 10, 15-21 (1989) 0 1989 Elsevier Science Publishing Co., Inc.

0164-1212/89/$3.50

16

D. A. Ctias and W. G. Bulgren

(c/bit) ;i -6 lo-’ 10-S lo-*

1o-7

1o-6

1o-5

1o-4

1o-3

lo-*

10-l

1

10

Access Time (Seconds) Figure 1. Comparison of data storage technologies. Redrawn from Ref. 6.

manager needs no knowledge of the write-once nature of the secondary storage device. Besides eliminating the need for changes to existing operating systems, the use of magnetic media as a buffer can minimize the writing of “temporary” data to the optical disk. Paging and segmentation. techniques used by many operating systems in the management of virtual memory require that blocks of data be swapped in and out along with updated page maps and directories [5]. A large amount of file storage and retrieval activity would quickly consume even a high-capacity write-once disk, leaving it full of useless “temporary” data. The multiple read/write buffer allows information to be written to the optical device only when the optical disk is physically ejected from the drive, or when the magnetic buffer becomes full and must locate some free space in order to proceed. Thus, “temporary” file activity is handled on the multiple-write media, and writes to the “permanent” optical media are minimized. The slower access times for optical devices also provides some justification for the use of a multiple read/write magnetic disk buffer. The average access time for optical drives currently exceeds 100 m in most eases (Figure 1). This is due in part to the fact that optical read/write heads with their lenses and prisms are considerably heavier than magnetic heads [7]. For many on-line applications, this is an unacceptable feature. The use of high-speed magnetic storage as a buffer can allow portions of a required file to be transferred to the buffer, where faster access times are possible. One advantage of optical devices is their high capacity, and hence very large files may be stored. The capacity of the optical media is much greater than that of the magnetic buffer; therefore, it is conceivable that a particular file may be too large to reside, in its entirety, on magnetic disk. Some form of loading portions of the file to the magnetic buffer is necessary. The design for a

magnetic buffer is based on a file management system that uses paging techniques for data transfer between the optical device and the magnetic device. Data may be transferred from the optical disk in minimum “block” sizes. For optimal performance, the size of a page on the magnetic buffer should be some multiple of this optical disk block size. In order to take full advantage of the permanence and high capacity of WORM media, and yet maximize tile storage and retrieval, a magnetic disk as a multiple read/ write buffer may be used. Many problems of write-once media can be hidden from the user by the complementary use of magnetic and optical devices [8]. 2.2 Description The primary duty of the magnetic buffer file manage-

ment system is to intercept commands sent by the FMS to secondary storage. Based upon the command, the MBFMS will issue requests to the ODFMS and/or access the magnetic disk, as needed, finally signaling the FMS that the requested function has or has not been accomplished (Figure 2). Figure 2. Flow of communication. OPERATING SYSTEM

FMS

System Directive

A

A

A File Management System for Optical Storage ODFMS directory

ODFMS page

Figure 3. Data structures used by the ODFMS.

Data Pages

The MBFMS attempts to maintain files in a state that will minimize necessary updates on the optical disk. To do so, it keeps track of an optical page number for each page of data. The optical page number corresponds to the physical page number as known by the ODFMS. This info~ation may be passed, along with the updated data page, to the ODFMS when the fde is forced to optical storage. Since the ODFMS maintains a copy of the tile as it was first loaded into the magnetic disk, only altered (“dirty”) pages, rather than the entire file, need be rewritten on the optical disk. Since data transfers are made only on page boundaries, an additional responsibility of the MBFMS is to calculate the required page(s) for data transfers. The MBFMS keeps track of which pages are transferred to the FMS read/write buffer, so that the alteration of these pages may be recorded and ultimately transferred to the optical disk. An additional responsibility of the MBFMS is to locate victim files for garbage collection when space is needed on the magnetic disk. To the extent possible, MBFMS will free up the space being used by non-altered (“clean”) pages for system use. If necessary, the MBFMS will then force files to be updates on the optical disk, and place the associated file space on its free list for system use. By translating commands from the FMS into appropriate action on magnetic and/or optical storage, the MBFMS maintains transparency of the optical device to the existing operating system. In addition, file management techniques and garbage-collection strategies are designed to provide minimal updates to optical storage. The resulting system takes advantage of attributes of each technology to enhance the other, thus providing the user with advantages of both magnetic and optical storage. 3. ODFMS AND MBFMS DATA STRUCTURES 3.1 Introduction

In order to achieve system transparency while maintaining information on write-once media, the MBFMS and ODFMS must manage directory and file info~tion in

appropriate data structures. A description of the minimial set of info~ation and corresponding data structures required for this design, for both the ODFMS and the MBFMS, follows. Pseudocode specifying the manipulation of the MBFMS and the ODFMS data structures can be found in Refs. 9 and 10, respectively. 3.2 ODFMS Data Structures

The ODFMS data structures are the facilities that the ODFMS uses to manage files resident on the WORM disk (Figure 3). The data structures of the ODFMS permit the current state of stored files to be reconstructed from the info~ation retained in the magnetic buffer and the previous version of the file permanently stored on the optical disk. Virtually no constraints other than available disk space are placed on file and directory growth. This is accomplished by physically separating the directory area and the data area of the optical disk. Pages and page table(s) associated with a file are organized in blocks (not necessarily contiguous blocks) allocated in the data area. The ODFMS file directory consists of contiguous records of file information allocated in the directory area. The directory is recorded starting at one end of the disk, and the files and their page tables at the other end. The directory area and the data area are permitted to grow until the two meet. ODFMS file directory structure. Entries in the ODFMS file directory are contiguously stored records. Each record consists of specific file info~tion recorded in fixed-length fields. The minimum set of information needed includes: a unique file identifier, a unique file name, file size, date of file creation, the date the file was last modified, a bit indicting that access to the file is not permitted, status information, and an optical disk address designating the location of the page table associated with the file. ODFMS page divided into pages storage in the data page table of a

table structure. Data in a file are mapped into noncontiguous blocks of area on the optical disk (Figure 3). A file is stored in a separate block

18

D. A. Ctias and W. G. Bulgren

f

File A

Data Pages for File A

ata Pages for File B

Figure 4. Data structures managed by the MBFMS. containing pointers to storage blocks that have been allocated to data pages. For large files, the page table may have more than one level, in which case it may overflow into several noncontiguous blocks on the disk. All blocks assigned to a specific page table are allocated in the data area and are sequentially chained. Such links are concentrated in the header of each page table. Each block of the page table consists of a header and a linked list of records. The information comprising the page table include: a backwards link to the previous block of the associated page table, a forward link to the next sequential block of the associated page table, and a bit indicating that no available free space exists in the block. Information comprising the page table entries include: a pointer to the page table entry for the next sequential data page of the file, a link chaining the entry for the original data page and the page table entry for the new version or replacement data page, and the location (block address) of the data page on the optical disk. ODFMS page structure. The data page, the unit of data access, is the fraction of a file that can be transferred as the result of a single optical read or optical write operation. In addition to the actual data, the page length is recorded as the first record of the block associated with each data page. The page length is the actual number of bytes of data stored in the block. Logical pages and physical pages on the optical disk are identical, although, physical pages of the file do not necessarily correspond to full blocks on the optical disk. The pages of a file remain logically contiguous, but corresponding blocks may be allocated noncontiguously. All blocks relating to data pages are allocated in the data area on the optical disk. 3.3 MBFMS Data Structures The file organization on optical storage must is the same as the file organization on magnetic buffer [2]. This will

avoid recopying the entire file when it has to be updated on the optical device. Figure 4 provides a general illustration of the data structures in this design. This design describes only a minimal set of information to be maintained for each file. Each element of the MBFMS data structures, along with additional information that must be kept as part of file management, is described below. Directory structure. Information kept as part of the file control block (FCB) for each file recorded in the MBFMS directory include: a unique identifier assigned by the system upon creation of a file, a name specified by the user, the data of file creation, the date of the most recent modification of the file, a bit used to indicate .whether the file has been deleted, status information (file organization, access control data, file type, etc.), file size and the number of clean pages of this file which currently reside on the magnetic disk, and a pointer to the beginning of the page table for the file. File structure. A paged file scheme requires data structures for a page table and for the page of data. Changes to a file, prior to saving or committing, must be kept separate from the original file. The working page table is used as a “temporary” table for keeping information of pages that have been altered but have not yet been saved. Page table structure. Each directory entry for an active file includes the starting address of a table containing information about the pages of the file. There is one entry in the page table for each page of the file transferred from optical storage, or appended to the end of the file. The page table consists of: a logical page number assigned sequentially to each physical page as it is created or read from optical storage, the total amount of information associated with the logical page (more than one physical pages may be represented in the page size as shown in Figure 5), a bit used to indicate pages

A File Management System for Optical Storage

that must be updated on optical storage, and a pointer to the beginning of the (first) physical page of data associated with the page number. Working page table structure. The working page table is similar to other page tables managed by the system. However, some different information is maintained in the working page table and, in addition, a link to the original file must be kept. Entries in the working page table include: a pointer to the file entry in the directory, a page number, page size, a bit that indicates that the page currently resides in the FMS read/write buffer, and a pointer to the beginning of the (first) physical page containing the updated version of the associated page. The page table entry of any page read into the FMS read/write buffer is recorded in the working page table. When information is transferred from the FMS buffer, the data page(s) is linked to the working page table entry(s) marked as current in the buffer. A transfer of page(s) to the FMS buffer results in marking any pages “current-in-the-buffer” as not “current-in-the-buffer,” prior to the data transfer. Subsequent to the data transfer, the appropriate working page table entry(s) is made and is marked as “current-in-the-buffer.” A dirty mark is not required in the working page table since any entry linked to a data page must be dirty. Page structure. It is necessary, for efficient updating of files on optical storage, to maintain logical pages on the magnetic disk. Logical pages correspond to optical disk pages, and may consist of multiple physical pages. Besides the actual data, each physical page contains the page length and a pointer to the extension of this physical page, which is made necessary by insertion or expansion of the (logical) page. Since the size of a logical page may be altered on the magnetic buffer, provision must be made by MBFMS to associate multiple physical pages with one logical page. This is accomplished by linking an “extension” (new physical page) to the previous physical page associated with the logical page.

Figure 5. A logical page may be associated with multiple

physical pages.

4. SORWARE

INTERFACE

Communication between the magnetic buffer and the optical disk is coordinated by the MBFMS/ODFMS interface. The MBFMS issues requests to the ODFMS when information needs to be transferred between the magnetic disk and the optical disk. For each request received by the ODFMS, a flag is returned to the MBFMS in addition to any necessary parameters. If the ODFMS cannot perform the requested operation, it responds with the appropriate signal. In the same manner, if a request is successfully completed, the MBFMS is informed. Furthermore, it is assumed that all operations will signal exceptional termination. The ODFMS is responsible for storing and retrieving files on the disk. A current directory of files is maintained. Likewise, pages and page tables associated with file updates are managed allowing file pages to be referenced in a logically sequential order. In the following discussion, a file is resident if it has been recorded on the optical disk, and a file is called active if it resides on the optical disk and has not been marked for deletion. During a file update, there are essentially two current versions of a file: the last committed version on the optical disk and the new updated version on the magnetic buffer.

4.1 MBFMS Issued Requests

The magnetic buffer file manager issues requests to the optical disk file manager whenever access to information on the optical disk is required (Figure 6). As an example, consider the usual OPEN command issued by the FMS. The FMS issues a command to OPEN a specific file. The MBFMS intercepts the open command and attempts to OPEN the file using the magnetic buffer directory. If the OPEN is successful, the MBFMS proceeds to create a working page table and a link associating it with the original file. If the attempt

20

D. A. Caiias and W. G. Bulgren

7

MBFMS

opcreate: create a file on the optical disk opopen_read: open for read opopen_write: open for write opread: read a page opwrite_page: write a page opdelete: delete a file oprename: rename a file opcopy: duplicate a file oprestore: restore a deleted file dir-request: directory of files , check_opt_dir: check existence of a file

1 ODFMS 1

Figure 6. Requests issued by the MBFMS.

fails, the MBFMS issues an OPGPEN-READ command with the specific file name to the ODFMS. If the OPOPEN-READ command verifies the existence of the specific fde as an active file on the optical disk, the file directory info~ation is transferred to tbe MBFMS. The MBFMS records the directory information in the magnetic buffer directory, creates the required page tables and links for the original and working page tables, signals the FMS, and returns tbe required directory information. If the OPOPEN-READ command returns a flag of false to the MBFMS, a signal is returned to the FMS indicating the file is not active on either the optical or the magnetic disk. It is not the purpose of this paper to detail the datastructure m~ipulation. Clearly, this is an important part of an implementation and if the reader is interested in these details, please contact the authors. 5. CONCLUSION

Optical disk storage offers major advantages over traditional magnetic storage, such as extremely high capacity, low-cost off-line and on-line storage, removable media, and the elimination of head crashes, head wear, and media wear. They also have some disadvantages compared to magnetic storage including long average access times, complicated error management routines, and nonerasability. For the near term, optical disk technology is a writeonce, read-m~tiple-times technology. The use of WORM disks requires software developers to address a unique difftculty in that info~ation written to the disk cannot be erased and hence requiring modifications to existing I/O subroutines and tile managers to support this new peripheral, This paper outlines a tile system for write-once optical storage based on the integration of a multple read/write magnetic buffer as an interface bewteen the optical disk and the existing file management system. The magnetic

buffer eliminates the need for alterations of the file ~agement system to support the WORM disk as online secondary storage. The MBFMS provides the interface allowing the resident FMS to gain access to data on the optical disk. Directives issued by the FMS are intercepted by the MBFMS and redirected in the form of requests to the ODFMS. A request from the FMS for information in optical storage causes the requested information to be transferred to the magnetic buffer prior to being accessed by the FMS. Furthermore, all info~ation to be stored on the optical disk is initially recorded on the magnetic buffer. The ODFMS is responsible for storing and retrieving files on the optical disk. The ODFMS stores all information pertaining to a file in a contiguous tile directory in optical storage. Files are divided into pages that are mapped into noncontiguous blocks in the optical disk data area. File page tables are stored in separate blocks, each confining pointers to data pages. The file page table organization designed allows the ODFMS to store, update, and retrieve pages of a file in a logically sequential order. Optical disk storage presents a new area of research in both on-line and off-line secondary storage systems. Implementation of the system described in this design is an area for further development. In addition, several details of the optical disk file system will profit from performance analysis. Analysis of the system may help to resolve certain unanswered questions pertaining to trade-offs within the system. In particular, statistical analysis could determine the significant of “reverse garbage collection” on a WORM optical disk. Reverse garbage collection refers to the collection of all the good files on the disk and their transfer to a new disk. File compaction on the optical disk is inefficient in regard to the ad~tio~ blocks consumed during the compaction process. However, since there is also a subs~nti~ performance penalty in transferring fragmented files, research and analysis in this area could disclose instances when file compaction might actually be beneticial. Future research could involve the design of a file system providing the following features: retaining a complete history of all files and directories (providing a true audit trail of tile updates), selection and retrieval of earlier versions of a file, provi~ng a multilevel or hierarchical tile system, identifying individual optical disk cartridges, and keeping a master list of all cartridges introduced into the system. Opportunities also exist for research of other system designs incorporating optical disk storage. One such design might involve a direct communication link between the operating system and the WORM disk,

A File Management System for Optical Storage allowing the user the option of designating magnetic or optical file storage. This area might also include an investigation of WORM optical disk storage attached to a local area network, granting all computes on the network access to files on the optical disk. Even though several attributes of optical storage are attractive, optical disks are not necessarily best suited for all applications. There is currently a great demand for a comprehensive comparison of magnetic storage, optical storage, and magnetic/optical storage to determine the best solution to storage requirements of present and future applications. Many additional areas of research involving optical storage technology are open for investigation and the number will increase as improvements are made in optical disk drives and media. Furthermore, the opportunities for future research will be unbounded with the intr~uction of erasable optical disks into the market place.

REFERENCES

1. J. K. Gosa, A Design of a Buffered File System for WriteOnce Disk Storage Technology, Masters Thesis, University of Kansas, Lawrence, Kansas, 1986.

21 2. L. R. McCormick, A File System for Write-Once Media, Masters Thesis, University of Kansas, Lawrence, Kansas, 1986. 3. E. I. Organic, The Mu&its System: An ~arn~na~io~ of Its Structure, MIT Press, Boston, Massachusetts, 1972, pp. 217-235. 4. M. K. McKusick, W. N. Joy, S. J. Leffler, and R. S. Fabry, A Fast File System for UNIX, Berkeley UNIX Documentation (4.2), 1983. 5. J. L. Peterson and A. Silberschatz, Operating System Concepts, 2nd ed., Addison-Wesley, Reading, Massachusetts, 1985, pp. 204-210. 6. L. Helgerson, The Business of OD3: An Overview of Digital Data Disk Technology, Videodisc and Optical Disk 4(6), 294-306 (1984). 7. C. Panasuk, Advanced Computer Mass Storage, Electronic Design 32(9), 259-274 (1984). 8. L. Fujitani, Laser Optical Disk: The Coming Revolution in On-Line Storage, Commun. ACM 27(6), 546-554 (1984). 9. J. K. Gosa, A Description of the File Management System for a Magnetic Disk Used as a Buffer to Write-Once Opticat Storage, TR-86-13, University of Kansas, Lawrence, Kansas, 1986. 10. L. R. McCormick, A Description of the File System for WORM Optical Disk Storage, TR-86-14, University of Kansas, Lawrence, Kansas, May 1986.