- Email: [email protected]
The LIMS database
d
Laboratory automation and information management Laboratory Automation and Information Management 3 1( 1996) 161- I7 1
Tutorial
The LIMS database Mary Hinton, Phillip R. Hinton Jr. Applied Computer Systems, 3540 Country Ct. N., Mobile, AL 36619. USA
Received 15 September 1995; accepted 3 October 1995
Abstract A LIMS amounts of implement advantages fits in with
is one of the many information management systems on the market today that manages complex and massive data. A history of database technology is presented, followed by a look at the current database models used to a LIMS. How successful each model is in handling complex scientific data is discussed, along with the and disadvantages of each model. The LIMS database model of the future is projected. How the LIMS database the overall enterprise-wide data management design is also examined.
LIMS; Database; Model; Design; Management
Keywords:
Contents ......................................................... 2. Historical use of databases. ................................................. 2.1. Hierarchical model ................................................... 2.2. Network model. ..................................................... 2.3. Relational model. .................................................... 2.3.1. Client-server relational model ........................................ Network-relational hybrid ............................................... 2.4. Object-relational hybrid ................................................ 2.5. 2.6. Object database model ................................................. 2.6.1. Object client-server architecture ....................................... 2.6.1.1. Semer-centric ............................................. 2.6.1.2. Client-centric ............................................. 2.6.1.3. Object server .................................
162 162 162
1. Introduction.
2.6.2. ObjectSQL
....................................................
............................................. ............................. 2.6.3.2. Transparent data distribution .................................... ........................................ 2.6.3.3. Schema management. 2.6.3.4. Object mapping. ...........................................
2.6.3. Object model features 2.6.3.1. Referential
integrity in the object model
092%5281/96/$15.00 0 1996 Elsevier Science B.V. All rights resewed SSDI 0925-5281(95)00023-2
163 163
164 165
165 165 166 166 167
, ...........
167 167 167 167 167 168 168
M. Hinton, P.R. Hinton Jr./Laboratory
162
Automation and Information Management 31 (1996) 161-171
2.6.3.5. User-defined .types .......................................... 2.6.4. Object techniques and design tools. ..................................... Databases for complex data ................................................. Examples of complex scientific data ............................................ Standardization of scientific data formats ......................................... Modeling the information flow through a laboratory .................................. LIMS vendors’ solutions ................................................... Data exchange between databases across the enterprise. ................................
3. 4. 5. 6. 7. 8. 9. Conclusions ........................................................... References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1. Introduction A LIMS is one of the many information management systems on the market today that manages complex and massive amounts of data. A common goal for all information management systems is the development of a data management paradigm for the 21st century. The core of any LIMS is the database. With an ever-increasing load placed on the LIMS, there has been a dramatic slowdown in the information flow in some laboratories. Even when computers that run the LIMS software are replaced with faster computers, inevitably the LIMS begins to bog down again as the database grows in size and complexity. Most vendors are using the relational model as the underlying database. LIMS and other information management system vendors are taking a second look at this model for complex data management.
2. Historical
168 168 168 168 169 169 169 170 170 171
between the parent and child node is called a set with the parent record being the owner of the set and the child records being members of the set. This structure has built-in integrity because of the use of linked lists to define the set relationships. Although the hierarchical model lacks the flexibility to add new data relationships while the database is on-line, it is still the database of choice for many large information management systems. Fig. 1 is an example of the hierarchical model. In this diagram, it models some of the relationships that may exist between entities found in a LIMS for a manufacturing laboratory. The relationships are represented by the branches linking the parent and child nodes. To implement the hierarchical model, a child node can be the parent of one or more nodes, but each child node can have only one parent. In other words, a child node can be a member of only one set, but can be the owner of one or more sets [l].
use of databases
To explain the latest database technology available for LIMS, a brief history and description of the various database models would be appropriate. 2.1. Hierarchical model The hierarchical model was the first widely used standard database model. This model can be visualized as a tree structure containing parent and child nodes. The topmost level has one parent called the root node from which child nbdes can descend; each child in turn, can have descendants. The relationship
Fig. 1. Hierarchical Model. Reprinted courtesy of Marcel Dekker, Inc.
from Ref. Ill, p. 218 by
M. Hinton, P.R. Hinton Jr./Laboratory
Automation
Because of the restrictions placed on parent-child relationships, some records in Fig. 1 must be duplicated, such as Plant and Grade. Redundancy is common with the hierarchical model.
and Information Management 31 (1996) 161-I 71
163
simultaneously, duplicate Plant and Grade records have been eliminated [l]. 2.3. Relational model
2.2. Network model Next came the network database model, which is also based on set relationships. It is actually a superset of the hierarchical model as it allows child nodes to descend from more than one parent as long as each parent is in a different set (relationship). The network model as the name implies has a network structure of parent and child records linked together by set relationships. A child can be owned by more than one parent and it can even own itself. The network like the hierarchical is very fast because of its reliance on linked list database pointers to navigate directly to data described by set relationships. Since each record contains the address of related records, navigating through the database from one record to a related record is fast, even as the database grows large. As with the hierarchical model, changes to data records or data relationships must occur off-line because of the embedded database addresses used for direct navigation. Because of the nature of the set relationships, referential integrity is automatically assured. Fig. 2 uses the same entities as presented for the hierarchical model, except this time the network model is implemented. In this representation, there is no need for redundant information. Since the Plant entity can now be owned by Customer and Grade
Fig. 2. Network Model. Reprinted from Ref. [I], p. 219 by courtesy of Marcel Dekker, Inc.
The relational model followed soon after. This model is much easier to implement for data that can be described in simple tabular format (as found in many business applications). It does not build on the hierarchical or network concepts, but instead is based on set theory (specifically, relational algebra). The relational model can be visualized as tables containing rows of data records. The columns are the fields or attributes of each data record. Each record has a field or set of fields that serves as a primary key that uniquely identifies each row. Relationships are implicitly described by using a field that is common between two tables as a key for accessing the related records in the tables 121. Data is manipulated using SQL (structured query language), which has set operations, such as selection, join, union, intersection, product, and projection. This model is more flexible than the hierarchical or network model. In a hierarchical or network database, record definitions are fixed in order to embed linked list data pointers in the records. Since the relational model uses keyed fields to relate records, record definitions may be easily changed and new records created. However, this flexibility comes with an integrity and performance price. To ensure integrity, user-defined integrity constraints must be established and checked before processing transactions. In the hierarchical or network model, the constraints are automatically established when the sets are implemented, while in the relational model, the constraints are user-defined in the application program. Since it is so easy to get the system up and running, the design of the user-integrity constraints may not be well thought out in advance and as the database grows in size and complexity, more tables are often added to the system, and therefore more joins are needed to access information in the database. (A join is one of the set operators of relational algebra used to concatenate data found in different tables.) The more joins needed to process a query, the slower the system responds. Another disadvantage for some information manage-
M. Hinton, P.R. Hinton Jr./Laboratory
164
Automation andlnformation
I
Plant ID
*CustomerID 4
Plant Name Plant Address
Plmt-Omde Table
*Rimrum Fig. 3. Relational
l‘S-d=YKy
Model.
ment systems based on the relational model is that complex data cannot be adequately described in simple rows and columns. In the LIMS environment, using the relational model to describe complex data, it is not uncommon to do 10 or more record joins to process a single information request. Despite its disadvantages, this model requires less initial planning and design to implement and is the quickest and easiest of the models to set up. It soon became the most recognized and widely accepted database model for microcomputers and minicomputers. The relational model became the panacea for every information storage problem. Whenever changes or additions to a database were needed, new tables were added or existing tables extended, often without determining the performance impact on the as information systems system. Consequently, changed over time, the performance often degraded dramatically. To implement the relational model for the same partial LIMS schema presented in Fig. 1 for the hierarchical model, tables would need to be created for each entity relationship in the schema. Fig. 3 shows the mapping required in the relational model for the Customer-Plant and Plant-Grade relationships of the LIMS schema, where each customer (company) has one or more plant sites and each plant purchases one or more grades of products. A table is created for the Customer, which includes fields for Customer ID and Customer Name. A table is also created for the Plant, which contains the Customer
Management 31 (1996) 161-171
ID, Plant ID, Plant Name, and Plant Address. In this situation, the Customer ID would be the primary key in the customer table, but the secondary key in the plant table. It links the information contained within the two tables and implicitly forms a relationship between the two entities. To describe the Plant-Grade relationship, Grade and Plant-Grade tables are added. Instead of direct navigation as described in the hierarchical and network models, the relational model requires joins for the Customer, Plant, and Grade tables to find out which Customer purchases a particular Product-Grade. Fig. 3 shows that in order to navigate to a related record, the secondary key is used to find the primary key in an index file, which contains a sorted list of keys and physical addresses for each table record. Multiple database accesses are usually required to locate one record in an index, and as the table grows larger, the more database accesses are required. This is why relational databases slow down as they get larger. Unlike the hierarchical and network databases, which depend on the physical structure of data, a relational database provides data independence, which means that the physical and logical structures are separated so that the user applications do not have to worry about the actual physical address of data. Other features of relational databases include concurrency, recovery, transaction, and an ad hoc query facility. 2.3.1. Client-server relational model Originally all database files were stored on mainframes or minicomputers. Since dumb terminals were used to access the host computer, all processing occurred on the host. Relational databases were developed in this environment, in which the database server resided on the host and the application programs were processes on the host. When a large number of users needed access to the database information simultaneously, a bottleneck occurred on the host CPU. Dumb terminals were eventually replaced with powerful microcomputer workstations and the client-server architecture was developed to take advantage of the power of the workstation. The application processes were now run on the clients, while the database server remained on the host. In the meantime, the microcomputer became popular and a need to share information between micro-
M. Hinton, P.R. Hinton Jr./Laboratory
Automation und Informdon
computers became necessary, thus the local area network (LAN) was created. On the LAN, usually a large microcomputer was designated as a LAN file server. Flat files used for common data storage were kept on the file server. In this configuration all the processing is done on the client and when many users try to access the files on the LAN file server, a network traffic bottleneck can occur. A way of filtering the data before it goes to the client workstation was needed. The relational database vendors filled this void by migrating the client-server relational model used on the minicomputers to the LAN. In contrast to the old file-server architecture previously used on the LAN, in which the clients managed the database services, in the client-server relational model, a database engine is located on the microcomputer that serves as the LAN file server and this file server becomes the database server. The database engine is responsible for integrity, security, concurrency, and query optimization, while the clients execute the application programs [3]. In the client-server relational model, the client will send a request for database information to the server where the tables reside. The client also handles any non-database application functions, for instance, the display of the received information on the screen. The server controls access to the database tables and sends the information requested to the client. This process reduces network traffic but increases the network server CPU processing load. Variations on the client-server relational model used by LIMS’ vendors include a two-tier approach and a three-tier approach. The two-tier architecture has one database server for many clients, while the three-tier architecture consists of a database server and one or more LIMS servers with attached clients. Some information systems vendors claim the threetier architecture is more advanced than the two-tier [4], and other vendors claim that the two-tier architecture is more intuitive, has less network traffic, and faster response times [51. The client-server concept has also been adopted by non-relational models and will be discussed further in the object model section. 2.4. Network-relational
hybrid
Three other types of models worth discussing are: (1) the network-relational hybrid, (2) the object-rela-
Munugement
31 (1996) 161-171
165
tional hybrid, and (3) the object model. The network-relational hybrid combines features of both models into a single database. The network component is used to describe database entities that fit easily into set relationships, while indexes are used to model data requiring random or sorted access. Since the model allows the representation of data to be made using either set relationships or indexes, the selection is made by the database designer based on efficiency and complexity of the data. When the database design is based on a thorough analysis of the information to be modeled, this hybrid performs very well. The hybrid allows SQL to be used to retrieve data, but not to update data or otherwise modify the database. 2.5. Object-relational
hybrid
Another hybrid is the object-relational model. The object model will be discussed in the next section. Many relational database vendors have added an object shell to the relational database to handle complex data. Since the underlying database is still relational, there are serious performance problems. However, BLOBS (binary large objects) can now be stored. The problem with using this model for complex data is that the relational model was not designed to manipulate this type of data. This hybrid does not support inheritance and polymorphism that are a natural part of the object database model. The object-relational model has been a stopgap measure until object database technology can live up to its promise. 2.6. Object database model The object database model is the most recent. Unlike the relational model, there is no accepted mathematical basis for this model 161. When this model reaches full maturity, it is expected to have approximately the performance of the network model and the flexibility of the relational. In this model, data and data relationships are stored together to create objects. Each object has a unique id, which is used to relate objects instead of using keyed fields as in the relational model. Usually the object id is an immutable logical id, which allows the database user to easily move objects around in a database or in the case of a distributed database environment to move
166
M. Hinfon, P.R. Hinton Jr./Laboratory
Automation and Information
objects from one node to another. It should be noted that at least one very successful object database company has a different concept of the object model and relies on physical addresses for accessing the objects using a virtual memory mapping process. The object database is inherently faster than the relational, since relationships are stored as part of the object, while in the relational model, the relationships must be recreated at run time by joining records based on a common field in various tables. Fig. 4 uses the object model for the conceptual design of the partial LIMS schema first presented in the hierarchical model. 2.61. Object client-server architecture The object model was developed around the same time that the client-server architecture became popu-
Manqemenr
31 (1996)
I61-17i
lar in the relational database. Consequently, object databases also implemented the client-server architecture in several ways. Three approaches are: (1) the server-centric or database server, (2) the client-centric or page server, and (3) the balanced client-server or object server [8,9]. 2.6.1 .I. Seruer-centric. In the server-centric model, most of the database processing is done in the server. The client sends a request to the server, the server performs all the necessary processing of the data, and sends only the required data back to the client. This does not take advantage of the power of the client workstation and creates network traffic for data requests that could be cached in the client until needed. Many relational models implement this architecture [8,9].
/
/’ :
\.
(persis)
-\
[
/
; ‘.
{persis)
;’
‘\
Fig. 4. Object Model. Reprinted from Ref. [7].
/ ‘_ (persis) .\
;
(
M. Hinton, P.R. Hinton Jr./Laboratory
Automation and Infbrmation Management
2.6.1.2. Client-centric. In the page server or clientcentric model, the server provides database pages to the client’s request and the processing of the data is done almost entirely in the client. The power of the client is harnessed in this model, but the server’s database processing power is under-utilized. Network traffic is increased with this model since all the data on the pages is passed to the client to process [8,9]. To speed the time used for each client request, the server may use multi-threaded techniques to start a thread for each client [lo]. This paged-based object model can use the underlying computer hardware, which is also paged-based, to notify the object database that a memory fault has occurred and the object is not in the client’s cache, but must be retrieved from the server [l 11. If an object is made up of other objects, they may be grouped together on the same page. This is called object level clustering and is used to minimize concurrency conflicts for page based systems [ 11I.
2.6.1.3. Object server. In the object server model, data processing occurs in both the client and the server. The server is responsible for data integrity, query optimization, lock management, logging, and recovery. The client manages the transactions. The server sends objects to the client and the client manages the objects requested from the application programs. To minimize network traffic and utilize the power of the clients and the server, caching is done on both ends; page level at the server and object level at the client [8,9]. Because the server processes the data queries, less network traffic occurs than in the client-centric model [9].
2.6.2. Object SQL As with relational databases, SQL can be used to retrieve information from an object database. Since navigation between records is direct instead of through keys, processing is usually quicker than relational queries. The SQL for object databases has been expanded to include object methods, which can be executed on the client or server in the object-server model. If object methods can only be executed on the client, then final data filtering must be performed on the client, which results in unnecessary network traffic.
31 (1996) 161-171
167
2.6.3. Object model features Features found in the relational model may also be found in the object model including: concurrency control, data integrity, recovery, transaction logging, relocation, performance monitoring, and support of large databases. Two new features found in the object database are: (1) support for versioning (an object may have more than one version in the database) and (2) long transactions (a transaction may be processed over a long period of time). Techniques for implementing some common object features are discussed in the next sections. 2.6.3.1. Referential integrity in the object model. Since the relationships are stored in the database, referential integrity is easy to achieve. Some object databases automatically delete any relationships to other objects as well as the objects themselves. Problems can occur with this approach. Sometimes this is not the desired behavior and the database application user may wish to be notified that there are still objects connected to an object he plans to delete. Rather than the application programmer specifically designing the behavior constraints for each object, the object database could notify the client that relationships still exist for the object set for deletion. If a client application can be notified of any existing relationships to other objects, the programmer can use this behavior to disallow the application user from deleting an object with attached objects unless the user first deletes the items connected. For example, in a LIMS database, a lab technician may wish to delete a particular product grade for testing a sample. He has not realized that there have already been tests performed specifically for this grade and tries to delete the grade. The LIMS could notify the user that test results are already attached to this grade and cannot be deleted. Good quality assurance practices would not allow the technician to delete a grade from the sample list unless data has not been collected for the grade. 2.6.3.2. Transparent data distribution. Some object databases implement transparent data distribution, which means that the database users do not have to be aware of where any object is located. The object will be automatically retrieved from the appropriate database. If transparency is independent of location,
168
M. Hinton, P.R. Hinton Jr./Laboratory
Automation and Information Management 31 (1996) 161-l 71
an object must have a unique network-wide logical id 191. 2.6.3.3. Schema management. The database schema describes the structure of the data in the database. It is physically part of the hierarchical, network, and object databases. In the hierarchical and network database, the schema contains the records, fields, and sets, which link the records. In an object database, it contains the objects, relationships, and methods to store the objects. Schema management is very important in a dynamic environment. The lack of flexibility in modifying the database schema was one of the reasons that the network model did not become as popular as the relational model. In the network model, the database has to be taken off-line to modify the schema, and unless it is a minor schema change, the data must be ported into a new database. In the object model, the schema is stored in the database and can be updated when needed. Some object database vendors can update a schema while the database is on-line, other vendors require the database to be taken off-line. Object databases that can modify the schema on-line have flexibility that is comparable to the relational model. 2.6.3.4. Object mapping. In the object model, transient and persistent objects are handled the same way by the application code. Because a close link between object-oriented languages and object databases exists, there is no need to write mapping code to store objects in the database [ 11,121. This is a big improvement when complex data has to be modeled. 2.6.3.5. User-defined types. Complex data cannot always be defined using the normal data types, such as numeric, alphanumeric, integer, etc. Being able to create user-defined data types in the object model, can be very beneficial in a LIMS that handles all types of data. 2.6.4. Object techniques and design tools Data and the methods used to store and retrieve the data are encapsulated into objects using objectoriented techniques, such as polymorphism and inheritance. There are numerous tools on the marketplace to help create a graphical object design. These tools are useful for designing object-oriented pro-
grams and developing the object schema for object databases. Since abstract concepts are easier to describe using objects, a graphical representation of the object model is very helpful for describing complex data relationships. Fig. 4 presented an example of the object model and was developed using Booth’s object-oriented notation with the Rational Rose design tool [ 131. A person familiar with this object-oriented design notation can tell at a glance what types of relationships are described. Without using a graphical object model, the descriptions of objects would fill many text pages and the details of the design would be much harder to grasp.
3. Databases for complex data It is recognized that complex data cannot always be modeled using relational tables. In the banking industry, there has been a reluctance to change their information systems from the proven safety of the hierarchical and batch-transaction oriented databases to the slower and less reliable relational model. Currently this industry is investigating the feasibility of the object paradigm. One bank has converted to an object database model for describing the complex relationships of derivatives trading [ 141. The LIMS industry also needs to investigate using object databases to expand LIMS capabilities to include complex data generated in laboratories and to formulate objects that better describe the information flow in laboratories.
4. Examples of complex scientific data Scientific data and the information flow in laboratories are even more complex than business data and harder to map to relational tables. Examples of scientific data can include variable length arrays, multidimensional data with implicit relationships among the data elements, BLOBS, and hyperslabs. Blobs may include video, graphical, or other types of unstructured data. A hyperslab is a piece of a multidimensional array described by the indices of a comer and the edge lengths of each dimension of the variable [ 1.51.One of the dimensions may be unbounded. A
M. Hinton. P.R. Hinton Jr./Laboratory
Automation and Information Management
chromatographic sequence run is an example of multidimensional data that needs to be modeled by a LIMS. A sequence run may consist of one or more standard injections, sample injections, controls and blanks.
5. Standardization
of scientific
data formats
Since chromatographic data is too complex to fit well in tables, the Analytical Instrument Association (AIA) adopted the Network Common Data form (NetCDF) format to store chromatographic data in files. The AIA format is called the AND1 (analytical data interchange) standard [16]. The AIA uses variations of this format for other types of data acquisition including FI-IR and mass spectrometry. Most data acquisition systems vendors are supporting the AND1 format and LIMS vendors are also adding support for data stored in this format.
6. Modeling the information
flow through a labo-
ratory The information flow to, from, and within a laboratory is very complex. The laboratory constantly sends and receives information to and from other departments in a company. This could be process control, shipping, or inventory in a manufacturing plant; billing or receiving in a testing services laboratory. How the laboratory interacts with these and other departments and what type of information is required between departments needs to be modeled. Within the laboratory itself, the information generated is used for better resource planning and improved data results. For example, the number of data requests and the completion status of samples can be used for scheduling resources. Data results can be improved with the use of quality control charts. Information may also flow from one test to another. Examples are: (1) dam results from one test could be used as input into another test method, and (2) standards can be prepared that are used in many different tests. Information about sample groups may need to be tracked. This would not be information used outside the laboratory, but kept for quality assurance purposes within the laboratory. For in-
31 (1996) 161-171
169
stance, samples tested as a batch may need to be tracked as part of a batch method so that if for any reason a standard used in the batch calculations had to be reevaluated, the results for every sample in the batch could be recalculated. A LIMS should model all changes that occur in the laboratory. Test method revisions and specification revisions are examples of changes frequently made in a laboratory that need to be tracked. Data should be reported according to the specifications in effect at the time the results were recorded, and if information about a sample previously tested is required, the LIMS needs to be able to provide the specifications in effect at the time the sample was tested [ 171. The information used in a laboratory also includes the raw data. All raw data should be stored in the LIMS. The number of sample repetitions, the raw data for each repetition, any standards, blanks, or control values should be stored along with the final results. Often the number of repetitions for a sample is variable. This is especially true for process samples (work-in-progress samples). The LIMS should handle variable sample runs. The batch itself may have a variable number of samples, since a batch is usually prepared from all the samples that are ready to go at a particular time. For a LIMS to manage the information flow within and through a laboratory, all the above mentioned information flows should be modeled in objects and relationships that can be stored in the LIMS database.
7. LIMS vendors’ solutions Since the LIMS database must manage complex scientific data and convoluted information flows through the laboratory, LIMS vendors are beginning to recognize the drawbacks of the relational model for complex data management. Some LIMS vendors are using their own proprietary databases for performance reasons, while others are using database engines supplied from third parties in the form of software libraries 1181. With both approaches, the LIMS developer has low and high level control of database functions, which he can use to boost performance. When the database library is part of the
170
M. Hinton. P.R. Hinton Jr./Laborutory
Automation and Information Management
LIMS, an added advantage is that a separate database does not have to be purchased. Although most LIMS vendors are converting their relational database to the relational-object hybrids, a few vendors have decided to use a database hybrid that is a combination of the faster performing network model and the relational model. An object interface can be added for flexibility and data encapsulation to this hybrid. Others are experimenting with the newer object databases but recognize that object technology will not replace relational databases any time soon [ 141. On the whole, the LIMS industry is moving toward the object paradigm and is preparing the groundwork to replace relational databases with object databases when the object design is more established.
8. Data exchange
between
databases
across the
enterprise
Companies are attempting to provide an enterprise-wide data solution. One way to accomplish this is to insist on the use of the same database vendor for all applications on the enterprise network, the other is to have data exchange methods for the various databases on the network. When more than one database is used on a network, standards are needed for data exchange between databases. The first and still widely-used format for data exchange is the ASCII format. The ASCII format is readable by most text editors and programs can easily be developed to import or export data using this format. Another format developed by Microsoft so that relational data can be more efficiently transported across the enterprise network from database to database is the ODBC (open database connectivity) standard. Most relational databases used on a LAN are either ODBC-compliant or provide an ODBC driver. A few object databases have also added an ODBC feature. The newest Microsoft standard is OLE DB (object linking and embedding). It is due out next year and is expected to support the transfer of relational and non-relational data across the enterprise [19]. COBRA (common object request broker architecture) is a standard backed by a consortium of 500 companies called the Object Management Group [20].
31 (1996) 161-171
It enables objects to be shared and reused. It is more common in the Unix and OS/2 environments [21].
9. Conclusions Data exchange standards for a range of data types, including spreadsheet data, flat files, relational data, and object data, are still being formulated. Acceptable standards for data exchange across the enterprise will help the LIMS vendor. Often a LIMS database is different than the enterprise database in order to handle complex data. Much of the laboratory data (for example, raw data, chromatograms, binary images, etc.> stored in a LIMS database is not necessary for enterprise-wide distribution. As long as the information important to the enterprise can be exported to the enterprise database, using a different database should not be a problem. To capture the dynamic environment of a laboratory, the object database is the most likely paradigm for the LIMS of the future. An object database will enhance the capabilities of a LIMS, but more than a new database model is required. To realize the benefits of this model, a change in mindset to object technology is necessary [20]. Current LIMS cannot satisfy all the needs of laboratories. However, the full potential of a LIMS could be realized with object technology. This technology includes an object design, working with an object language and an object database. This mindset must be changed from the older established software development process to the newer object methodology. The new object-oriented software development cycle is an iterative process, which stresses continual feedback from one stage of development to another [13]. The conceptual design of the information system merges with the actual object structure of the database because the analysis and design phase of software development can use tools and methodology that support object programming and object databases [6]. Unlike the relational where the conceptual model must be mapped to the relational model, the conceptual object design does not have to be mapped into the object database but becomes the schema for the object database.
M. Hinton, P.R. Hinton Jr./Laboratory
Automation and Information
References HI M. Hinton, Laboratory Information Management Systems: Development and Implementation for a Quality Assurance Laboratory, Marcel Dckker, New York, NY, 1995, pp. 217225. Dl P.K. Andleigh and M.R. Grctzinger, Distributed Object-Oriented Data-Systems Design, Prentice Hall, Englewood Cliffs, NJ, pp. 23-54. [31 S. Khoshafian, Object-Oriented Databases, Wiley, New York, NY, 1993, pp. 23-26. [41 M. Brando. Am. Lab., March (1995) 44. July 3 (1995) 80. [51 D. Bartholomew, InformationWeek, Programming, February [61 M.E.S. Loomis, J. Object-Oriented (1990) 79. Manual, Ap[71P. Hinton, ACS-LIMS Design Specifications plied Computer Systems, Mobile, AL, Company Literature, 1995. [81 M.E.S. Loomis, J. Object-Oriented Programming, February (1992) 40. 191Versant, How to Evaluate Object Database Management Systems, Versant Object Technology Corporation, Menlo Park, CA, Company Literature, 1992.
Management
31 (1996) 161-171
[lOI 1. Blake, Adv. Syst., April (1995). ill1 Object Design, Inc., Technical Overview,
171
Object Design, Inc., Burlington, MA, Company Literature, 1994. 1121 Versant, Versant in Network Management, Versant Object Technology Corporation, Menlo Park, CA, Company Literature, 1993. [131G. Booth, Object Oriented Design with Applications, Benjamin/Cummings, Menlo Park, CA, 1991, pp. 198-206. July 17 (1995) 41. 1141F. Hayes, InformationWeek, 1151R. Rew and G. Davis, IEEE Comput. Graphics Appl., July (1990) 76. [161Analytical Instrument Association, AND1 - Analytical Data Interchange Chromatography Data, Alexandria, VA, 1992, pp. 4-18. t171 M. Hinton, Am. Lab., October (1992) 31. 1181 R. Grehan, B. Smith and J. Udell, BYTE, January (19921 204. May 29 (1995) 74. [191 K. Bull, InformationWeek, August 14 (1995) 86. t201 J. Cash, InformationWeek, August 14 (1995) 70. 1211 F. Hayes, InformationWeek,
Laboratory automation and information management Laboratory Automation and Information Management 3 1( 1996) 161- I7 1
Tutorial
The LIMS database Mary Hinton, Phillip R. Hinton Jr. Applied Computer Systems, 3540 Country Ct. N., Mobile, AL 36619. USA
Received 15 September 1995; accepted 3 October 1995
Abstract A LIMS amounts of implement advantages fits in with
is one of the many information management systems on the market today that manages complex and massive data. A history of database technology is presented, followed by a look at the current database models used to a LIMS. How successful each model is in handling complex scientific data is discussed, along with the and disadvantages of each model. The LIMS database model of the future is projected. How the LIMS database the overall enterprise-wide data management design is also examined.
LIMS; Database; Model; Design; Management
Keywords:
Contents ......................................................... 2. Historical use of databases. ................................................. 2.1. Hierarchical model ................................................... 2.2. Network model. ..................................................... 2.3. Relational model. .................................................... 2.3.1. Client-server relational model ........................................ Network-relational hybrid ............................................... 2.4. Object-relational hybrid ................................................ 2.5. 2.6. Object database model ................................................. 2.6.1. Object client-server architecture ....................................... 2.6.1.1. Semer-centric ............................................. 2.6.1.2. Client-centric ............................................. 2.6.1.3. Object server .................................
162 162 162
1. Introduction.
2.6.2. ObjectSQL
....................................................
............................................. ............................. 2.6.3.2. Transparent data distribution .................................... ........................................ 2.6.3.3. Schema management. 2.6.3.4. Object mapping. ...........................................
2.6.3. Object model features 2.6.3.1. Referential
integrity in the object model
092%5281/96/$15.00 0 1996 Elsevier Science B.V. All rights resewed SSDI 0925-5281(95)00023-2
163 163
164 165
165 165 166 166 167
, ...........
167 167 167 167 167 168 168
M. Hinton, P.R. Hinton Jr./Laboratory
162
Automation and Information Management 31 (1996) 161-171
2.6.3.5. User-defined .types .......................................... 2.6.4. Object techniques and design tools. ..................................... Databases for complex data ................................................. Examples of complex scientific data ............................................ Standardization of scientific data formats ......................................... Modeling the information flow through a laboratory .................................. LIMS vendors’ solutions ................................................... Data exchange between databases across the enterprise. ................................
3. 4. 5. 6. 7. 8. 9. Conclusions ........................................................... References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1. Introduction A LIMS is one of the many information management systems on the market today that manages complex and massive amounts of data. A common goal for all information management systems is the development of a data management paradigm for the 21st century. The core of any LIMS is the database. With an ever-increasing load placed on the LIMS, there has been a dramatic slowdown in the information flow in some laboratories. Even when computers that run the LIMS software are replaced with faster computers, inevitably the LIMS begins to bog down again as the database grows in size and complexity. Most vendors are using the relational model as the underlying database. LIMS and other information management system vendors are taking a second look at this model for complex data management.
2. Historical
168 168 168 168 169 169 169 170 170 171
between the parent and child node is called a set with the parent record being the owner of the set and the child records being members of the set. This structure has built-in integrity because of the use of linked lists to define the set relationships. Although the hierarchical model lacks the flexibility to add new data relationships while the database is on-line, it is still the database of choice for many large information management systems. Fig. 1 is an example of the hierarchical model. In this diagram, it models some of the relationships that may exist between entities found in a LIMS for a manufacturing laboratory. The relationships are represented by the branches linking the parent and child nodes. To implement the hierarchical model, a child node can be the parent of one or more nodes, but each child node can have only one parent. In other words, a child node can be a member of only one set, but can be the owner of one or more sets [l].
use of databases
To explain the latest database technology available for LIMS, a brief history and description of the various database models would be appropriate. 2.1. Hierarchical model The hierarchical model was the first widely used standard database model. This model can be visualized as a tree structure containing parent and child nodes. The topmost level has one parent called the root node from which child nbdes can descend; each child in turn, can have descendants. The relationship
Fig. 1. Hierarchical Model. Reprinted courtesy of Marcel Dekker, Inc.
from Ref. Ill, p. 218 by
M. Hinton, P.R. Hinton Jr./Laboratory
Automation
Because of the restrictions placed on parent-child relationships, some records in Fig. 1 must be duplicated, such as Plant and Grade. Redundancy is common with the hierarchical model.
and Information Management 31 (1996) 161-I 71
163
simultaneously, duplicate Plant and Grade records have been eliminated [l]. 2.3. Relational model
2.2. Network model Next came the network database model, which is also based on set relationships. It is actually a superset of the hierarchical model as it allows child nodes to descend from more than one parent as long as each parent is in a different set (relationship). The network model as the name implies has a network structure of parent and child records linked together by set relationships. A child can be owned by more than one parent and it can even own itself. The network like the hierarchical is very fast because of its reliance on linked list database pointers to navigate directly to data described by set relationships. Since each record contains the address of related records, navigating through the database from one record to a related record is fast, even as the database grows large. As with the hierarchical model, changes to data records or data relationships must occur off-line because of the embedded database addresses used for direct navigation. Because of the nature of the set relationships, referential integrity is automatically assured. Fig. 2 uses the same entities as presented for the hierarchical model, except this time the network model is implemented. In this representation, there is no need for redundant information. Since the Plant entity can now be owned by Customer and Grade
Fig. 2. Network Model. Reprinted from Ref. [I], p. 219 by courtesy of Marcel Dekker, Inc.
The relational model followed soon after. This model is much easier to implement for data that can be described in simple tabular format (as found in many business applications). It does not build on the hierarchical or network concepts, but instead is based on set theory (specifically, relational algebra). The relational model can be visualized as tables containing rows of data records. The columns are the fields or attributes of each data record. Each record has a field or set of fields that serves as a primary key that uniquely identifies each row. Relationships are implicitly described by using a field that is common between two tables as a key for accessing the related records in the tables 121. Data is manipulated using SQL (structured query language), which has set operations, such as selection, join, union, intersection, product, and projection. This model is more flexible than the hierarchical or network model. In a hierarchical or network database, record definitions are fixed in order to embed linked list data pointers in the records. Since the relational model uses keyed fields to relate records, record definitions may be easily changed and new records created. However, this flexibility comes with an integrity and performance price. To ensure integrity, user-defined integrity constraints must be established and checked before processing transactions. In the hierarchical or network model, the constraints are automatically established when the sets are implemented, while in the relational model, the constraints are user-defined in the application program. Since it is so easy to get the system up and running, the design of the user-integrity constraints may not be well thought out in advance and as the database grows in size and complexity, more tables are often added to the system, and therefore more joins are needed to access information in the database. (A join is one of the set operators of relational algebra used to concatenate data found in different tables.) The more joins needed to process a query, the slower the system responds. Another disadvantage for some information manage-
M. Hinton, P.R. Hinton Jr./Laboratory
164
Automation andlnformation
I
Plant ID
*CustomerID 4
Plant Name Plant Address
Plmt-Omde Table
*Rimrum Fig. 3. Relational
l‘S-d=YKy
Model.
ment systems based on the relational model is that complex data cannot be adequately described in simple rows and columns. In the LIMS environment, using the relational model to describe complex data, it is not uncommon to do 10 or more record joins to process a single information request. Despite its disadvantages, this model requires less initial planning and design to implement and is the quickest and easiest of the models to set up. It soon became the most recognized and widely accepted database model for microcomputers and minicomputers. The relational model became the panacea for every information storage problem. Whenever changes or additions to a database were needed, new tables were added or existing tables extended, often without determining the performance impact on the as information systems system. Consequently, changed over time, the performance often degraded dramatically. To implement the relational model for the same partial LIMS schema presented in Fig. 1 for the hierarchical model, tables would need to be created for each entity relationship in the schema. Fig. 3 shows the mapping required in the relational model for the Customer-Plant and Plant-Grade relationships of the LIMS schema, where each customer (company) has one or more plant sites and each plant purchases one or more grades of products. A table is created for the Customer, which includes fields for Customer ID and Customer Name. A table is also created for the Plant, which contains the Customer
Management 31 (1996) 161-171
ID, Plant ID, Plant Name, and Plant Address. In this situation, the Customer ID would be the primary key in the customer table, but the secondary key in the plant table. It links the information contained within the two tables and implicitly forms a relationship between the two entities. To describe the Plant-Grade relationship, Grade and Plant-Grade tables are added. Instead of direct navigation as described in the hierarchical and network models, the relational model requires joins for the Customer, Plant, and Grade tables to find out which Customer purchases a particular Product-Grade. Fig. 3 shows that in order to navigate to a related record, the secondary key is used to find the primary key in an index file, which contains a sorted list of keys and physical addresses for each table record. Multiple database accesses are usually required to locate one record in an index, and as the table grows larger, the more database accesses are required. This is why relational databases slow down as they get larger. Unlike the hierarchical and network databases, which depend on the physical structure of data, a relational database provides data independence, which means that the physical and logical structures are separated so that the user applications do not have to worry about the actual physical address of data. Other features of relational databases include concurrency, recovery, transaction, and an ad hoc query facility. 2.3.1. Client-server relational model Originally all database files were stored on mainframes or minicomputers. Since dumb terminals were used to access the host computer, all processing occurred on the host. Relational databases were developed in this environment, in which the database server resided on the host and the application programs were processes on the host. When a large number of users needed access to the database information simultaneously, a bottleneck occurred on the host CPU. Dumb terminals were eventually replaced with powerful microcomputer workstations and the client-server architecture was developed to take advantage of the power of the workstation. The application processes were now run on the clients, while the database server remained on the host. In the meantime, the microcomputer became popular and a need to share information between micro-
M. Hinton, P.R. Hinton Jr./Laboratory
Automation und Informdon
computers became necessary, thus the local area network (LAN) was created. On the LAN, usually a large microcomputer was designated as a LAN file server. Flat files used for common data storage were kept on the file server. In this configuration all the processing is done on the client and when many users try to access the files on the LAN file server, a network traffic bottleneck can occur. A way of filtering the data before it goes to the client workstation was needed. The relational database vendors filled this void by migrating the client-server relational model used on the minicomputers to the LAN. In contrast to the old file-server architecture previously used on the LAN, in which the clients managed the database services, in the client-server relational model, a database engine is located on the microcomputer that serves as the LAN file server and this file server becomes the database server. The database engine is responsible for integrity, security, concurrency, and query optimization, while the clients execute the application programs [3]. In the client-server relational model, the client will send a request for database information to the server where the tables reside. The client also handles any non-database application functions, for instance, the display of the received information on the screen. The server controls access to the database tables and sends the information requested to the client. This process reduces network traffic but increases the network server CPU processing load. Variations on the client-server relational model used by LIMS’ vendors include a two-tier approach and a three-tier approach. The two-tier architecture has one database server for many clients, while the three-tier architecture consists of a database server and one or more LIMS servers with attached clients. Some information systems vendors claim the threetier architecture is more advanced than the two-tier [4], and other vendors claim that the two-tier architecture is more intuitive, has less network traffic, and faster response times [51. The client-server concept has also been adopted by non-relational models and will be discussed further in the object model section. 2.4. Network-relational
hybrid
Three other types of models worth discussing are: (1) the network-relational hybrid, (2) the object-rela-
Munugement
31 (1996) 161-171
165
tional hybrid, and (3) the object model. The network-relational hybrid combines features of both models into a single database. The network component is used to describe database entities that fit easily into set relationships, while indexes are used to model data requiring random or sorted access. Since the model allows the representation of data to be made using either set relationships or indexes, the selection is made by the database designer based on efficiency and complexity of the data. When the database design is based on a thorough analysis of the information to be modeled, this hybrid performs very well. The hybrid allows SQL to be used to retrieve data, but not to update data or otherwise modify the database. 2.5. Object-relational
hybrid
Another hybrid is the object-relational model. The object model will be discussed in the next section. Many relational database vendors have added an object shell to the relational database to handle complex data. Since the underlying database is still relational, there are serious performance problems. However, BLOBS (binary large objects) can now be stored. The problem with using this model for complex data is that the relational model was not designed to manipulate this type of data. This hybrid does not support inheritance and polymorphism that are a natural part of the object database model. The object-relational model has been a stopgap measure until object database technology can live up to its promise. 2.6. Object database model The object database model is the most recent. Unlike the relational model, there is no accepted mathematical basis for this model 161. When this model reaches full maturity, it is expected to have approximately the performance of the network model and the flexibility of the relational. In this model, data and data relationships are stored together to create objects. Each object has a unique id, which is used to relate objects instead of using keyed fields as in the relational model. Usually the object id is an immutable logical id, which allows the database user to easily move objects around in a database or in the case of a distributed database environment to move
166
M. Hinfon, P.R. Hinton Jr./Laboratory
Automation and Information
objects from one node to another. It should be noted that at least one very successful object database company has a different concept of the object model and relies on physical addresses for accessing the objects using a virtual memory mapping process. The object database is inherently faster than the relational, since relationships are stored as part of the object, while in the relational model, the relationships must be recreated at run time by joining records based on a common field in various tables. Fig. 4 uses the object model for the conceptual design of the partial LIMS schema first presented in the hierarchical model. 2.61. Object client-server architecture The object model was developed around the same time that the client-server architecture became popu-
Manqemenr
31 (1996)
I61-17i
lar in the relational database. Consequently, object databases also implemented the client-server architecture in several ways. Three approaches are: (1) the server-centric or database server, (2) the client-centric or page server, and (3) the balanced client-server or object server [8,9]. 2.6.1 .I. Seruer-centric. In the server-centric model, most of the database processing is done in the server. The client sends a request to the server, the server performs all the necessary processing of the data, and sends only the required data back to the client. This does not take advantage of the power of the client workstation and creates network traffic for data requests that could be cached in the client until needed. Many relational models implement this architecture [8,9].
/
/’ :
\.
(persis)
-\
[
/
; ‘.
{persis)
;’
‘\
Fig. 4. Object Model. Reprinted from Ref. [7].
/ ‘_ (persis) .\
;
(
M. Hinton, P.R. Hinton Jr./Laboratory
Automation and Infbrmation Management
2.6.1.2. Client-centric. In the page server or clientcentric model, the server provides database pages to the client’s request and the processing of the data is done almost entirely in the client. The power of the client is harnessed in this model, but the server’s database processing power is under-utilized. Network traffic is increased with this model since all the data on the pages is passed to the client to process [8,9]. To speed the time used for each client request, the server may use multi-threaded techniques to start a thread for each client [lo]. This paged-based object model can use the underlying computer hardware, which is also paged-based, to notify the object database that a memory fault has occurred and the object is not in the client’s cache, but must be retrieved from the server [l 11. If an object is made up of other objects, they may be grouped together on the same page. This is called object level clustering and is used to minimize concurrency conflicts for page based systems [ 11I.
2.6.1.3. Object server. In the object server model, data processing occurs in both the client and the server. The server is responsible for data integrity, query optimization, lock management, logging, and recovery. The client manages the transactions. The server sends objects to the client and the client manages the objects requested from the application programs. To minimize network traffic and utilize the power of the clients and the server, caching is done on both ends; page level at the server and object level at the client [8,9]. Because the server processes the data queries, less network traffic occurs than in the client-centric model [9].
2.6.2. Object SQL As with relational databases, SQL can be used to retrieve information from an object database. Since navigation between records is direct instead of through keys, processing is usually quicker than relational queries. The SQL for object databases has been expanded to include object methods, which can be executed on the client or server in the object-server model. If object methods can only be executed on the client, then final data filtering must be performed on the client, which results in unnecessary network traffic.
31 (1996) 161-171
167
2.6.3. Object model features Features found in the relational model may also be found in the object model including: concurrency control, data integrity, recovery, transaction logging, relocation, performance monitoring, and support of large databases. Two new features found in the object database are: (1) support for versioning (an object may have more than one version in the database) and (2) long transactions (a transaction may be processed over a long period of time). Techniques for implementing some common object features are discussed in the next sections. 2.6.3.1. Referential integrity in the object model. Since the relationships are stored in the database, referential integrity is easy to achieve. Some object databases automatically delete any relationships to other objects as well as the objects themselves. Problems can occur with this approach. Sometimes this is not the desired behavior and the database application user may wish to be notified that there are still objects connected to an object he plans to delete. Rather than the application programmer specifically designing the behavior constraints for each object, the object database could notify the client that relationships still exist for the object set for deletion. If a client application can be notified of any existing relationships to other objects, the programmer can use this behavior to disallow the application user from deleting an object with attached objects unless the user first deletes the items connected. For example, in a LIMS database, a lab technician may wish to delete a particular product grade for testing a sample. He has not realized that there have already been tests performed specifically for this grade and tries to delete the grade. The LIMS could notify the user that test results are already attached to this grade and cannot be deleted. Good quality assurance practices would not allow the technician to delete a grade from the sample list unless data has not been collected for the grade. 2.6.3.2. Transparent data distribution. Some object databases implement transparent data distribution, which means that the database users do not have to be aware of where any object is located. The object will be automatically retrieved from the appropriate database. If transparency is independent of location,
168
M. Hinton, P.R. Hinton Jr./Laboratory
Automation and Information Management 31 (1996) 161-l 71
an object must have a unique network-wide logical id 191. 2.6.3.3. Schema management. The database schema describes the structure of the data in the database. It is physically part of the hierarchical, network, and object databases. In the hierarchical and network database, the schema contains the records, fields, and sets, which link the records. In an object database, it contains the objects, relationships, and methods to store the objects. Schema management is very important in a dynamic environment. The lack of flexibility in modifying the database schema was one of the reasons that the network model did not become as popular as the relational model. In the network model, the database has to be taken off-line to modify the schema, and unless it is a minor schema change, the data must be ported into a new database. In the object model, the schema is stored in the database and can be updated when needed. Some object database vendors can update a schema while the database is on-line, other vendors require the database to be taken off-line. Object databases that can modify the schema on-line have flexibility that is comparable to the relational model. 2.6.3.4. Object mapping. In the object model, transient and persistent objects are handled the same way by the application code. Because a close link between object-oriented languages and object databases exists, there is no need to write mapping code to store objects in the database [ 11,121. This is a big improvement when complex data has to be modeled. 2.6.3.5. User-defined types. Complex data cannot always be defined using the normal data types, such as numeric, alphanumeric, integer, etc. Being able to create user-defined data types in the object model, can be very beneficial in a LIMS that handles all types of data. 2.6.4. Object techniques and design tools Data and the methods used to store and retrieve the data are encapsulated into objects using objectoriented techniques, such as polymorphism and inheritance. There are numerous tools on the marketplace to help create a graphical object design. These tools are useful for designing object-oriented pro-
grams and developing the object schema for object databases. Since abstract concepts are easier to describe using objects, a graphical representation of the object model is very helpful for describing complex data relationships. Fig. 4 presented an example of the object model and was developed using Booth’s object-oriented notation with the Rational Rose design tool [ 131. A person familiar with this object-oriented design notation can tell at a glance what types of relationships are described. Without using a graphical object model, the descriptions of objects would fill many text pages and the details of the design would be much harder to grasp.
3. Databases for complex data It is recognized that complex data cannot always be modeled using relational tables. In the banking industry, there has been a reluctance to change their information systems from the proven safety of the hierarchical and batch-transaction oriented databases to the slower and less reliable relational model. Currently this industry is investigating the feasibility of the object paradigm. One bank has converted to an object database model for describing the complex relationships of derivatives trading [ 141. The LIMS industry also needs to investigate using object databases to expand LIMS capabilities to include complex data generated in laboratories and to formulate objects that better describe the information flow in laboratories.
4. Examples of complex scientific data Scientific data and the information flow in laboratories are even more complex than business data and harder to map to relational tables. Examples of scientific data can include variable length arrays, multidimensional data with implicit relationships among the data elements, BLOBS, and hyperslabs. Blobs may include video, graphical, or other types of unstructured data. A hyperslab is a piece of a multidimensional array described by the indices of a comer and the edge lengths of each dimension of the variable [ 1.51.One of the dimensions may be unbounded. A
M. Hinton. P.R. Hinton Jr./Laboratory
Automation and Information Management
chromatographic sequence run is an example of multidimensional data that needs to be modeled by a LIMS. A sequence run may consist of one or more standard injections, sample injections, controls and blanks.
5. Standardization
of scientific
data formats
Since chromatographic data is too complex to fit well in tables, the Analytical Instrument Association (AIA) adopted the Network Common Data form (NetCDF) format to store chromatographic data in files. The AIA format is called the AND1 (analytical data interchange) standard [16]. The AIA uses variations of this format for other types of data acquisition including FI-IR and mass spectrometry. Most data acquisition systems vendors are supporting the AND1 format and LIMS vendors are also adding support for data stored in this format.
6. Modeling the information
flow through a labo-
ratory The information flow to, from, and within a laboratory is very complex. The laboratory constantly sends and receives information to and from other departments in a company. This could be process control, shipping, or inventory in a manufacturing plant; billing or receiving in a testing services laboratory. How the laboratory interacts with these and other departments and what type of information is required between departments needs to be modeled. Within the laboratory itself, the information generated is used for better resource planning and improved data results. For example, the number of data requests and the completion status of samples can be used for scheduling resources. Data results can be improved with the use of quality control charts. Information may also flow from one test to another. Examples are: (1) dam results from one test could be used as input into another test method, and (2) standards can be prepared that are used in many different tests. Information about sample groups may need to be tracked. This would not be information used outside the laboratory, but kept for quality assurance purposes within the laboratory. For in-
31 (1996) 161-171
169
stance, samples tested as a batch may need to be tracked as part of a batch method so that if for any reason a standard used in the batch calculations had to be reevaluated, the results for every sample in the batch could be recalculated. A LIMS should model all changes that occur in the laboratory. Test method revisions and specification revisions are examples of changes frequently made in a laboratory that need to be tracked. Data should be reported according to the specifications in effect at the time the results were recorded, and if information about a sample previously tested is required, the LIMS needs to be able to provide the specifications in effect at the time the sample was tested [ 171. The information used in a laboratory also includes the raw data. All raw data should be stored in the LIMS. The number of sample repetitions, the raw data for each repetition, any standards, blanks, or control values should be stored along with the final results. Often the number of repetitions for a sample is variable. This is especially true for process samples (work-in-progress samples). The LIMS should handle variable sample runs. The batch itself may have a variable number of samples, since a batch is usually prepared from all the samples that are ready to go at a particular time. For a LIMS to manage the information flow within and through a laboratory, all the above mentioned information flows should be modeled in objects and relationships that can be stored in the LIMS database.
7. LIMS vendors’ solutions Since the LIMS database must manage complex scientific data and convoluted information flows through the laboratory, LIMS vendors are beginning to recognize the drawbacks of the relational model for complex data management. Some LIMS vendors are using their own proprietary databases for performance reasons, while others are using database engines supplied from third parties in the form of software libraries 1181. With both approaches, the LIMS developer has low and high level control of database functions, which he can use to boost performance. When the database library is part of the
170
M. Hinton. P.R. Hinton Jr./Laborutory
Automation and Information Management
LIMS, an added advantage is that a separate database does not have to be purchased. Although most LIMS vendors are converting their relational database to the relational-object hybrids, a few vendors have decided to use a database hybrid that is a combination of the faster performing network model and the relational model. An object interface can be added for flexibility and data encapsulation to this hybrid. Others are experimenting with the newer object databases but recognize that object technology will not replace relational databases any time soon [ 141. On the whole, the LIMS industry is moving toward the object paradigm and is preparing the groundwork to replace relational databases with object databases when the object design is more established.
8. Data exchange
between
databases
across the
enterprise
Companies are attempting to provide an enterprise-wide data solution. One way to accomplish this is to insist on the use of the same database vendor for all applications on the enterprise network, the other is to have data exchange methods for the various databases on the network. When more than one database is used on a network, standards are needed for data exchange between databases. The first and still widely-used format for data exchange is the ASCII format. The ASCII format is readable by most text editors and programs can easily be developed to import or export data using this format. Another format developed by Microsoft so that relational data can be more efficiently transported across the enterprise network from database to database is the ODBC (open database connectivity) standard. Most relational databases used on a LAN are either ODBC-compliant or provide an ODBC driver. A few object databases have also added an ODBC feature. The newest Microsoft standard is OLE DB (object linking and embedding). It is due out next year and is expected to support the transfer of relational and non-relational data across the enterprise [19]. COBRA (common object request broker architecture) is a standard backed by a consortium of 500 companies called the Object Management Group [20].
31 (1996) 161-171
It enables objects to be shared and reused. It is more common in the Unix and OS/2 environments [21].
9. Conclusions Data exchange standards for a range of data types, including spreadsheet data, flat files, relational data, and object data, are still being formulated. Acceptable standards for data exchange across the enterprise will help the LIMS vendor. Often a LIMS database is different than the enterprise database in order to handle complex data. Much of the laboratory data (for example, raw data, chromatograms, binary images, etc.> stored in a LIMS database is not necessary for enterprise-wide distribution. As long as the information important to the enterprise can be exported to the enterprise database, using a different database should not be a problem. To capture the dynamic environment of a laboratory, the object database is the most likely paradigm for the LIMS of the future. An object database will enhance the capabilities of a LIMS, but more than a new database model is required. To realize the benefits of this model, a change in mindset to object technology is necessary [20]. Current LIMS cannot satisfy all the needs of laboratories. However, the full potential of a LIMS could be realized with object technology. This technology includes an object design, working with an object language and an object database. This mindset must be changed from the older established software development process to the newer object methodology. The new object-oriented software development cycle is an iterative process, which stresses continual feedback from one stage of development to another [13]. The conceptual design of the information system merges with the actual object structure of the database because the analysis and design phase of software development can use tools and methodology that support object programming and object databases [6]. Unlike the relational where the conceptual model must be mapped to the relational model, the conceptual object design does not have to be mapped into the object database but becomes the schema for the object database.
M. Hinton, P.R. Hinton Jr./Laboratory
Automation and Information
References HI M. Hinton, Laboratory Information Management Systems: Development and Implementation for a Quality Assurance Laboratory, Marcel Dckker, New York, NY, 1995, pp. 217225. Dl P.K. Andleigh and M.R. Grctzinger, Distributed Object-Oriented Data-Systems Design, Prentice Hall, Englewood Cliffs, NJ, pp. 23-54. [31 S. Khoshafian, Object-Oriented Databases, Wiley, New York, NY, 1993, pp. 23-26. [41 M. Brando. Am. Lab., March (1995) 44. July 3 (1995) 80. [51 D. Bartholomew, InformationWeek, Programming, February [61 M.E.S. Loomis, J. Object-Oriented (1990) 79. Manual, Ap[71P. Hinton, ACS-LIMS Design Specifications plied Computer Systems, Mobile, AL, Company Literature, 1995. [81 M.E.S. Loomis, J. Object-Oriented Programming, February (1992) 40. 191Versant, How to Evaluate Object Database Management Systems, Versant Object Technology Corporation, Menlo Park, CA, Company Literature, 1992.
Management
31 (1996) 161-171
[lOI 1. Blake, Adv. Syst., April (1995). ill1 Object Design, Inc., Technical Overview,
171
Object Design, Inc., Burlington, MA, Company Literature, 1994. 1121 Versant, Versant in Network Management, Versant Object Technology Corporation, Menlo Park, CA, Company Literature, 1993. [131G. Booth, Object Oriented Design with Applications, Benjamin/Cummings, Menlo Park, CA, 1991, pp. 198-206. July 17 (1995) 41. 1141F. Hayes, InformationWeek, 1151R. Rew and G. Davis, IEEE Comput. Graphics Appl., July (1990) 76. [161Analytical Instrument Association, AND1 - Analytical Data Interchange Chromatography Data, Alexandria, VA, 1992, pp. 4-18. t171 M. Hinton, Am. Lab., October (1992) 31. 1181 R. Grehan, B. Smith and J. Udell, BYTE, January (19921 204. May 29 (1995) 74. [191 K. Bull, InformationWeek, August 14 (1995) 86. t201 J. Cash, InformationWeek, August 14 (1995) 70. 1211 F. Hayes, InformationWeek,