- Email: [email protected]
Organization of a knowledge base by Q-analysis
Organization
of a Knowledge Base by Q-Analysis
L. Duckstein
Systems Engineering Department Case Western Reserve University Cleveland, Ohio 44106 P. H. Bartels Optical Sciences Center and Department The University of Arimnu Tucson, Arimnu 85721
of Pathology
and J. E. Weber
Department of Statistics The University of Arizona Tucson, Arizona 85721
ABSTRACT Q-analysis is used in organizing a histopathological knowledge base which is a component of the diagnostic expert system at the University of Arizona. This expert system has three subsystems or modules: the first module guides the dynamic reconfiguration of the computer system; the second module guides scene decomposition and the extraction of diagnostic information; the third module uses a rule-based system to obtain a diagnostic assessment. A specific example of the usefulness of Q-analysis is given in the context of the third module; the data represent expert opinion concerning diagnosis of cobnic cancer and are summarized in a matrix representing four diagnostic categories and nineteen diagnostic clues. The example shows that Q-analysis may be helpful to diagnosticians in defining and explaining the process they use in arriving at a diagnosis and in using this information as a basis for structuring the knowledge base.
INTRODUCTION The purpose of this study is to demonstrate in organizing a histopathological knowledge
APPLIED
MATHEMATICS
OJ. E. W&r,
1988
the usefulness of Q-analysis [l] base; the specific knowledge
AND COMPUTATION 26:289-301
(1988)
289
290
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
base is a component of the histo- and cytopathological diagnostic expert system at the University of Arizona, which has three subsystems or modules [2]. The first module guides the dynamic reconfiguration of processor elements in a multiprocessor computer system; the second module uses prior knowledge to guide the scene decomposition and the extraction of diagnostic information; the third module uses a rule-based system to obtain a classification of the available information for diagnostic assessment. A specific example of Q-analysis of a knowledge base for the third module is given; here expert opinion provides, for colonic sections, four diagnostic categories and nineteen diagnostic clues (six concerning overall tissue architecture, six concerning characteristics of individual glands, and seven concerning characteristics of nuclei) [ 111. The technique of Q-analysis helps in structuring this set of diagnostic clues and may provide a forum for finding a compromise or consensus among human experts. This paper is organized as follows: the problem is stated specifically; Q-analysis is applied to the data; the results are presented and discussed; conclusions are drawn concerning the analyses presented; and additional research points to be considered are noted.
STATEMENT
OF PROBLEM
As stated above, the purpose of this study is to demonstrate the use of polyhedral dynamics or Q-analysis [l, 5, 121 in organizing a knowledge base. The specific example originates as follows: module 3 of the diagnostic expert system, with which we are concerned, was originally built in a somewhat ad hoc manner. Although the module was adequate in many respects, it was difficult to update and modify the knowledge base, resulting in some inconsistencies; in addition, it was difficult to transfer the module to other systems. Q-analysis appears to be useful in organizing and systematizing the knowledge base for these purposes. The expert system under consideration was designed to incorporate into the knowledge base the procedure actually used by diagnosticians. In practice, a pathologist looks at slides (see Figure 1) and makes one of four possible diagnoses: normal, adenoma, atypical adenoma, or malignant. These categories can be thought of as representing fairly distinguishable points on a continuum from normal to abnormal. The diagnostic clues are listed in Table 1; they were specified during a number of conferences involving the authors and Professors S. Paplanus and A. Graham of the Department of Pathology of the University of Arizona. Note that these clues are grouped according to whether they refer to overall tissue architecture, individuaI glands, or individual nuclei. An indication of
Knowledge Base and Q-Analysis
la
lb FIG. 1. (a) A section from normal colon; (b) an example of tubular adenoma; and (c) a case of adenocarcinoma of the colon.
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
292
1C FIG. 1.
(Continued)
the pattern of occurrence or the “value” to the diagnostician of a particular characteristic for each of the four diagnostic categories is given by the entries in the table. The pathologists were also asked to indicate the importance of a particular diagnostic clue, indicated by A, B, C, D, in decreasing order of importance, and its recognizability, indicated by Y (yes), M (medium), and N (no). The incidence matrix A for the relationship between the diagnostic clue set A and the diagnostic category set B is defined as
A=
v(j, k)
with q(+) =
1 0
if
{A(j)J@))
CA
otheese
Members of the two sets used in the example are given with their abbreviations in Table 1. The set A is referred to as the simplex set, and the set B is referred to as the vertex set. In our example, m = 19 and n = 4. The m X n matrix A describes the relationships or linkages between sets A and B and defines a complex KA( B, A); here such a complex represents the data seen from the viewpoint of the diagnostic clues. If the vertex and simplex sets
293
Knowledge Base and Q-Analysis TABLE 1 PRIMARY
Imp.
Rec.
MATRIX
FOR
DIAGNOSTIC
Clue
CLUES
Normal
AND
DIAGNOSTIC
Adenoma
CATEGORIES
Atypical Adenoma
Malignant
3
0
Glandular tissue architecture Y
A
Ease of recognition
5
5
M
C
Re-sty spacing
3
3
0
1
3
3
0
1
5
5
5
0
1
1
0
1
1
of gland
IRSP
M
B
Regularity of gland shape IRSH
Y
A
Location of gland inside mucosa LIMU
M
C
Regularity of gland size
0
RESI
N
D
Regularity of gland lumen
3
0
RELU
Gland characteristics Y
B
Epithelial lining
1
3
3
0
1
4
3
0
1
5
3
3
1
3
4
5
1
1
3
5
4
0
(layering) EPLI
Y
Crowding
(pattern)
CFU’T
M
Number of nuclei NONU
Y
Migration of nuclei MIGT
Y
Orientation of nuclear polarity (variance) ORNU
Y
A
Secretion
0
0
SECR
Nuclei N
D
Size
1
0
3
3
4
5
SIZE
Y
B
Shape SIiAP
0
1
294
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER TABLE 1
Imp.
Rec.
Clue
(Continued)
Normal
Adenoma
Atypical Adenoma
Malignant
Nuclei M
A
Chromatin pattern
0
1
4
5
CPA-r
M
C
1
Nucleoli
1
0
4
NUCL
M
A
Nuclear membrane
0
1
3
5
NUME M
B
Mitosis
1
4
1
1
0
0
MIT0 Y
A
Variance of size
3
5
VARS
are interchanged, the conjugate complex KB( A, X) is obtained. The conjugate complex represents the data seen from the viewpoint of the diagnostic categories. Initially, the incidence matrix may be composed of any real numbers or ordinal symbols. Then a (0,l) matrix can be obtained using a threshold or slicing function e( j, kl:
dj, k) =
0 1
if
e(j,k)
if
O(j,k)aP
j=l >..*,
m,
k=l,...,n,
where 8* takes on several values specified by the decision maker. Specifying different values of 8* permits comparison of the results of slicing at different levels. In our example, numerical and ordinal values for different diagnostic clues in relation to different diagnostic categories were obtained as shown in Table 1. Analyses were then performed for slicing at levels 1 through 5, as discussed below. The complex K indicates the global relationship between sets A and B. A more detailed study of the relationship between simplices forming a complex can be based on q-connectivity, which defines sets of equivalence classes within the simplicial complex 13, 71. Each level of q-connectivity has an associated set of equivalence classes. These equivalence classes indicate diagnostic clues that provide similar information, with respect to the assumptions of the analysis. ANALYSIS AND RESULTS As noted above, the two sets used for the Q-analysis are the diagnostic clues, set A (simplices) and the diagnoses or types of abnormality, set B
295
Knowledge Base and Q-Analysis
(vertices). The matrix ABT represents, here, an indication (on an ordinal scale) of the value (or extent of worthiness) of each diagnostic clue for each diagnostic category. The Q-analysis of the direct complex provides a measure of the usefulness of the clues. Thus, if it were possible to use only two diagnostic clues, it would be more advantageous to choose two from different equivalence classes than two from the same equivalence class. It should be noted that the order of the members in an equivalence class is immaterial, by definition of an equivalence class. In fact, note that the order of appearance in an equivalence class is the same as the order of appearance in the list of variables. In order to investigate how the simplices (diagnostic clues) conform into the complex (the histologic example represented by the matrix given above) and whether or not there are any simplices (clues) that are totally disconnected, the following indices may be used: eccentricity, complexity, connectivity, pattern, and obstruction vector [3, 121. These indices are discussed below, and several are exemplified in the present study. The conjugate complex analysis, which considers the data from the viewpoint of the knowledge base, helps in determining a satisfactory number of diagnostic categories. Thus Q-analysis helps to identify the diagnostic categories which are more easily detected. Q-analyses were performed using slicings 1 through 5 for the complete set of diagnostic clues and for the seven subsets consisting, respectively, of diagnostic clues of specific level of importance (4 levels) or recognizability (3 levels). The numbers of diagnostic clues belonging to each of the various subsets are summarized in Table 2. Note that subset D (least important) and subset N (not easily recognizable) have only (the same) two members: regulularityof gland l~en and nuclear size; the Q-analysis results for these two subsets are thus not discussed. Results for the other subsets are summarized for slicing parameters 2 and 3 in Table 3. Eccentricity expresses if and how much a criterion stands out from other criteria, with respect to equivalence class membership. Specifically, given TABLE 2 CLASSIFICATION
OF DIAGNOSTIC
CLUES
BY IMPORTANCE
AND
RECOGNIZABILITY
Recognizability Importance
Y
M
N
Total
A B C D
7 2 0 0
2 2 4 0
0 0 0 2
9 4 4 2
Total
9
8
2
19
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
296
TABLE 3 SUMMARY
OF EQUIVALENCE
CLASSES
Members of class Slicing value 2
Clue set
All
Yes
EREC
IRSP
IFtSH
LIMU
EPLI
CRPT
NONU
MIGT
SECR
SHAP
CPAT
NUME
EREC m
LIMU [
MICT
3
SIZE
MIT0
EREC
LIMU
NONU
MIGT
EREC
LIMU
MIGT
Medium NONU
A
EREC m pq
B
SHAF
C
LIMU
ERJX
MIGT
MIGT
pcpAT
IRSH EPLI
NONU
m
LIMU
NUME
IRSH m
EPLI SHAP
pGq NONU
NONU
that a criterion first appears alone in a component, eccentricity indicates for how many levels this is the case before another criterion also appears in the component. Thus a nonzero eccentricity is found only for criteria that first appear alone in a component (or equivalence class). Connectivity indicates the relationships among equivalence classes and essentially is a complementary way of looking at eccentricity. Thus a simplex which is completely detached has connectivity zero, and infinite eccentricity; higher degrees of connectivity indicate greater degrees of integration of the complex. The complexity index provides yet another measure of the relatedness of the data; it is high if there are numerous connections or paths between A and B, that is, between the simplices and the vertices. The obstruction vector and the pattern are other indicators which may be relevant for some studies; they are outside the scope of the present study. The objective of Q-analysis is to provide a basis for understanding the structure and connectivity of a data set. Thus, although the full set of results
Knowledge Base and Q-Analysis
297
should be examined, identifying small equivalence classes at relatively high Q-level is of particular interest. This provides an indication of the connectivity between diagnostic clues, with respect to diagnostic categories. In our example, analysis of the primary matrices, that is, the structure of the diagnostic clues, is of particular importance, and most of our discussion concerns this analysis. The equivalence classes corresponding to the largest q-values are given in Table 3. The following specific points of the analysis should be noted: (1) The results for slicing values 2 and 3 are reported; examination of the analyses for all slicing values indicated that the results for these two values best meet the desirability criteria of relatively small equivalence classes for relatively large g-levels. (2) The diagnostic clues RESI, RELU, emu, NUCL, and VARSare omitted from Table 3 because they appear only at the lowest q-level for any subset for either slicing level. (3) The following three pairs of diagnostic clues have identical entries in the original matrix: IRSP and IRSH; SHAP and CPAT; and ORW and VARS.Note that whenever one of these clues occurs in an equivalence class, the other also occurs. (4) Slicing level is an important factor in determining the usefulness of a Q-analysis. For simplicity, each of the analyses in this example was done for a constant slicing value across different diagnostic clues. However, Q-analysis permits specification of different slicing levels for different diagnostic clues in the same analysis. Since the slicing level determines whether a particular diagnostic clue is considered to occur for each of the different diagnostic categories, it would probably be preferable to specify different slicing levels for different diagnostic clues; this will be done in a subsequent study. Also, combining slicing levels using a weighted q-level for each diagnostic clue, as in multicriterion Q-analysis [6, 81, might be useful. For this example, the conjugate analysis is less interesting than the primal analysis, for at least two reasons. First, for the practical reason that the diagnostic categories are essentially determined prior to the analysis and are not subject to choice. Second, for the mathematical reason that the matrix is 19 X 4, thus providing, for the conjugate analysis, equivalence classes with a maximum of 4 elements for g-values 1 through 18. Interestingly, for slicing values 2 and 3, ATAD (atypical adenoma) is the only element in the only equivalence class for the first few (largest) values of 9 for the total set All and for the subsets Yes and A. Results for other values of the slicing parameter and for the other subsets do not provide useful information for the analysis, presumably because of the small and predetermined number of diagnostic categories.
298
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
DISCUSSION Tlne example above concerns a data set (primal matrix) obtained by discussion, compromise, and finally agreement among pathologists (experts). The inevitable variability among ratings by different pathologists is obscured by this averaging or consensus procedure. However, variability may itself be relevant in considering alternative procedures for combining expert opinions. There has been considerable interest recently in the problem of expert resolution, and a variety of methods for combining expert opinions have been suggested [4, 9, 10, 131. The procedure used in obtaining the initial rating matrix in our example was informal and essentially involved discussion, which eventually resulted in compromise and agreement. Thus the rating matrix does not actually represent the opinion of any one of the experts, although it does represent some form of agreement or consensus with respect to the combination of their opinions. Analysis of the example shows the expected sensitivity of the Q-analysis results to slicing value or threshold. To consider this in more detail, suppose, for illustration, that we assume that the differences between two experts can be represented by different (constant) slicing parameters applied to the matrix given in Table 1. Although this is not a particularly realistic example, it will serve for illustration. The entries in the original rating matrix are integers between 0 and 5. For slicing value 2, entries 0 and 1 in the rating matrix are 0 in the incidence matrix, and entries 2, 3, 4, and 5 in the rating matrix are 1 in the incidence matrix. For slicing value 3, entries 0, 1, and 2 in the rating matrix are 0 in the incidence matrix, and entries 3, 4, and 5 in the rating matrix are 1 in the incidence matrix. Thus the only entries that differ for the two incidence matrices are those that correspond to an entry of 2 in the rating matrix; these entries are 1 in the incidence matrix for slicing value 2, and 0 in the incidence matrix for slicing value 3. These entries are circled in Table 1. The diagnostic clues which occur in the equivalence sets for q-levels 1, 2, and 3 for slicing value 3, but not for slicing value 2, are marked by rectangles in Table 3. Note that each of these clues has rating 2 in the original data matrix and thus different entries in the incidence matrices for slicing values 2 and 3. Note also that each diagnostic clue occurring at any q-level for slicing value 3 but not for slicing value 2 has an incidence matrix entry of 1 for slicing value 2, and 0 for slicing value 3. This example illustrates the sensitivity of the results of Q-analysis to seemingly rather minor variations in the rating and corresponding incidence matrices. This sensitivity is contrasted with the lack of sensitivity to values
Knowledge Base and Q-Analysis
299
other than those affected by alternative slicing values; e.g., for the first diagnostic clue, EREC, any combination of 3’s, 4’s, and 5’s for normal, adenoma, and atypical adenoma with either 0 or 1 for malignant would result in the same incidence matrix for both slicing values. This is inevitable when ratings are converted to O’s and l’s, and must be kept in mind in interpreting the results of Q-analysis and in deciding whether or not this technique is appropriate for a particular problem. Incidence matrices recorded for different experts clearly reveal relationships between diagnostic clues and outcomes for which agreement exists. A difference matrix can be used to show those relationships for which there is no consensus; such difference matrices can be used to pinpoint the areas where consensus by mapping, change in slicing level, or forming a set theoretical union may be sought. When ratings are given as real numbers and are obtained from one or several sufficiently large groups of experts, estimates of the mean rating value, its confidence limits, and its tolerance limits can be made, and provide a basis for considering probability assessments of individual experts. In our examples, the slicing level remains constant across diagnostic clues, which may not be a realistic representation of human expert opinion. A subsequent study will consider the results of Q-analyses where the slicing value is specified separately for each diagnostic clue by each of several experts (pathologists). These specifications, obtained independently from different experts, will then be used as a basis or starting point for studying procedures for achieving consensus; variability among the pathologists with respect to the rating of different diagnostic clues and the importance attached to the respective clues will be considered as factors presumably influencing the consensus. Q-analysis may also be useful in another aspect of developing a diagnostic expert system. The extraction of the diagnostic clues from the digitized microscopic images involves a substantial number of image processing tasks. In the diagnostic expert system under development at the University of Arizona, this image processing is carried out by means of a 36 unit multimicroprocessor computer system. This system is capable of dynamic reconfiguration of its architecture, so that it can adapt to the processing needs of various information extraction tasks. Pools of processing elements connected by privileged communication queues are assigned to different tasks. We thus have a set of sources of information (image data as processed by the information extraction tasks) and a set of consumers or users of this information (computer resources or pools of microprocessers). Q-analysis may be useful in examining the relationship between these two sets, and may thus provide guidance for the assignment decisions.
309
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
CONCLUSIONS This example was chosen for the purpose of demonstrating the applicability of Q-analysis in helping to organize a histological knowledge base. The results are encouraging. The direct (primary) analysis provides a measure of the connectivity among diagnostic clues based on the equivalence classes of the diagnostic clues, which cut across histological categories; it also identifies five diagnostic clues whose usefulness might be reexamined. The patterns change for different subsets of diagnostic clues and for different values of the slicing parameter, providing additional insight into the structure of the data base. In particular, the crucial importance of the value of the slicing parameter is demonstrated. In using Q-analysis, the investigator must consider this choice carefully, since it essentially determines the incidence or data matrix. More specifically, consideration should be given to specifying different slicing values for different diagnostic clues, based on expert knowledge. Results for several specifications can be compared, using the analyses discussed above. Additional indices, including eccentricity, connectivity, and complexity, can also be used as a basis of comparison. In summary, this example demonstrates that Q-analysis may be helpful to diagnosticians in defining and explaining the process they use in arriving at a diagnosis and in using this information as a basis for updating the knowledge base. Experience with Q-analysis and, in particular, with specifying different slicing levels for different diagnostic clues, according to diagnostic experience, should increase the utility of this method in helping pathologists organize and update their knowledge bases.
This work was supported
in part by NlH grant l-POl-CA-38548.
REFERENCES 1
2
3 4
R. H. Atkin and J. Casti, Polyhedral Dynamics and the Geometry of Systems, RR-77-6, International Institute for Applied Systems Analysis, Laxenburg, Austria, 1977. P. H. Bartels, A. Graham, S. Paplanus, R. Shoemaker, and R. Maenner, Computer configurations for the processing of diagnostic imagery in histopathology, in Proceedings, International Workshop on Computer Architectures in Image Processing, Tucson, Arizona (K. Preston and L. Uhr, Eds.), Academic, 1984. J. Casti, Connectioity, Complexity, and Catastrophe in LargeScale Systems, International Series on Applied Systems Analysis, Vol. 7, Wiley, London, 1979. R. T. Clemen, Calibration and the aggregation of probabilities, Management Sci. 32(3):312-314 (1986).
Knowledge Base and Q-Analysis 5 6
7
8 9 10 11 12
13
301
L. Duckstein, Evaluation of the performance of a distribution system by Q-analysis, Appl. Math. Cornput. 13:173-185 (1983). L. Duckstein, J. Kempf, and J. Casti, Design and management of regional systems by fuzzy ratings and polyhedral dynamics (MCQA), in Macro Ecutwmic Planning with Conflicting Coals (P. Nijkamp, M. Despontin, and I. Spronk, Eds.), Lecture Notes in Economics and Mathematical Systems, Springer, New York, 1984. B. N. Featherkile and L. Duckstein, A Structural Analysis of au Ecological System Using Polyhedral Dynamics, Working Paper 86-019, Dept. of Systems and Industrial Engineering, Univ. of Arizona, Tucson, 1986. H. Hiessl, L. Duckstein, and E. J. Plate, Multiobjective Q-analysis with concordance and discordance concepts, A&. Math. Comput. 17:107-122 (1985). P. A. Morris, Observations on expert aggregation, Management Sci. 32(3): 321-328 (1986). P. A. Morris, Combining expert judgments: A Bayesian approach, Management Sci. 23(7):679-693 (1983). S. Paplanus, A. Graham, J. Layton, and P. H. Bartels, Statistical histometry in the diagnostic assessment of tissue sections, 1. Anal. Quant. Cytol. 7:32-37 (1985). R. T. Pfaff and L. Duckstein, Ranking alternative plans for the Santa Cruz River basin by Q-analysis, in Proceedings, Joint Arizona Section, AWRA, and Hydrology Section, Arizona-Nevada Acad. Sci., Tucson, 1981. R. L. Winkler, Expert resolution, Management Sci. 32(3):298-303 (1986).
of a Knowledge Base by Q-Analysis
L. Duckstein
Systems Engineering Department Case Western Reserve University Cleveland, Ohio 44106 P. H. Bartels Optical Sciences Center and Department The University of Arimnu Tucson, Arimnu 85721
of Pathology
and J. E. Weber
Department of Statistics The University of Arizona Tucson, Arizona 85721
ABSTRACT Q-analysis is used in organizing a histopathological knowledge base which is a component of the diagnostic expert system at the University of Arizona. This expert system has three subsystems or modules: the first module guides the dynamic reconfiguration of the computer system; the second module guides scene decomposition and the extraction of diagnostic information; the third module uses a rule-based system to obtain a diagnostic assessment. A specific example of the usefulness of Q-analysis is given in the context of the third module; the data represent expert opinion concerning diagnosis of cobnic cancer and are summarized in a matrix representing four diagnostic categories and nineteen diagnostic clues. The example shows that Q-analysis may be helpful to diagnosticians in defining and explaining the process they use in arriving at a diagnosis and in using this information as a basis for structuring the knowledge base.
INTRODUCTION The purpose of this study is to demonstrate in organizing a histopathological knowledge
APPLIED
MATHEMATICS
OJ. E. W&r,
1988
the usefulness of Q-analysis [l] base; the specific knowledge
AND COMPUTATION 26:289-301
(1988)
289
290
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
base is a component of the histo- and cytopathological diagnostic expert system at the University of Arizona, which has three subsystems or modules [2]. The first module guides the dynamic reconfiguration of processor elements in a multiprocessor computer system; the second module uses prior knowledge to guide the scene decomposition and the extraction of diagnostic information; the third module uses a rule-based system to obtain a classification of the available information for diagnostic assessment. A specific example of Q-analysis of a knowledge base for the third module is given; here expert opinion provides, for colonic sections, four diagnostic categories and nineteen diagnostic clues (six concerning overall tissue architecture, six concerning characteristics of individual glands, and seven concerning characteristics of nuclei) [ 111. The technique of Q-analysis helps in structuring this set of diagnostic clues and may provide a forum for finding a compromise or consensus among human experts. This paper is organized as follows: the problem is stated specifically; Q-analysis is applied to the data; the results are presented and discussed; conclusions are drawn concerning the analyses presented; and additional research points to be considered are noted.
STATEMENT
OF PROBLEM
As stated above, the purpose of this study is to demonstrate the use of polyhedral dynamics or Q-analysis [l, 5, 121 in organizing a knowledge base. The specific example originates as follows: module 3 of the diagnostic expert system, with which we are concerned, was originally built in a somewhat ad hoc manner. Although the module was adequate in many respects, it was difficult to update and modify the knowledge base, resulting in some inconsistencies; in addition, it was difficult to transfer the module to other systems. Q-analysis appears to be useful in organizing and systematizing the knowledge base for these purposes. The expert system under consideration was designed to incorporate into the knowledge base the procedure actually used by diagnosticians. In practice, a pathologist looks at slides (see Figure 1) and makes one of four possible diagnoses: normal, adenoma, atypical adenoma, or malignant. These categories can be thought of as representing fairly distinguishable points on a continuum from normal to abnormal. The diagnostic clues are listed in Table 1; they were specified during a number of conferences involving the authors and Professors S. Paplanus and A. Graham of the Department of Pathology of the University of Arizona. Note that these clues are grouped according to whether they refer to overall tissue architecture, individuaI glands, or individual nuclei. An indication of
Knowledge Base and Q-Analysis
la
lb FIG. 1. (a) A section from normal colon; (b) an example of tubular adenoma; and (c) a case of adenocarcinoma of the colon.
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
292
1C FIG. 1.
(Continued)
the pattern of occurrence or the “value” to the diagnostician of a particular characteristic for each of the four diagnostic categories is given by the entries in the table. The pathologists were also asked to indicate the importance of a particular diagnostic clue, indicated by A, B, C, D, in decreasing order of importance, and its recognizability, indicated by Y (yes), M (medium), and N (no). The incidence matrix A for the relationship between the diagnostic clue set A and the diagnostic category set B is defined as
A=
v(j, k)
with q(+) =
1 0
if
{A(j)J@))
CA
otheese
Members of the two sets used in the example are given with their abbreviations in Table 1. The set A is referred to as the simplex set, and the set B is referred to as the vertex set. In our example, m = 19 and n = 4. The m X n matrix A describes the relationships or linkages between sets A and B and defines a complex KA( B, A); here such a complex represents the data seen from the viewpoint of the diagnostic clues. If the vertex and simplex sets
293
Knowledge Base and Q-Analysis TABLE 1 PRIMARY
Imp.
Rec.
MATRIX
FOR
DIAGNOSTIC
Clue
CLUES
Normal
AND
DIAGNOSTIC
Adenoma
CATEGORIES
Atypical Adenoma
Malignant
3
0
Glandular tissue architecture Y
A
Ease of recognition
5
5
M
C
Re-sty spacing
3
3
0
1
3
3
0
1
5
5
5
0
1
1
0
1
1
of gland
IRSP
M
B
Regularity of gland shape IRSH
Y
A
Location of gland inside mucosa LIMU
M
C
Regularity of gland size
0
RESI
N
D
Regularity of gland lumen
3
0
RELU
Gland characteristics Y
B
Epithelial lining
1
3
3
0
1
4
3
0
1
5
3
3
1
3
4
5
1
1
3
5
4
0
(layering) EPLI
Y
Crowding
(pattern)
CFU’T
M
Number of nuclei NONU
Y
Migration of nuclei MIGT
Y
Orientation of nuclear polarity (variance) ORNU
Y
A
Secretion
0
0
SECR
Nuclei N
D
Size
1
0
3
3
4
5
SIZE
Y
B
Shape SIiAP
0
1
294
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER TABLE 1
Imp.
Rec.
Clue
(Continued)
Normal
Adenoma
Atypical Adenoma
Malignant
Nuclei M
A
Chromatin pattern
0
1
4
5
CPA-r
M
C
1
Nucleoli
1
0
4
NUCL
M
A
Nuclear membrane
0
1
3
5
NUME M
B
Mitosis
1
4
1
1
0
0
MIT0 Y
A
Variance of size
3
5
VARS
are interchanged, the conjugate complex KB( A, X) is obtained. The conjugate complex represents the data seen from the viewpoint of the diagnostic categories. Initially, the incidence matrix may be composed of any real numbers or ordinal symbols. Then a (0,l) matrix can be obtained using a threshold or slicing function e( j, kl:
dj, k) =
0 1
if
e(j,k)
if
O(j,k)aP
j=l >..*,
m,
k=l,...,n,
where 8* takes on several values specified by the decision maker. Specifying different values of 8* permits comparison of the results of slicing at different levels. In our example, numerical and ordinal values for different diagnostic clues in relation to different diagnostic categories were obtained as shown in Table 1. Analyses were then performed for slicing at levels 1 through 5, as discussed below. The complex K indicates the global relationship between sets A and B. A more detailed study of the relationship between simplices forming a complex can be based on q-connectivity, which defines sets of equivalence classes within the simplicial complex 13, 71. Each level of q-connectivity has an associated set of equivalence classes. These equivalence classes indicate diagnostic clues that provide similar information, with respect to the assumptions of the analysis. ANALYSIS AND RESULTS As noted above, the two sets used for the Q-analysis are the diagnostic clues, set A (simplices) and the diagnoses or types of abnormality, set B
295
Knowledge Base and Q-Analysis
(vertices). The matrix ABT represents, here, an indication (on an ordinal scale) of the value (or extent of worthiness) of each diagnostic clue for each diagnostic category. The Q-analysis of the direct complex provides a measure of the usefulness of the clues. Thus, if it were possible to use only two diagnostic clues, it would be more advantageous to choose two from different equivalence classes than two from the same equivalence class. It should be noted that the order of the members in an equivalence class is immaterial, by definition of an equivalence class. In fact, note that the order of appearance in an equivalence class is the same as the order of appearance in the list of variables. In order to investigate how the simplices (diagnostic clues) conform into the complex (the histologic example represented by the matrix given above) and whether or not there are any simplices (clues) that are totally disconnected, the following indices may be used: eccentricity, complexity, connectivity, pattern, and obstruction vector [3, 121. These indices are discussed below, and several are exemplified in the present study. The conjugate complex analysis, which considers the data from the viewpoint of the knowledge base, helps in determining a satisfactory number of diagnostic categories. Thus Q-analysis helps to identify the diagnostic categories which are more easily detected. Q-analyses were performed using slicings 1 through 5 for the complete set of diagnostic clues and for the seven subsets consisting, respectively, of diagnostic clues of specific level of importance (4 levels) or recognizability (3 levels). The numbers of diagnostic clues belonging to each of the various subsets are summarized in Table 2. Note that subset D (least important) and subset N (not easily recognizable) have only (the same) two members: regulularityof gland l~en and nuclear size; the Q-analysis results for these two subsets are thus not discussed. Results for the other subsets are summarized for slicing parameters 2 and 3 in Table 3. Eccentricity expresses if and how much a criterion stands out from other criteria, with respect to equivalence class membership. Specifically, given TABLE 2 CLASSIFICATION
OF DIAGNOSTIC
CLUES
BY IMPORTANCE
AND
RECOGNIZABILITY
Recognizability Importance
Y
M
N
Total
A B C D
7 2 0 0
2 2 4 0
0 0 0 2
9 4 4 2
Total
9
8
2
19
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
296
TABLE 3 SUMMARY
OF EQUIVALENCE
CLASSES
Members of class Slicing value 2
Clue set
All
Yes
EREC
IRSP
IFtSH
LIMU
EPLI
CRPT
NONU
MIGT
SECR
SHAP
CPAT
NUME
EREC m
LIMU [
MICT
3
SIZE
MIT0
EREC
LIMU
NONU
MIGT
EREC
LIMU
MIGT
Medium NONU
A
EREC m pq
B
SHAF
C
LIMU
ERJX
MIGT
MIGT
pcpAT
IRSH EPLI
NONU
m
LIMU
NUME
IRSH m
EPLI SHAP
pGq NONU
NONU
that a criterion first appears alone in a component, eccentricity indicates for how many levels this is the case before another criterion also appears in the component. Thus a nonzero eccentricity is found only for criteria that first appear alone in a component (or equivalence class). Connectivity indicates the relationships among equivalence classes and essentially is a complementary way of looking at eccentricity. Thus a simplex which is completely detached has connectivity zero, and infinite eccentricity; higher degrees of connectivity indicate greater degrees of integration of the complex. The complexity index provides yet another measure of the relatedness of the data; it is high if there are numerous connections or paths between A and B, that is, between the simplices and the vertices. The obstruction vector and the pattern are other indicators which may be relevant for some studies; they are outside the scope of the present study. The objective of Q-analysis is to provide a basis for understanding the structure and connectivity of a data set. Thus, although the full set of results
Knowledge Base and Q-Analysis
297
should be examined, identifying small equivalence classes at relatively high Q-level is of particular interest. This provides an indication of the connectivity between diagnostic clues, with respect to diagnostic categories. In our example, analysis of the primary matrices, that is, the structure of the diagnostic clues, is of particular importance, and most of our discussion concerns this analysis. The equivalence classes corresponding to the largest q-values are given in Table 3. The following specific points of the analysis should be noted: (1) The results for slicing values 2 and 3 are reported; examination of the analyses for all slicing values indicated that the results for these two values best meet the desirability criteria of relatively small equivalence classes for relatively large g-levels. (2) The diagnostic clues RESI, RELU, emu, NUCL, and VARSare omitted from Table 3 because they appear only at the lowest q-level for any subset for either slicing level. (3) The following three pairs of diagnostic clues have identical entries in the original matrix: IRSP and IRSH; SHAP and CPAT; and ORW and VARS.Note that whenever one of these clues occurs in an equivalence class, the other also occurs. (4) Slicing level is an important factor in determining the usefulness of a Q-analysis. For simplicity, each of the analyses in this example was done for a constant slicing value across different diagnostic clues. However, Q-analysis permits specification of different slicing levels for different diagnostic clues in the same analysis. Since the slicing level determines whether a particular diagnostic clue is considered to occur for each of the different diagnostic categories, it would probably be preferable to specify different slicing levels for different diagnostic clues; this will be done in a subsequent study. Also, combining slicing levels using a weighted q-level for each diagnostic clue, as in multicriterion Q-analysis [6, 81, might be useful. For this example, the conjugate analysis is less interesting than the primal analysis, for at least two reasons. First, for the practical reason that the diagnostic categories are essentially determined prior to the analysis and are not subject to choice. Second, for the mathematical reason that the matrix is 19 X 4, thus providing, for the conjugate analysis, equivalence classes with a maximum of 4 elements for g-values 1 through 18. Interestingly, for slicing values 2 and 3, ATAD (atypical adenoma) is the only element in the only equivalence class for the first few (largest) values of 9 for the total set All and for the subsets Yes and A. Results for other values of the slicing parameter and for the other subsets do not provide useful information for the analysis, presumably because of the small and predetermined number of diagnostic categories.
298
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
DISCUSSION Tlne example above concerns a data set (primal matrix) obtained by discussion, compromise, and finally agreement among pathologists (experts). The inevitable variability among ratings by different pathologists is obscured by this averaging or consensus procedure. However, variability may itself be relevant in considering alternative procedures for combining expert opinions. There has been considerable interest recently in the problem of expert resolution, and a variety of methods for combining expert opinions have been suggested [4, 9, 10, 131. The procedure used in obtaining the initial rating matrix in our example was informal and essentially involved discussion, which eventually resulted in compromise and agreement. Thus the rating matrix does not actually represent the opinion of any one of the experts, although it does represent some form of agreement or consensus with respect to the combination of their opinions. Analysis of the example shows the expected sensitivity of the Q-analysis results to slicing value or threshold. To consider this in more detail, suppose, for illustration, that we assume that the differences between two experts can be represented by different (constant) slicing parameters applied to the matrix given in Table 1. Although this is not a particularly realistic example, it will serve for illustration. The entries in the original rating matrix are integers between 0 and 5. For slicing value 2, entries 0 and 1 in the rating matrix are 0 in the incidence matrix, and entries 2, 3, 4, and 5 in the rating matrix are 1 in the incidence matrix. For slicing value 3, entries 0, 1, and 2 in the rating matrix are 0 in the incidence matrix, and entries 3, 4, and 5 in the rating matrix are 1 in the incidence matrix. Thus the only entries that differ for the two incidence matrices are those that correspond to an entry of 2 in the rating matrix; these entries are 1 in the incidence matrix for slicing value 2, and 0 in the incidence matrix for slicing value 3. These entries are circled in Table 1. The diagnostic clues which occur in the equivalence sets for q-levels 1, 2, and 3 for slicing value 3, but not for slicing value 2, are marked by rectangles in Table 3. Note that each of these clues has rating 2 in the original data matrix and thus different entries in the incidence matrices for slicing values 2 and 3. Note also that each diagnostic clue occurring at any q-level for slicing value 3 but not for slicing value 2 has an incidence matrix entry of 1 for slicing value 2, and 0 for slicing value 3. This example illustrates the sensitivity of the results of Q-analysis to seemingly rather minor variations in the rating and corresponding incidence matrices. This sensitivity is contrasted with the lack of sensitivity to values
Knowledge Base and Q-Analysis
299
other than those affected by alternative slicing values; e.g., for the first diagnostic clue, EREC, any combination of 3’s, 4’s, and 5’s for normal, adenoma, and atypical adenoma with either 0 or 1 for malignant would result in the same incidence matrix for both slicing values. This is inevitable when ratings are converted to O’s and l’s, and must be kept in mind in interpreting the results of Q-analysis and in deciding whether or not this technique is appropriate for a particular problem. Incidence matrices recorded for different experts clearly reveal relationships between diagnostic clues and outcomes for which agreement exists. A difference matrix can be used to show those relationships for which there is no consensus; such difference matrices can be used to pinpoint the areas where consensus by mapping, change in slicing level, or forming a set theoretical union may be sought. When ratings are given as real numbers and are obtained from one or several sufficiently large groups of experts, estimates of the mean rating value, its confidence limits, and its tolerance limits can be made, and provide a basis for considering probability assessments of individual experts. In our examples, the slicing level remains constant across diagnostic clues, which may not be a realistic representation of human expert opinion. A subsequent study will consider the results of Q-analyses where the slicing value is specified separately for each diagnostic clue by each of several experts (pathologists). These specifications, obtained independently from different experts, will then be used as a basis or starting point for studying procedures for achieving consensus; variability among the pathologists with respect to the rating of different diagnostic clues and the importance attached to the respective clues will be considered as factors presumably influencing the consensus. Q-analysis may also be useful in another aspect of developing a diagnostic expert system. The extraction of the diagnostic clues from the digitized microscopic images involves a substantial number of image processing tasks. In the diagnostic expert system under development at the University of Arizona, this image processing is carried out by means of a 36 unit multimicroprocessor computer system. This system is capable of dynamic reconfiguration of its architecture, so that it can adapt to the processing needs of various information extraction tasks. Pools of processing elements connected by privileged communication queues are assigned to different tasks. We thus have a set of sources of information (image data as processed by the information extraction tasks) and a set of consumers or users of this information (computer resources or pools of microprocessers). Q-analysis may be useful in examining the relationship between these two sets, and may thus provide guidance for the assignment decisions.
309
L. DUCKSTEIN, P. H. BARTELS, AND J. E. WEBER
CONCLUSIONS This example was chosen for the purpose of demonstrating the applicability of Q-analysis in helping to organize a histological knowledge base. The results are encouraging. The direct (primary) analysis provides a measure of the connectivity among diagnostic clues based on the equivalence classes of the diagnostic clues, which cut across histological categories; it also identifies five diagnostic clues whose usefulness might be reexamined. The patterns change for different subsets of diagnostic clues and for different values of the slicing parameter, providing additional insight into the structure of the data base. In particular, the crucial importance of the value of the slicing parameter is demonstrated. In using Q-analysis, the investigator must consider this choice carefully, since it essentially determines the incidence or data matrix. More specifically, consideration should be given to specifying different slicing values for different diagnostic clues, based on expert knowledge. Results for several specifications can be compared, using the analyses discussed above. Additional indices, including eccentricity, connectivity, and complexity, can also be used as a basis of comparison. In summary, this example demonstrates that Q-analysis may be helpful to diagnosticians in defining and explaining the process they use in arriving at a diagnosis and in using this information as a basis for updating the knowledge base. Experience with Q-analysis and, in particular, with specifying different slicing levels for different diagnostic clues, according to diagnostic experience, should increase the utility of this method in helping pathologists organize and update their knowledge bases.
This work was supported
in part by NlH grant l-POl-CA-38548.
REFERENCES 1
2
3 4
R. H. Atkin and J. Casti, Polyhedral Dynamics and the Geometry of Systems, RR-77-6, International Institute for Applied Systems Analysis, Laxenburg, Austria, 1977. P. H. Bartels, A. Graham, S. Paplanus, R. Shoemaker, and R. Maenner, Computer configurations for the processing of diagnostic imagery in histopathology, in Proceedings, International Workshop on Computer Architectures in Image Processing, Tucson, Arizona (K. Preston and L. Uhr, Eds.), Academic, 1984. J. Casti, Connectioity, Complexity, and Catastrophe in LargeScale Systems, International Series on Applied Systems Analysis, Vol. 7, Wiley, London, 1979. R. T. Clemen, Calibration and the aggregation of probabilities, Management Sci. 32(3):312-314 (1986).
Knowledge Base and Q-Analysis 5 6
7
8 9 10 11 12
13
301
L. Duckstein, Evaluation of the performance of a distribution system by Q-analysis, Appl. Math. Cornput. 13:173-185 (1983). L. Duckstein, J. Kempf, and J. Casti, Design and management of regional systems by fuzzy ratings and polyhedral dynamics (MCQA), in Macro Ecutwmic Planning with Conflicting Coals (P. Nijkamp, M. Despontin, and I. Spronk, Eds.), Lecture Notes in Economics and Mathematical Systems, Springer, New York, 1984. B. N. Featherkile and L. Duckstein, A Structural Analysis of au Ecological System Using Polyhedral Dynamics, Working Paper 86-019, Dept. of Systems and Industrial Engineering, Univ. of Arizona, Tucson, 1986. H. Hiessl, L. Duckstein, and E. J. Plate, Multiobjective Q-analysis with concordance and discordance concepts, A&. Math. Comput. 17:107-122 (1985). P. A. Morris, Observations on expert aggregation, Management Sci. 32(3): 321-328 (1986). P. A. Morris, Combining expert judgments: A Bayesian approach, Management Sci. 23(7):679-693 (1983). S. Paplanus, A. Graham, J. Layton, and P. H. Bartels, Statistical histometry in the diagnostic assessment of tissue sections, 1. Anal. Quant. Cytol. 7:32-37 (1985). R. T. Pfaff and L. Duckstein, Ranking alternative plans for the Santa Cruz River basin by Q-analysis, in Proceedings, Joint Arizona Section, AWRA, and Hydrology Section, Arizona-Nevada Acad. Sci., Tucson, 1981. R. L. Winkler, Expert resolution, Management Sci. 32(3):298-303 (1986).