- Email: [email protected]
Towards A Knowledge Base for Software Developers to Choose Suitable Traceability Techniques
Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
ScienceDirect Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science 159 (2019) 1075–1084 Procedia Computer Science 00 (2019) 000–000
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
23rd International Conference on Knowledge Based and Intelligent Information and Engineering 23rd International Conference on KnowledgeSystems Based and Intelligent Information and Engineering Systems
Towards A Knowledge Base for Software Developers to Choose Towards A Knowledge for Software Developers to Choose SuitableBase Traceability Techniques Suitable Traceability Techniques a* b c d Haruhiko Kaiya , Atsuo Hazeyama , Shinpei Ogata , Takao Okubo , Nobukazu e b f , Hironori Haruhiko Kaiyaa*, AtsuoYoshioka Hazeyama , ShinpeiWashizaki Ogatac, Takao Okubod, Nobukazu e a Kanagawa University, HiratsukaWashizaki 259-1293, Japanf Yoshioka , Hironori b Tokyo Gakugei University, Tokyo 84-8501, Japan a Kanagawa University, Hiratsuka 259-1293, Japan c Shinshu University, Nagano 380-0928, Japan b Tokyo University, Tokyo 84-8501, Japan dGakugei IISEC, Yokohama 221-0835, Japan c Shinshu University, Nagano 380-0928, Japan e NII, Tokyo 100-0003, Japan d IISEC, Yokohama 221-0835, Japan f Waseda University, Tokyo 169-8555 , Japan e NII, Tokyo 100-0003, Japan f Waseda
University, Tokyo 169-8555 , Japan
Abstract Abstract Huge amount of techniques for creating, maintaining and/or recovering traceability among software development artifacts have been proposed. It is thus not easy for a potential user of such techniques to choose a technique suitable for his/her project because Huge amount of techniques for creating, maintaining and/or recovering traceability among software development artifacts have too many techniques exist and projects are different from each other. In this paper, we proposed a model for characterizing a been proposed. It is thus not easy for a potential user of such techniques to choose a technique suitable for his/her project because traceability technique with respect to its users. The model can become a meta-model of the knowledge base for such users. The too many techniques exist and projects are different from each other. In this paper, we proposed a model for characterizing a model is designed on the basis of the contents of existing technical papers so that we can easily describe model instances on the traceability technique with respect to its users. The model can become a meta-model of the knowledge base for such users. The basis of technical papers. The model is represented in a feature model, and it has four mandatory features: source, destination, model is designed on the basis of the contents of existing technical papers so that we can easily describe model instances on the consequence and process. The user can at least understand a technique is unsuitable for him/her by referring such features. If a basis of technical papers. The model is represented in a feature model, and it has four mandatory features: source, destination, technique estimates the traceability relationships, the instance of its feature model may contain the quality metrics of its estimation consequence and process. The user can at least understand a technique is unsuitable for him/her by referring such features. If a such as precision and recall. It has also several optional features: assumptions, preprocess and tool. Although we sometimes technique estimates the traceability relationships, the instance of its feature model may contain the quality metrics of its estimation cannot obtain such optional features from technical papers, they are so helpful for a user to decide a technique suitable for him/her. such as precision and recall. It has also several optional features: assumptions, preprocess and tool. Although we sometimes On the basis of the feature model, we described model instances of several techniques in technical papers. We also examined and cannot obtain such optional features from technical papers, they are so helpful for a user to decide a technique suitable for him/her. discussed who the suitable user for each technique is. On the basis of the feature model, we described model instances of several techniques in technical papers. We also examined and c 2019 The Authors. Published by Elsevier B.V. discussed who the suitable user for each technique © 2019 The Authors. Published by B.V. is. Peer-review under responsibility of Elsevier KES International. c 2019 The Authors. by Elsevier B.V. This is an open accessPublished article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review Software under of International. under responsibility responsibility of KES KES International. Keywords: Traceability, Feature Model, User Perspective, Knowledge Base, Survey Keywords: Software Traceability, Feature Model, User Perspective, Knowledge Base, Survey
a a
Corresponding author. Tel.: +81-463-59-4111 E-mail address: [email protected] Corresponding author. Tel.: +81-463-59-4111 E-mail address: [email protected]
c 2019 The Authors. Published by Elsevier B.V. 1877-0509 1877-0509 2019responsibility The Authors. of Published by Elsevier B.V. Peer-review©under KES International. This is an open access article under the CC BY-NC-ND c 2019 1877-0509 The Authors. Published by Elsevier B.V. license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review underresponsibility responsibility of KES International. Peer-review under of KES International. 10.1016/j.procs.2019.09.276
1076 2
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1. Introduction Introducing a software traceability technique in a software development project is crucial for many reasons 1 . For example, most software products are developed based on the previous version of each product. To know the impact on design and source codes by requirements changes, traceability relationships are required. All software products today are very huge. Even a small command such as info-zip contains 60 thousand lines of codes. Some operating system consists of more than 100 million lines of codes. To understand such huge products, relationships among the same level of products such as among codes are crucial. Relationships among different levels such as requirements and codes are also crucial. Traceability techniques contribute to improving such understandability even if the product has only one version. Traceability techniques also contribute to checking completeness and correctness of a product and finding reusable components in a product. Because of the importance of traceability techniques, huge amount of papers have been published and consumed by researchers. In most papers, comparison between a proposed technique and others is shown, and the superiority of the proposed one is loudly presented in certain criteria. Several benchmark problems 2 are shared among researchers, and used for such comparison. However, primal users of traceability techniques, i.e. software developers, are not interested in such feeble improvement. They want to know which technique is valuable and suitable for their projects. Projects are different from each other. For example, a rigorous model driven development is performed in a project. On the other hand in another project, artifacts such as requirements and design documents are written in natural language, and their changes are often notified and shared by email. According to the background mentioned above, we setup the following research questions (RQs). • RQ1: What kinds of information about a traceability technique are required for its user to decide whether it is suitable for his/her project? • RQ2: What kinds of such information can be obtained from published documents such as technical papers? On the basis of our survey 1, we propose a feature model of a traceability technique as answers of the RQs. The model can be a basis of the knowledge base for software developers to choose traceability techniques suitable for their project. Although there already exists some frameworks for categorizing traceability techniques 1,3,4 , they do not intend to help users of the techniques to choose suitable ones. Such frameworks usually intend to improve and progress the research activities. The rest of this paper is organized as follows. In the next section, related works are briefly reviewed and discussed. On the basis of papers in our previous survey 1, we introduce a feature model of traceability techniques in section 3. Because the contents of technical papers are limited, we choose features acquirable from such papers. In section 4, several model instances based on actual papers are shown. Finally, we summarize our current results and show the future issues.
2. Related Work Plenty of techniques for maintaining, developing and recovering traceability relationships among software artifacts are proposed in huge number of technical papers. Their survey have been also summarized in several papers 1,3,4,5,6 . However, few papers focus on how to choose suitable technique(s) for each user. In a paper 5 , understanding stakeholders and traceability strategizing techniques are focused as future research directions. Although both directions are slightly related to traceability selection, the paper did not mention such relationships at least. In a survey 7 , usage scenarios of requirements traceability are summarized. However, how to choose the technique in each usage is not investigated. Bangchao Wang et al 8,9 focus on technology transfer from academy to industry. They proposed a maturity model of a technology whether it can be transferred to industry. The model contains many aspects of a technology such as usability, usefulness, customizability, quality, risk and cost. Although the model seems to be reasonable, it is not realistic to gather such many aspects only from technical papers related to a technology.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1077 3
Traceability Technique
source
destination
meaning
Legends
assumption
consequence
determinate
estimate
preprocess
batch
process
interactive
none
tool
parts
integrated
Mandatory quality
Optional
participation
Alternative precision
recall
tradeoff
Fig. 1. A Feature Model
3. A Feature Model of Traceability Techniques According to papers mentioned in our survey 1 , we develop a feature model of traceability techniques in Figure 1. As mentioned in introduction, there is no frameworks for software developers to choose a traceability technique so that they try to use it. Some kind of knowledge base of traceability techniques will contribute to helping such developers. The model in Figure 1 is designed so that the model can become a meta-model of such knowledge base. Before explaining each feature, we explain the criteria of selecting features in the model. We also explain the difference between mandatory and optional features in this model. We only focus on features that can be obtained from existing technical papers. Therefore, the kinds of features in our model are not complete, but model instances are usually described without missing features. To help software developers to choose their technique easily and suitably, necessary information for using each technique should be provided as much as possible. Features in the model are also chosen on the basis of such criteria. Instead, technical details how to establish traceability are intentionally excluded from this model even if such information can be obtained from papers. For example, we regard kinds of information retrieval methods such as Support Vector Machine (SVM) or Latent Semantic Indexing (LSI) are not necessary in this model. We regard developers cannot choose suitable technique without mandatory features in the model. Characteristics of a project using a traceability technique are not directly specified. Instead, assumptions for using a technique indirectly specify projects suitable for the technique as shown in section 3.4. Because we expect users in a project choose their technique(s) by themselves or they participate in selecting technique(s), such assumptions will contribute to selecting technique(s) suitable for the project. 3.1. Source (Mandatory) A traceability technique creates relationships between parts of software development elements. Examples of such parts are a paragraph in a requirements specification 10 , a Java class 11 , a feature of a software product line 12 , and UML model elements 13 . In some case, a member of a project, i.e. people, can be a source of a traceability technique. A model of the technique thus has to specify one of such parts. 3.2. Destination (Mandatory) The feature “destination” specifies another end of a traceability relationship. In many cases, it is not clearly specified that a traceability relationship has direction. Once a directional traceability relationship is established, its reverse direction can be easily added. Therefore, we do not distinguish the source and destination features. These features simply specify one end and another of a traceability relationship.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1078 4 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
1
2 recall
3 precision
4
5
6
7
some parameter
Fig. 2. An example of precision curve and recall curve
3.3. Meaning A traceability relationship creating and maintaining a technique sometimes has its specific usage. Suppose a feature in a software product line is implemented as a class of Java, and a technique maintains such relationships between features and codes 14 . In such a case, the user of the technique can identify features which are not implemented in codes of a specific application. We regard the meaning of the traceability is “implement” in our model. Other examples of the meaning are “refinement”, “design rationale” and “impact”. Kinds of traceability links such as overlap, satisfiability, conflict and dependency are enumerated in a handbook 15. Such links types are useful to specify the meaning. When such meaning is specified, potential users of the technique can easily examine and choose the technique. According to a survey 7, about 80 % of projects applied traceability techniques because they expect their benefits. The feature meaning clarifies such benefits to users. However, in several papers, such meaning is not clearly specified in each technique 1. 3.4. Assumption There is no general and standard software development project. Each traceability technique thus has several assumptions about a development project where the technique is used. When a user examine and choose existing traceability techniques, these assumptions are very important. If a project of the user does not satisfy such assumptions, the technique cannot be applied to the project. Otherwise, the project has to be modified so that the technique can be applied. The assumptions thus help users to estimate potential costs of introducing the technique. Examples of such assumptions are as follows. In some techniques, information retrieval techniques are applied to documents, and many of them assume the documents are written in English 16 . The naming convention of classes is sometimes assumed 11 . Several techniques 17,18,19,10 require software development history or logs. A technique 20 assumes domain ontology that are used for semantic matching among words and terms. Development styles such as software product line 12,21 and model driven development 13 are also assumed for each technique. When traceability relationships are directly and manually created and maintained, some additional tasks are added to its software development. Such additional tasks are also kinds of assumptions. For example, using some domain specific language 22 is a kind of assumptions. Scalability of a traceability technique is important in general 9 . However, general descriptions or metrics of scalability do not help users to choose a technique. Instead, assumptions exemplified above let users know whether a technique has a certain level of scalability suitable for them. 3.5. Consequence (Mandatory) Each technique recovers or develops traceability relationships. If a technique recovers the relationships on the basis of a probabilistic approach such as information retrieval methods, not all relationships are correct in general. On the other hand, all the relationships are basically correct if a technique enforces some notations and tasks to maintain the
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1079 5
1 0.9
precision
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0
0.2
0.4
0.6
0.8
1
recall Fig. 3. An example of precision/recall curve
relationships. For users of the technique, we have to clarify whether a technique is first or last type. We call the first type as “estimate”, and the second on as “determinate”. For the first type, we also have to specify the quality of estimated relationships. Following two metrics are useful to know the quality of a technique. Precision = Recall =
| (estimated relationships) ∩ (correct relationships) | | estimated relationships |
| (estimated relationships) ∩ (correct relationships) | | correct relationships |
Note that | relationships | is the number of relationships. Composite metrics such as F-measure are not useful for people who want to choose a traceability techniques because it does not let them know two different criteria respectively; one is how much wrong relationships are contained (false positive) in the results by the technique and another is how much right relationships are missing (false negative) by the technique. The tradeoffs between precision and recall are also very useful for technique choosers. In some research 23 , some parameter is used and changes of recall and precision values are shown along the progress of the parameter as shown in Figure 2. Note that values of the vertical axis in the figure take 0 to 1. If the value takes 1, precision or recall is perfect, i.e. no false positive or no false negative. If the value of precision or recall takes 0, all predicted results are false positive or false negative. Precision/recall curves as exemplified in Figure 3 are more useful than the changes of precision and recall respectively because users can directly know the tradeoffs between precision and recall. Note that values of the both axes in the figure are the same as the values of the vertical axis in Figure 2. Because metrics such as precision and recall are calculated on the basis of some data such as benchmark examples, the values do not show the quality of a technique in general. Therefore, it is better to explain the characteristics of such data, e.g. size, complexity and artifacts in a class room or an actual project. In the context of writing technical papers, comparing a technique to others are very important. However, such kind of comparison is not so important in the context of using techniques because the absolute level of the quality is important. For example, it is meaningless for users that the recall of a technique becomes 1.2 times better than existing one if the absolute values of the recall are only 10% and 12%. It is too terrible that about 80 % of correct relationships are missing by the technique. 3.6. Preprocess In the case of estimated techniques, several preprocessing tasks are required before recovering traceability relationships. Such tasks are normally negative factors of introducing a technique because software developers have to perform additional tasks. Although some preprocessing tasks can be performed automatically, such automation normally negatively affects the quality of the preprocessed data. Especially for textual artifacts such requirements documents, such automation is not so easy. For example, tasks such as sectioning documents, unifying terms, removing general terms 16,24,20,18,25,26,27 are typically required and they are not fully automated. Other types of preprocessing
1080 6
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
are setting the list of keywords 14,28,10 and also setting some value as the threshold 14. There is a tradeoff between the amount of preprocessing tasks and the degree of processing automation. In the case of determinate techniques, there are few preprocessing tasks because tasks for creating and maintaining traceability relationships are embedded in the software development process itself. In our model, such issue is noted in the “assumption” feature mentioned above. 3.7. Process (Mandatory) In each technique, several tasks are performed for creating, maintaining and/or recovering traceability relationships. The user of such techniques is interested in what and how many additional operations he/she has to perform. We call a technique performs a “batch” process if the user performs nothing for traceability during the process. On the other hand, a technique performs an “interactive” process, and its user has to do something during the process. A typical operation in an interactive process is user’s judgement. In a technique 29 , users give their judgement of classified traceability relationships. 3.8. Tool If tools for performing a traceability technique exist, the number of its potential users increases. We thus specify the status of such tools for each technique. With respect to tools, we categorize techniques into the following three types: none, parts, integrated. If a technique or its papers state there is no tool support, we categorize the technique is “none”. If they do not introduce or do not mention any tools, this feature is not specified. When some pieces of tools are used in a technique, we categorized it is “parts”. For example, Java packages for vector space model, LSI and Sufficient Dimensionality Reduction (SDR) implementations in MATLAB are used in a technique 16. Several techniques provide integrated tools to perform them. In such cases, we call a technique is “integrated”. Some tools are stand alone 12 , and others are built on a well-known IDE such as Eclipse 20,25 . 4. Model Instances In this section, we describe several model instances of traceability techniques on the basis of existing technical papers. We also discuss for what kind of users each technique is suitable. 4.1. Information Retrieval based Traceability Recovery using User Feedback In a paper 29 , traceability between documents are recovered on the basis of their textual similarity. Source: document Destination: document Assumption: Artifact types such as requirements, design, test and codes are identified. Consequence: Estimate. – Precision: Yes. – Recall: Yes. – Tradeoff: Yes, in the same way as shown in Figure 3. Data of following three projects were used to calculate them: Easy-Clinic, iTrust, Moderate Resolution Imaging Spectroradiometer (MODIS). First two projects are contents for university education. The last one is an open source software by NASA. Project Web pages introduced in this paper do not exist in March 2019. • Process: Interactive. A user has to judge whether first candidate of a traceability relationship is correct or not repeatedly. • • • •
If a user can perform the judgement mentioned in this technique, this technique is suitable for him/her. Unfortunately, the paper does not mention tool supports although empirical evaluation is shown. Therefore, the user has to manually perform the technique, or to develop its automated tool.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1081 7
4.2. Change Impact Analysis for Software Revision in a software non-intensive project In a paper 23 , software revision due to requirements changes is focused, and two techniques of estimating traceability between documents and source codes are proposed. The first technique is developed on the basis of answers of questionnaires from practitioners. The second one is developed on the basis of the feedback about the experiences of the first one by the practitioners. Because software is not a central part in projects the technique will be applied, they cannot easily introduce software development methods such as software product line techniques and model driven development. According to our feature model in Figure 1, we characterized the first technique as follows. • • • • •
• • •
Source: Sections in requirements document. Destination: Functions in source codes. Meaning: Change impacts. Assumption: Users can specify suitable keywords and terms corresponding to a requirements change. Indirect impacts are out of scope. Consequence: Estimate. The quality features are as follows: – Precision: Yes. – Recall: Yes. – Tradeoff: Yes. Using the number of query keywords, the tradeoff is depicted in the same way in Figure 2. Data in actual industrial projects were used to calculate the metrics above. The detail of the projects were not published. Preprocess: Several samples of actual impacted codes should be manually identified. Process: Interactive. Several trials are required. During the trial, keywords may be updated. Pseudo-precision and recall are provided on the basis of samples defined in the preprocess so that the user can judge whether the keywords should be updated or not. Tool: integrated. The tool is developed as an independent application.
A model instance of the second technique is also as follows. • • • • •
• •
•
Source: Sections in requirements document. Destination: Functions in source codes. Meaning: Change impacts. Assumption: Users can choose suitable keywords and terms corresponding to a requirements change from the candidates provided by the technique. The technique provides the candidates of the keywords. Consequence: Estimate. The quality features are as follows: – Precision: Yes. – Recall: Yes. – Tradeoff: No. Data in actual industrial projects were also used to calculate the metrics above. The detail of the projects were not also published. Preprocess: Several samples of actual impacted codes should be manually identified. Process: Interactive. Several trials are required. During the trial, keywords may be updated. Pseudo-precision and recall are provided on the basis of samples defined in the preprocess so that the user can judge whether the keywords should be updated or not. The technique recommends the candidates of new additional samples so that the number of samples increases. Tool: integrated. The tool is developed as an independent application.
The amount of user participation decreases when the technique is update from the first technique to second one. For example, the user simply chooses the keywords from candidates while he/she has to directly specify them in the first one. The second technique makes the number of samples for calculating pseudo-precision and recall increases so that the pseudo-metrics become more actual than ever.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1082 8
In the techniques, the meaning of traceability relationships is clearly defined. Therefore, users can easily decide whether the techniques are suitable for them. In addition, the amount of the user participation is clearly stated. The potential users can also discuss whether they can accept additional efforts caused by the user participation. 4.3. MDD and Model Transformation In a paper 13 , Model-driven development (MDD) and the traceability among the models are focused. In MDD, a model of high-level specification is systematically transformed into the lower-level models including source codes. The technique in this paper also generates trace models (traceability relationships) in the same way. The technique can be represented as follows according to our feature model. • • • • • •
Sources: Model elements including source codes. Destination: The same as sources. Assumption: MDD is performed. Traceability meta-models and their transformation rules are predefined. Consequence: Determinate. Process: Batch. Tool: Integrated as an Eclipse plugin.
If MDD is fully used in projects of a user, there is some possibility of introducing this technique. However, several additional tasks are required in advance such as writing traceability meta-models. If the user accept such additional tasks, this technique is suitable for him/her. 5. Conclusion In this paper, we proposed a feature model of traceability techniques. The model is designed so that the potential users of traceability techniques can choose a technique suitable for them. The model can be thus a basis of knowledge base for selecting suitable traceability techniques. Features of the model are chosen so that they can be obtained from published documents such as technical papers. Our feature model can be thus an answer of RQs mentioned in introduction. We also described model instances of several techniques on the basis of the feature model, and discussed which kinds of users were suitable for each technique. We finally summarize the problems and future issues. Of course, our first future issue is to implement the knowledge base based on our feature model. The main challenge is to describe and register the instances of traceability techniques. In reality, each proposed technique in technical papers has its life span. For example, tools of a technique proposed in old papers are not supported or opened to the public. We also take such life span into account when we develop the knowledge base. A unique reason for using traceability is the request by regulatory code or for certification. According to a survey 7, about 40 % of projects had such reason. Currently, we do not take such issue into account. Therefore, we have to examine the regulations and standards such as ISO 12207, ISO 26262 or SPICE. Our model is designed on the basis of papers published before September 2016 1. We thus have to describe model instances of the techniques in recent papers, and the model should be revised if it should be. Finally, we want to evaluate our model. Currently, we just described some model instances and assumed each of them would be suitable and effective in a certain project. The simplest way of the evaluation is to ask several practitioners to choose a technique by using the knowledge base. We then ask them to use the chosen technique and to report the effects. Otherwise, we ask researchers to investigate and to examine the activities of practitioners to explore the suitable techniques for the practitioners. In such a case, we will be able to obtain more detailed feedback from the researchers. References 1. Kaiya, H., Sato, R., Hazeyama, A., Ogata, S., Okubo, T., Tanaka, T., et al. Preliminary systematic literature review of software and systems traceability. In: Knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 21st International Conference KES-2017, Marseille, France, 6-8 September 2017. 2017, p. 1141–1150. URL: https://doi.org/10.1016/j.procs.2017.08.152. doi:10.1016/j.procs.2017.08.152.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1083 9
2. Cleland-Huang, J., Gotel, O., Zisman, A.. Software and Systems Traceability. Springer Publishing Company, Incorporated; 2012. ISBN 1447122380, 9781447122388. 3. Tufail, H., Masood, M.F., Zeb, B., Azam, F., Anwar, M.W.. A systematic review of requirement traceability techniques and tools. In: 2017 2nd International Conference on System Reliability and Safety (ICSRS). 2017, p. 450–454. URL: https://doi.org/10.1109/ICSRS. 2017.8272863. 4. Torkar, R., Gorschek, T., Feldt, R., Svahnberg, M., Raja, U.A., Kamran, K.. Requirements traceability: a systematic review and industry case study. International Journal of Software Engineering and Knowledge Engineering 2012;22(3):385–434. URL: https://doi.org/ 10.1142/S021819401250009X. doi:10.1142/S021819401250009X. 5. Cleland-Huang, J., Gotel, O., Hayes, J.H., M¨ader, P., Zisman, A.. Software traceability: trends and future directions. In: Proceedings of the on Future of Software Engineering, FOSE 2014, Hyderabad, India, May 31 - June 7, 2014. 2014, p. 55–69. URL: https://doi.org/ 10.1145/2593882.2593891. doi:10.1145/2593882.2593891. 6. Maro, S., Stegh¨ofer, J., Staron, M.. Software traceability in the automotive domain: Challenges and solutions. Journal of Systems and Software 2018;141:85–110. URL: https://doi.org/10.1016/j.jss.2018.03.060. doi:10.1016/j.jss.2018.03.060. 7. Bouillon, E., M¨ader, P., Philippow, I.. A survey on usage scenarios for requirements traceability in practice. In: Requirements Engineering: Foundation for Software Quality - 19th International Working Conference, REFSQ 2013, Essen, Germany, April 8-11, 2013. Proceedings. 2013, p. 158–173. URL: https://doi.org/10.1007/978-3-642-37422-7\_12. doi:10.1007/978-3-642-37422-7\_12. 8. Wang, B.. Requirements traceability technologies selection for industry. In: 24th IEEE International Requirements Engineering Conference, RE 2016, Beijing, China, September 12-16, 2016. 2016, p. 450–455. URL: https://doi.org/10.1109/RE.2016.72. doi:10.1109/RE. 2016.72. 9. Wang, B., Peng, R., Li, Y., Lai, H., Wang, Z.. Requirements traceability technologies and technology transfer decision support: A systematic review. Journal of Systems and Software 2018;146:59–79. URL: https://doi.org/10.1016/j.jss.2018.09.001. doi:10. 1016/j.jss.2018.09.001. 10. Tsuchiya, R., Kato, T., Washizaki, H., Kawakami, M., Fukazawa, Y., Yoshimura, K.. Recovering traceability links between requirements and source code in the same series of software products. In: 17th International Software Product Line Conference, SPLC 2013, Tokyo, Japan - August 26 - 30, 2013. 2013, p. 121–130. URL: https://doi.org/10.1145/2491627.2491633. doi:10.1145/2491627.2491633. 11. Rompaey, B.V., Demeyer, S.. Establishing traceability links between unit test cases and units under test. In: 13th European Conference on Software Maintenance and Reengineering, CSMR 2009, Architecture-Centric Maintenance of Large-SCale Software Systems, Kaiserslautern, Germany, 24-27 March 2009. 2009, p. 209–218. URL: https://doi.org/10.1109/CSMR.2009.39. doi:10.1109/CSMR.2009.39. 12. Kang, S., Kim, J., Kang, S., Eom, S.. A formal representation of platform feature-to-requirement traceability for software product line development. In: IEEE 38th Annual Computer Software and Applications Conference, COMPSAC 2014, Vasteras, Sweden, July 21-25, 2014. 2014, p. 211–218. URL: https://doi.org/10.1109/COMPSAC.2014.29. doi:10.1109/COMPSAC.2014.29. ´ Marcos, E.. Dealing with traceability in the mddof model transformations. IEEE Trans Software 13. Vara, J.M., Bollati, V.A., Jim´enez, A., Eng 2014;40(6):555–583. URL: https://doi.org/10.1109/TSE.2014.2316132. doi:10.1109/TSE.2014.2316132. 14. Salman, H.E., Seriai, A., Dony, C.. Feature-to-code traceability in legacy software variants. In: 39th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2013, Santander, Spain, September 4-6, 2013. 2013, p. 57–61. URL: https://doi.org/ 10.1109/SEAA.2013.65. doi:10.1109/SEAA.2013.65. 15. Spanoudakis, G., Zisman, A.. Software traceability: A roadmap. Handbook of Software Engineering and Knowledge Engineering 2005; 3:395–428. doi:10.1142/9789812775245_0014. 16. Abadi, A., Nisenson, M., Simionovici, Y.. A traceability technique for specifications. In: The 16th IEEE International Conference on Program Comprehension, ICPC 2008, Amsterdam, The Netherlands, June 10-13, 2008. 2008, p. 103–112. URL: https://doi.org/10. 1109/ICPC.2008.30. doi:10.1109/ICPC.2008.30. 17. David, J., Koegel, M., Naughton, H., Helming, J.. Traceability rearmed. In: Proceedings of the 33rd Annual IEEE International Computer Software and Applications Conference, COMPSAC 2009, Seattle, Washington, USA, July 20-24, 2009. Volume 1. 2009, p. 340–348. URL: https://doi.org/10.1109/COMPSAC.2009.52. doi:10.1109/COMPSAC.2009.52. 18. Ali, N., Gu´eh´eneuc, Y., Antoniol, G.. Trust-based requirements traceability. In: The 19th IEEE International Conference on Program Comprehension, ICPC 2011, Kingston, ON, Canada, June 22-24, 2011. 2011, p. 111–120. URL: https://doi.org/10.1109/ICPC. 2011.42. doi:10.1109/ICPC.2011.42. 19. Asuncion, H.U., Asuncion, A.U., Taylor, R.N.. Software traceability with topic modeling. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1, ICSE 2010, Cape Town, South Africa, 1-8 May 2010. 2010, p. 95–104. URL: https://doi.org/10.1145/1806799.1806817. doi:10.1145/1806799.1806817. 20. Hayashi, S., Yoshikawa, T., Saeki, M.. Sentence-to-code traceability recovery with domain ontologies. In: 17th Asia Pacific Software Engineering Conference, APSEC 2010, Sydney, Australia, November 30 - December 3, 2010. 2010, p. 385–394. URL: https://doi.org/ 10.1109/APSEC.2010.51. doi:10.1109/APSEC.2010.51. 21. Linsbauer, L., Lopez-Herrejon, R.E., Egyed, A.. Recovering traceability between features and code in product variants. In: 17th International Software Product Line Conference, SPLC 2013, Tokyo, Japan - August 26 - 30, 2013. 2013, p. 131–140. URL: https: //doi.org/10.1145/2491627.2491630. doi:10.1145/2491627.2491630. 22. Pfeiffer, R., Wasowski, A.. An aspect-based traceability mechanism for domain specific languages. In: Proceedings of the 6th ECMFA Traceability Workshop, ECMFA-TW 2010, Paris, France, June 15, 2010. 2010, p. 47–52. URL: https://doi.org/10.1145/1814392. 1814399. doi:10.1145/1814392.1814399. 23. Kaiya, H., Hara, K., Kobayashi, K., Osada, A., Kaijiri, K.. Exploring how to support software revision in software non-intensive projects using existing techniques. In: Workshop Proceedings of the 35th Annual IEEE International Computer Software and Applications Conference, COMPSAC Workshops 2011, Munich, Germany, 18-22 July 2011. 2011, p. 327–334. URL: https://doi.org/10.1109/ COMPSACW.2011.61. doi:10.1109/COMPSACW.2011.61. 24. Capobianco, G., Lucia, A.D., Oliveto, R., Panichella, A., Panichella, S.. Traceability recovery using numerical analysis. In: 16th Working Conference on Reverse Engineering, WCRE 2009, 13-16 October 2009, Lille, France. 2009, p. 195–204. URL: https://doi.org/10.
1084 10
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1109/WCRE.2009.14. doi:10.1109/WCRE.2009.14. 25. Chen, X., Grundy, J.C.. Improving automated documentation to code traceability by combining retrieval techniques. In: 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), Lawrence, KS, USA, November 6-10, 2011. 2011, p. 223–232. URL: https://doi.org/10.1109/ASE.2011.6100057. doi:10.1109/ASE.2011.6100057. 26. Alhindawi, N., Meqdadi, O., Bartman, B., Maletic, J.I.. A tracelab-based solution for identifying traceability links using LSI. In: 7th International Workshop on Traceability in Emerging Forms of Software Engineering, TEFSE 2013, 19 May, 2013, San Francisco, CA, USA. 2013, p. 79–82. URL: https://doi.org/10.1109/TEFSE.2013.6620159. doi:10.1109/TEFSE.2013.6620159. 27. Nishikawa, K., Washizaki, H., Fukazawa, Y., Oshima, K., Mibe, R.. Recovering transitive traceability links among software artifacts. In: 2015 IEEE International Conference on Software Maintenance and Evolution, ICSME 2015, Bremen, Germany, September 29 - October 1, 2015. 2015, p. 576–580. URL: https://doi.org/10.1109/ICSM.2015.7332517. doi:10.1109/ICSM.2015.7332517. 28. Mirakhorli, M., Cleland-Huang, J.. Transforming trace information in architectural documents into re-usable and effective traceability links. In: Proceedings of the 6th International Workshop on SHAring and Reusing Architectural Knowledge, SHARK 2011, Waikiki, Honolulu, HI, USA, May 24, 2011. 2011, p. 45–52. URL: https://doi.org/10.1145/1988676.1988685. doi:10.1145/1988676.1988685. 29. Panichella, A., Lucia, A.D., Zaidman, A.. Adaptive user feedback for ir-based traceability recovery. In: 8th IEEE/ACM International Symposium on Software and Systems Traceability, SST 2015, Florence, Italy, May 17, 2015. 2015, p. 15–21. URL: https://doi.org/10. 1109/SST.2015.10. doi:10.1109/SST.2015.10.
Available online at www.sciencedirect.com
Available online at www.sciencedirect.com
ScienceDirect Procedia Computer Science 00 (2019) 000–000 Procedia Computer Science 159 (2019) 1075–1084 Procedia Computer Science 00 (2019) 000–000
www.elsevier.com/locate/procedia www.elsevier.com/locate/procedia
23rd International Conference on Knowledge Based and Intelligent Information and Engineering 23rd International Conference on KnowledgeSystems Based and Intelligent Information and Engineering Systems
Towards A Knowledge Base for Software Developers to Choose Towards A Knowledge for Software Developers to Choose SuitableBase Traceability Techniques Suitable Traceability Techniques a* b c d Haruhiko Kaiya , Atsuo Hazeyama , Shinpei Ogata , Takao Okubo , Nobukazu e b f , Hironori Haruhiko Kaiyaa*, AtsuoYoshioka Hazeyama , ShinpeiWashizaki Ogatac, Takao Okubod, Nobukazu e a Kanagawa University, HiratsukaWashizaki 259-1293, Japanf Yoshioka , Hironori b Tokyo Gakugei University, Tokyo 84-8501, Japan a Kanagawa University, Hiratsuka 259-1293, Japan c Shinshu University, Nagano 380-0928, Japan b Tokyo University, Tokyo 84-8501, Japan dGakugei IISEC, Yokohama 221-0835, Japan c Shinshu University, Nagano 380-0928, Japan e NII, Tokyo 100-0003, Japan d IISEC, Yokohama 221-0835, Japan f Waseda University, Tokyo 169-8555 , Japan e NII, Tokyo 100-0003, Japan f Waseda
University, Tokyo 169-8555 , Japan
Abstract Abstract Huge amount of techniques for creating, maintaining and/or recovering traceability among software development artifacts have been proposed. It is thus not easy for a potential user of such techniques to choose a technique suitable for his/her project because Huge amount of techniques for creating, maintaining and/or recovering traceability among software development artifacts have too many techniques exist and projects are different from each other. In this paper, we proposed a model for characterizing a been proposed. It is thus not easy for a potential user of such techniques to choose a technique suitable for his/her project because traceability technique with respect to its users. The model can become a meta-model of the knowledge base for such users. The too many techniques exist and projects are different from each other. In this paper, we proposed a model for characterizing a model is designed on the basis of the contents of existing technical papers so that we can easily describe model instances on the traceability technique with respect to its users. The model can become a meta-model of the knowledge base for such users. The basis of technical papers. The model is represented in a feature model, and it has four mandatory features: source, destination, model is designed on the basis of the contents of existing technical papers so that we can easily describe model instances on the consequence and process. The user can at least understand a technique is unsuitable for him/her by referring such features. If a basis of technical papers. The model is represented in a feature model, and it has four mandatory features: source, destination, technique estimates the traceability relationships, the instance of its feature model may contain the quality metrics of its estimation consequence and process. The user can at least understand a technique is unsuitable for him/her by referring such features. If a such as precision and recall. It has also several optional features: assumptions, preprocess and tool. Although we sometimes technique estimates the traceability relationships, the instance of its feature model may contain the quality metrics of its estimation cannot obtain such optional features from technical papers, they are so helpful for a user to decide a technique suitable for him/her. such as precision and recall. It has also several optional features: assumptions, preprocess and tool. Although we sometimes On the basis of the feature model, we described model instances of several techniques in technical papers. We also examined and cannot obtain such optional features from technical papers, they are so helpful for a user to decide a technique suitable for him/her. discussed who the suitable user for each technique is. On the basis of the feature model, we described model instances of several techniques in technical papers. We also examined and c 2019 The Authors. Published by Elsevier B.V. discussed who the suitable user for each technique © 2019 The Authors. Published by B.V. is. Peer-review under responsibility of Elsevier KES International. c 2019 The Authors. by Elsevier B.V. This is an open accessPublished article under the CC BY-NC-ND license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review Software under of International. under responsibility responsibility of KES KES International. Keywords: Traceability, Feature Model, User Perspective, Knowledge Base, Survey Keywords: Software Traceability, Feature Model, User Perspective, Knowledge Base, Survey
a a
Corresponding author. Tel.: +81-463-59-4111 E-mail address: [email protected] Corresponding author. Tel.: +81-463-59-4111 E-mail address: [email protected]
c 2019 The Authors. Published by Elsevier B.V. 1877-0509 1877-0509 2019responsibility The Authors. of Published by Elsevier B.V. Peer-review©under KES International. This is an open access article under the CC BY-NC-ND c 2019 1877-0509 The Authors. Published by Elsevier B.V. license (https://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review underresponsibility responsibility of KES International. Peer-review under of KES International. 10.1016/j.procs.2019.09.276
1076 2
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1. Introduction Introducing a software traceability technique in a software development project is crucial for many reasons 1 . For example, most software products are developed based on the previous version of each product. To know the impact on design and source codes by requirements changes, traceability relationships are required. All software products today are very huge. Even a small command such as info-zip contains 60 thousand lines of codes. Some operating system consists of more than 100 million lines of codes. To understand such huge products, relationships among the same level of products such as among codes are crucial. Relationships among different levels such as requirements and codes are also crucial. Traceability techniques contribute to improving such understandability even if the product has only one version. Traceability techniques also contribute to checking completeness and correctness of a product and finding reusable components in a product. Because of the importance of traceability techniques, huge amount of papers have been published and consumed by researchers. In most papers, comparison between a proposed technique and others is shown, and the superiority of the proposed one is loudly presented in certain criteria. Several benchmark problems 2 are shared among researchers, and used for such comparison. However, primal users of traceability techniques, i.e. software developers, are not interested in such feeble improvement. They want to know which technique is valuable and suitable for their projects. Projects are different from each other. For example, a rigorous model driven development is performed in a project. On the other hand in another project, artifacts such as requirements and design documents are written in natural language, and their changes are often notified and shared by email. According to the background mentioned above, we setup the following research questions (RQs). • RQ1: What kinds of information about a traceability technique are required for its user to decide whether it is suitable for his/her project? • RQ2: What kinds of such information can be obtained from published documents such as technical papers? On the basis of our survey 1, we propose a feature model of a traceability technique as answers of the RQs. The model can be a basis of the knowledge base for software developers to choose traceability techniques suitable for their project. Although there already exists some frameworks for categorizing traceability techniques 1,3,4 , they do not intend to help users of the techniques to choose suitable ones. Such frameworks usually intend to improve and progress the research activities. The rest of this paper is organized as follows. In the next section, related works are briefly reviewed and discussed. On the basis of papers in our previous survey 1, we introduce a feature model of traceability techniques in section 3. Because the contents of technical papers are limited, we choose features acquirable from such papers. In section 4, several model instances based on actual papers are shown. Finally, we summarize our current results and show the future issues.
2. Related Work Plenty of techniques for maintaining, developing and recovering traceability relationships among software artifacts are proposed in huge number of technical papers. Their survey have been also summarized in several papers 1,3,4,5,6 . However, few papers focus on how to choose suitable technique(s) for each user. In a paper 5 , understanding stakeholders and traceability strategizing techniques are focused as future research directions. Although both directions are slightly related to traceability selection, the paper did not mention such relationships at least. In a survey 7 , usage scenarios of requirements traceability are summarized. However, how to choose the technique in each usage is not investigated. Bangchao Wang et al 8,9 focus on technology transfer from academy to industry. They proposed a maturity model of a technology whether it can be transferred to industry. The model contains many aspects of a technology such as usability, usefulness, customizability, quality, risk and cost. Although the model seems to be reasonable, it is not realistic to gather such many aspects only from technical papers related to a technology.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1077 3
Traceability Technique
source
destination
meaning
Legends
assumption
consequence
determinate
estimate
preprocess
batch
process
interactive
none
tool
parts
integrated
Mandatory quality
Optional
participation
Alternative precision
recall
tradeoff
Fig. 1. A Feature Model
3. A Feature Model of Traceability Techniques According to papers mentioned in our survey 1 , we develop a feature model of traceability techniques in Figure 1. As mentioned in introduction, there is no frameworks for software developers to choose a traceability technique so that they try to use it. Some kind of knowledge base of traceability techniques will contribute to helping such developers. The model in Figure 1 is designed so that the model can become a meta-model of such knowledge base. Before explaining each feature, we explain the criteria of selecting features in the model. We also explain the difference between mandatory and optional features in this model. We only focus on features that can be obtained from existing technical papers. Therefore, the kinds of features in our model are not complete, but model instances are usually described without missing features. To help software developers to choose their technique easily and suitably, necessary information for using each technique should be provided as much as possible. Features in the model are also chosen on the basis of such criteria. Instead, technical details how to establish traceability are intentionally excluded from this model even if such information can be obtained from papers. For example, we regard kinds of information retrieval methods such as Support Vector Machine (SVM) or Latent Semantic Indexing (LSI) are not necessary in this model. We regard developers cannot choose suitable technique without mandatory features in the model. Characteristics of a project using a traceability technique are not directly specified. Instead, assumptions for using a technique indirectly specify projects suitable for the technique as shown in section 3.4. Because we expect users in a project choose their technique(s) by themselves or they participate in selecting technique(s), such assumptions will contribute to selecting technique(s) suitable for the project. 3.1. Source (Mandatory) A traceability technique creates relationships between parts of software development elements. Examples of such parts are a paragraph in a requirements specification 10 , a Java class 11 , a feature of a software product line 12 , and UML model elements 13 . In some case, a member of a project, i.e. people, can be a source of a traceability technique. A model of the technique thus has to specify one of such parts. 3.2. Destination (Mandatory) The feature “destination” specifies another end of a traceability relationship. In many cases, it is not clearly specified that a traceability relationship has direction. Once a directional traceability relationship is established, its reverse direction can be easily added. Therefore, we do not distinguish the source and destination features. These features simply specify one end and another of a traceability relationship.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1078 4 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
1
2 recall
3 precision
4
5
6
7
some parameter
Fig. 2. An example of precision curve and recall curve
3.3. Meaning A traceability relationship creating and maintaining a technique sometimes has its specific usage. Suppose a feature in a software product line is implemented as a class of Java, and a technique maintains such relationships between features and codes 14 . In such a case, the user of the technique can identify features which are not implemented in codes of a specific application. We regard the meaning of the traceability is “implement” in our model. Other examples of the meaning are “refinement”, “design rationale” and “impact”. Kinds of traceability links such as overlap, satisfiability, conflict and dependency are enumerated in a handbook 15. Such links types are useful to specify the meaning. When such meaning is specified, potential users of the technique can easily examine and choose the technique. According to a survey 7, about 80 % of projects applied traceability techniques because they expect their benefits. The feature meaning clarifies such benefits to users. However, in several papers, such meaning is not clearly specified in each technique 1. 3.4. Assumption There is no general and standard software development project. Each traceability technique thus has several assumptions about a development project where the technique is used. When a user examine and choose existing traceability techniques, these assumptions are very important. If a project of the user does not satisfy such assumptions, the technique cannot be applied to the project. Otherwise, the project has to be modified so that the technique can be applied. The assumptions thus help users to estimate potential costs of introducing the technique. Examples of such assumptions are as follows. In some techniques, information retrieval techniques are applied to documents, and many of them assume the documents are written in English 16 . The naming convention of classes is sometimes assumed 11 . Several techniques 17,18,19,10 require software development history or logs. A technique 20 assumes domain ontology that are used for semantic matching among words and terms. Development styles such as software product line 12,21 and model driven development 13 are also assumed for each technique. When traceability relationships are directly and manually created and maintained, some additional tasks are added to its software development. Such additional tasks are also kinds of assumptions. For example, using some domain specific language 22 is a kind of assumptions. Scalability of a traceability technique is important in general 9 . However, general descriptions or metrics of scalability do not help users to choose a technique. Instead, assumptions exemplified above let users know whether a technique has a certain level of scalability suitable for them. 3.5. Consequence (Mandatory) Each technique recovers or develops traceability relationships. If a technique recovers the relationships on the basis of a probabilistic approach such as information retrieval methods, not all relationships are correct in general. On the other hand, all the relationships are basically correct if a technique enforces some notations and tasks to maintain the
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1079 5
1 0.9
precision
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 0
0.2
0.4
0.6
0.8
1
recall Fig. 3. An example of precision/recall curve
relationships. For users of the technique, we have to clarify whether a technique is first or last type. We call the first type as “estimate”, and the second on as “determinate”. For the first type, we also have to specify the quality of estimated relationships. Following two metrics are useful to know the quality of a technique. Precision = Recall =
| (estimated relationships) ∩ (correct relationships) | | estimated relationships |
| (estimated relationships) ∩ (correct relationships) | | correct relationships |
Note that | relationships | is the number of relationships. Composite metrics such as F-measure are not useful for people who want to choose a traceability techniques because it does not let them know two different criteria respectively; one is how much wrong relationships are contained (false positive) in the results by the technique and another is how much right relationships are missing (false negative) by the technique. The tradeoffs between precision and recall are also very useful for technique choosers. In some research 23 , some parameter is used and changes of recall and precision values are shown along the progress of the parameter as shown in Figure 2. Note that values of the vertical axis in the figure take 0 to 1. If the value takes 1, precision or recall is perfect, i.e. no false positive or no false negative. If the value of precision or recall takes 0, all predicted results are false positive or false negative. Precision/recall curves as exemplified in Figure 3 are more useful than the changes of precision and recall respectively because users can directly know the tradeoffs between precision and recall. Note that values of the both axes in the figure are the same as the values of the vertical axis in Figure 2. Because metrics such as precision and recall are calculated on the basis of some data such as benchmark examples, the values do not show the quality of a technique in general. Therefore, it is better to explain the characteristics of such data, e.g. size, complexity and artifacts in a class room or an actual project. In the context of writing technical papers, comparing a technique to others are very important. However, such kind of comparison is not so important in the context of using techniques because the absolute level of the quality is important. For example, it is meaningless for users that the recall of a technique becomes 1.2 times better than existing one if the absolute values of the recall are only 10% and 12%. It is too terrible that about 80 % of correct relationships are missing by the technique. 3.6. Preprocess In the case of estimated techniques, several preprocessing tasks are required before recovering traceability relationships. Such tasks are normally negative factors of introducing a technique because software developers have to perform additional tasks. Although some preprocessing tasks can be performed automatically, such automation normally negatively affects the quality of the preprocessed data. Especially for textual artifacts such requirements documents, such automation is not so easy. For example, tasks such as sectioning documents, unifying terms, removing general terms 16,24,20,18,25,26,27 are typically required and they are not fully automated. Other types of preprocessing
1080 6
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
are setting the list of keywords 14,28,10 and also setting some value as the threshold 14. There is a tradeoff between the amount of preprocessing tasks and the degree of processing automation. In the case of determinate techniques, there are few preprocessing tasks because tasks for creating and maintaining traceability relationships are embedded in the software development process itself. In our model, such issue is noted in the “assumption” feature mentioned above. 3.7. Process (Mandatory) In each technique, several tasks are performed for creating, maintaining and/or recovering traceability relationships. The user of such techniques is interested in what and how many additional operations he/she has to perform. We call a technique performs a “batch” process if the user performs nothing for traceability during the process. On the other hand, a technique performs an “interactive” process, and its user has to do something during the process. A typical operation in an interactive process is user’s judgement. In a technique 29 , users give their judgement of classified traceability relationships. 3.8. Tool If tools for performing a traceability technique exist, the number of its potential users increases. We thus specify the status of such tools for each technique. With respect to tools, we categorize techniques into the following three types: none, parts, integrated. If a technique or its papers state there is no tool support, we categorize the technique is “none”. If they do not introduce or do not mention any tools, this feature is not specified. When some pieces of tools are used in a technique, we categorized it is “parts”. For example, Java packages for vector space model, LSI and Sufficient Dimensionality Reduction (SDR) implementations in MATLAB are used in a technique 16. Several techniques provide integrated tools to perform them. In such cases, we call a technique is “integrated”. Some tools are stand alone 12 , and others are built on a well-known IDE such as Eclipse 20,25 . 4. Model Instances In this section, we describe several model instances of traceability techniques on the basis of existing technical papers. We also discuss for what kind of users each technique is suitable. 4.1. Information Retrieval based Traceability Recovery using User Feedback In a paper 29 , traceability between documents are recovered on the basis of their textual similarity. Source: document Destination: document Assumption: Artifact types such as requirements, design, test and codes are identified. Consequence: Estimate. – Precision: Yes. – Recall: Yes. – Tradeoff: Yes, in the same way as shown in Figure 3. Data of following three projects were used to calculate them: Easy-Clinic, iTrust, Moderate Resolution Imaging Spectroradiometer (MODIS). First two projects are contents for university education. The last one is an open source software by NASA. Project Web pages introduced in this paper do not exist in March 2019. • Process: Interactive. A user has to judge whether first candidate of a traceability relationship is correct or not repeatedly. • • • •
If a user can perform the judgement mentioned in this technique, this technique is suitable for him/her. Unfortunately, the paper does not mention tool supports although empirical evaluation is shown. Therefore, the user has to manually perform the technique, or to develop its automated tool.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1081 7
4.2. Change Impact Analysis for Software Revision in a software non-intensive project In a paper 23 , software revision due to requirements changes is focused, and two techniques of estimating traceability between documents and source codes are proposed. The first technique is developed on the basis of answers of questionnaires from practitioners. The second one is developed on the basis of the feedback about the experiences of the first one by the practitioners. Because software is not a central part in projects the technique will be applied, they cannot easily introduce software development methods such as software product line techniques and model driven development. According to our feature model in Figure 1, we characterized the first technique as follows. • • • • •
• • •
Source: Sections in requirements document. Destination: Functions in source codes. Meaning: Change impacts. Assumption: Users can specify suitable keywords and terms corresponding to a requirements change. Indirect impacts are out of scope. Consequence: Estimate. The quality features are as follows: – Precision: Yes. – Recall: Yes. – Tradeoff: Yes. Using the number of query keywords, the tradeoff is depicted in the same way in Figure 2. Data in actual industrial projects were used to calculate the metrics above. The detail of the projects were not published. Preprocess: Several samples of actual impacted codes should be manually identified. Process: Interactive. Several trials are required. During the trial, keywords may be updated. Pseudo-precision and recall are provided on the basis of samples defined in the preprocess so that the user can judge whether the keywords should be updated or not. Tool: integrated. The tool is developed as an independent application.
A model instance of the second technique is also as follows. • • • • •
• •
•
Source: Sections in requirements document. Destination: Functions in source codes. Meaning: Change impacts. Assumption: Users can choose suitable keywords and terms corresponding to a requirements change from the candidates provided by the technique. The technique provides the candidates of the keywords. Consequence: Estimate. The quality features are as follows: – Precision: Yes. – Recall: Yes. – Tradeoff: No. Data in actual industrial projects were also used to calculate the metrics above. The detail of the projects were not also published. Preprocess: Several samples of actual impacted codes should be manually identified. Process: Interactive. Several trials are required. During the trial, keywords may be updated. Pseudo-precision and recall are provided on the basis of samples defined in the preprocess so that the user can judge whether the keywords should be updated or not. The technique recommends the candidates of new additional samples so that the number of samples increases. Tool: integrated. The tool is developed as an independent application.
The amount of user participation decreases when the technique is update from the first technique to second one. For example, the user simply chooses the keywords from candidates while he/she has to directly specify them in the first one. The second technique makes the number of samples for calculating pseudo-precision and recall increases so that the pseudo-metrics become more actual than ever.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1082 8
In the techniques, the meaning of traceability relationships is clearly defined. Therefore, users can easily decide whether the techniques are suitable for them. In addition, the amount of the user participation is clearly stated. The potential users can also discuss whether they can accept additional efforts caused by the user participation. 4.3. MDD and Model Transformation In a paper 13 , Model-driven development (MDD) and the traceability among the models are focused. In MDD, a model of high-level specification is systematically transformed into the lower-level models including source codes. The technique in this paper also generates trace models (traceability relationships) in the same way. The technique can be represented as follows according to our feature model. • • • • • •
Sources: Model elements including source codes. Destination: The same as sources. Assumption: MDD is performed. Traceability meta-models and their transformation rules are predefined. Consequence: Determinate. Process: Batch. Tool: Integrated as an Eclipse plugin.
If MDD is fully used in projects of a user, there is some possibility of introducing this technique. However, several additional tasks are required in advance such as writing traceability meta-models. If the user accept such additional tasks, this technique is suitable for him/her. 5. Conclusion In this paper, we proposed a feature model of traceability techniques. The model is designed so that the potential users of traceability techniques can choose a technique suitable for them. The model can be thus a basis of knowledge base for selecting suitable traceability techniques. Features of the model are chosen so that they can be obtained from published documents such as technical papers. Our feature model can be thus an answer of RQs mentioned in introduction. We also described model instances of several techniques on the basis of the feature model, and discussed which kinds of users were suitable for each technique. We finally summarize the problems and future issues. Of course, our first future issue is to implement the knowledge base based on our feature model. The main challenge is to describe and register the instances of traceability techniques. In reality, each proposed technique in technical papers has its life span. For example, tools of a technique proposed in old papers are not supported or opened to the public. We also take such life span into account when we develop the knowledge base. A unique reason for using traceability is the request by regulatory code or for certification. According to a survey 7, about 40 % of projects had such reason. Currently, we do not take such issue into account. Therefore, we have to examine the regulations and standards such as ISO 12207, ISO 26262 or SPICE. Our model is designed on the basis of papers published before September 2016 1. We thus have to describe model instances of the techniques in recent papers, and the model should be revised if it should be. Finally, we want to evaluate our model. Currently, we just described some model instances and assumed each of them would be suitable and effective in a certain project. The simplest way of the evaluation is to ask several practitioners to choose a technique by using the knowledge base. We then ask them to use the chosen technique and to report the effects. Otherwise, we ask researchers to investigate and to examine the activities of practitioners to explore the suitable techniques for the practitioners. In such a case, we will be able to obtain more detailed feedback from the researchers. References 1. Kaiya, H., Sato, R., Hazeyama, A., Ogata, S., Okubo, T., Tanaka, T., et al. Preliminary systematic literature review of software and systems traceability. In: Knowledge-Based and Intelligent Information & Engineering Systems: Proceedings of the 21st International Conference KES-2017, Marseille, France, 6-8 September 2017. 2017, p. 1141–1150. URL: https://doi.org/10.1016/j.procs.2017.08.152. doi:10.1016/j.procs.2017.08.152.
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1083 9
2. Cleland-Huang, J., Gotel, O., Zisman, A.. Software and Systems Traceability. Springer Publishing Company, Incorporated; 2012. ISBN 1447122380, 9781447122388. 3. Tufail, H., Masood, M.F., Zeb, B., Azam, F., Anwar, M.W.. A systematic review of requirement traceability techniques and tools. In: 2017 2nd International Conference on System Reliability and Safety (ICSRS). 2017, p. 450–454. URL: https://doi.org/10.1109/ICSRS. 2017.8272863. 4. Torkar, R., Gorschek, T., Feldt, R., Svahnberg, M., Raja, U.A., Kamran, K.. Requirements traceability: a systematic review and industry case study. International Journal of Software Engineering and Knowledge Engineering 2012;22(3):385–434. URL: https://doi.org/ 10.1142/S021819401250009X. doi:10.1142/S021819401250009X. 5. Cleland-Huang, J., Gotel, O., Hayes, J.H., M¨ader, P., Zisman, A.. Software traceability: trends and future directions. In: Proceedings of the on Future of Software Engineering, FOSE 2014, Hyderabad, India, May 31 - June 7, 2014. 2014, p. 55–69. URL: https://doi.org/ 10.1145/2593882.2593891. doi:10.1145/2593882.2593891. 6. Maro, S., Stegh¨ofer, J., Staron, M.. Software traceability in the automotive domain: Challenges and solutions. Journal of Systems and Software 2018;141:85–110. URL: https://doi.org/10.1016/j.jss.2018.03.060. doi:10.1016/j.jss.2018.03.060. 7. Bouillon, E., M¨ader, P., Philippow, I.. A survey on usage scenarios for requirements traceability in practice. In: Requirements Engineering: Foundation for Software Quality - 19th International Working Conference, REFSQ 2013, Essen, Germany, April 8-11, 2013. Proceedings. 2013, p. 158–173. URL: https://doi.org/10.1007/978-3-642-37422-7\_12. doi:10.1007/978-3-642-37422-7\_12. 8. Wang, B.. Requirements traceability technologies selection for industry. In: 24th IEEE International Requirements Engineering Conference, RE 2016, Beijing, China, September 12-16, 2016. 2016, p. 450–455. URL: https://doi.org/10.1109/RE.2016.72. doi:10.1109/RE. 2016.72. 9. Wang, B., Peng, R., Li, Y., Lai, H., Wang, Z.. Requirements traceability technologies and technology transfer decision support: A systematic review. Journal of Systems and Software 2018;146:59–79. URL: https://doi.org/10.1016/j.jss.2018.09.001. doi:10. 1016/j.jss.2018.09.001. 10. Tsuchiya, R., Kato, T., Washizaki, H., Kawakami, M., Fukazawa, Y., Yoshimura, K.. Recovering traceability links between requirements and source code in the same series of software products. In: 17th International Software Product Line Conference, SPLC 2013, Tokyo, Japan - August 26 - 30, 2013. 2013, p. 121–130. URL: https://doi.org/10.1145/2491627.2491633. doi:10.1145/2491627.2491633. 11. Rompaey, B.V., Demeyer, S.. Establishing traceability links between unit test cases and units under test. In: 13th European Conference on Software Maintenance and Reengineering, CSMR 2009, Architecture-Centric Maintenance of Large-SCale Software Systems, Kaiserslautern, Germany, 24-27 March 2009. 2009, p. 209–218. URL: https://doi.org/10.1109/CSMR.2009.39. doi:10.1109/CSMR.2009.39. 12. Kang, S., Kim, J., Kang, S., Eom, S.. A formal representation of platform feature-to-requirement traceability for software product line development. In: IEEE 38th Annual Computer Software and Applications Conference, COMPSAC 2014, Vasteras, Sweden, July 21-25, 2014. 2014, p. 211–218. URL: https://doi.org/10.1109/COMPSAC.2014.29. doi:10.1109/COMPSAC.2014.29. ´ Marcos, E.. Dealing with traceability in the mddof model transformations. IEEE Trans Software 13. Vara, J.M., Bollati, V.A., Jim´enez, A., Eng 2014;40(6):555–583. URL: https://doi.org/10.1109/TSE.2014.2316132. doi:10.1109/TSE.2014.2316132. 14. Salman, H.E., Seriai, A., Dony, C.. Feature-to-code traceability in legacy software variants. In: 39th Euromicro Conference on Software Engineering and Advanced Applications, SEAA 2013, Santander, Spain, September 4-6, 2013. 2013, p. 57–61. URL: https://doi.org/ 10.1109/SEAA.2013.65. doi:10.1109/SEAA.2013.65. 15. Spanoudakis, G., Zisman, A.. Software traceability: A roadmap. Handbook of Software Engineering and Knowledge Engineering 2005; 3:395–428. doi:10.1142/9789812775245_0014. 16. Abadi, A., Nisenson, M., Simionovici, Y.. A traceability technique for specifications. In: The 16th IEEE International Conference on Program Comprehension, ICPC 2008, Amsterdam, The Netherlands, June 10-13, 2008. 2008, p. 103–112. URL: https://doi.org/10. 1109/ICPC.2008.30. doi:10.1109/ICPC.2008.30. 17. David, J., Koegel, M., Naughton, H., Helming, J.. Traceability rearmed. In: Proceedings of the 33rd Annual IEEE International Computer Software and Applications Conference, COMPSAC 2009, Seattle, Washington, USA, July 20-24, 2009. Volume 1. 2009, p. 340–348. URL: https://doi.org/10.1109/COMPSAC.2009.52. doi:10.1109/COMPSAC.2009.52. 18. Ali, N., Gu´eh´eneuc, Y., Antoniol, G.. Trust-based requirements traceability. In: The 19th IEEE International Conference on Program Comprehension, ICPC 2011, Kingston, ON, Canada, June 22-24, 2011. 2011, p. 111–120. URL: https://doi.org/10.1109/ICPC. 2011.42. doi:10.1109/ICPC.2011.42. 19. Asuncion, H.U., Asuncion, A.U., Taylor, R.N.. Software traceability with topic modeling. In: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1, ICSE 2010, Cape Town, South Africa, 1-8 May 2010. 2010, p. 95–104. URL: https://doi.org/10.1145/1806799.1806817. doi:10.1145/1806799.1806817. 20. Hayashi, S., Yoshikawa, T., Saeki, M.. Sentence-to-code traceability recovery with domain ontologies. In: 17th Asia Pacific Software Engineering Conference, APSEC 2010, Sydney, Australia, November 30 - December 3, 2010. 2010, p. 385–394. URL: https://doi.org/ 10.1109/APSEC.2010.51. doi:10.1109/APSEC.2010.51. 21. Linsbauer, L., Lopez-Herrejon, R.E., Egyed, A.. Recovering traceability between features and code in product variants. In: 17th International Software Product Line Conference, SPLC 2013, Tokyo, Japan - August 26 - 30, 2013. 2013, p. 131–140. URL: https: //doi.org/10.1145/2491627.2491630. doi:10.1145/2491627.2491630. 22. Pfeiffer, R., Wasowski, A.. An aspect-based traceability mechanism for domain specific languages. In: Proceedings of the 6th ECMFA Traceability Workshop, ECMFA-TW 2010, Paris, France, June 15, 2010. 2010, p. 47–52. URL: https://doi.org/10.1145/1814392. 1814399. doi:10.1145/1814392.1814399. 23. Kaiya, H., Hara, K., Kobayashi, K., Osada, A., Kaijiri, K.. Exploring how to support software revision in software non-intensive projects using existing techniques. In: Workshop Proceedings of the 35th Annual IEEE International Computer Software and Applications Conference, COMPSAC Workshops 2011, Munich, Germany, 18-22 July 2011. 2011, p. 327–334. URL: https://doi.org/10.1109/ COMPSACW.2011.61. doi:10.1109/COMPSACW.2011.61. 24. Capobianco, G., Lucia, A.D., Oliveto, R., Panichella, A., Panichella, S.. Traceability recovery using numerical analysis. In: 16th Working Conference on Reverse Engineering, WCRE 2009, 13-16 October 2009, Lille, France. 2009, p. 195–204. URL: https://doi.org/10.
1084 10
Haruhiko Kaiya et al. / Procedia Computer Science 159 (2019) 1075–1084 H. Kaiya / Procedia Computer Science 00 (2019) 000–000
1109/WCRE.2009.14. doi:10.1109/WCRE.2009.14. 25. Chen, X., Grundy, J.C.. Improving automated documentation to code traceability by combining retrieval techniques. In: 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011), Lawrence, KS, USA, November 6-10, 2011. 2011, p. 223–232. URL: https://doi.org/10.1109/ASE.2011.6100057. doi:10.1109/ASE.2011.6100057. 26. Alhindawi, N., Meqdadi, O., Bartman, B., Maletic, J.I.. A tracelab-based solution for identifying traceability links using LSI. In: 7th International Workshop on Traceability in Emerging Forms of Software Engineering, TEFSE 2013, 19 May, 2013, San Francisco, CA, USA. 2013, p. 79–82. URL: https://doi.org/10.1109/TEFSE.2013.6620159. doi:10.1109/TEFSE.2013.6620159. 27. Nishikawa, K., Washizaki, H., Fukazawa, Y., Oshima, K., Mibe, R.. Recovering transitive traceability links among software artifacts. In: 2015 IEEE International Conference on Software Maintenance and Evolution, ICSME 2015, Bremen, Germany, September 29 - October 1, 2015. 2015, p. 576–580. URL: https://doi.org/10.1109/ICSM.2015.7332517. doi:10.1109/ICSM.2015.7332517. 28. Mirakhorli, M., Cleland-Huang, J.. Transforming trace information in architectural documents into re-usable and effective traceability links. In: Proceedings of the 6th International Workshop on SHAring and Reusing Architectural Knowledge, SHARK 2011, Waikiki, Honolulu, HI, USA, May 24, 2011. 2011, p. 45–52. URL: https://doi.org/10.1145/1988676.1988685. doi:10.1145/1988676.1988685. 29. Panichella, A., Lucia, A.D., Zaidman, A.. Adaptive user feedback for ir-based traceability recovery. In: 8th IEEE/ACM International Symposium on Software and Systems Traceability, SST 2015, Florence, Italy, May 17, 2015. 2015, p. 15–21. URL: https://doi.org/10. 1109/SST.2015.10. doi:10.1109/SST.2015.10.