- Email: [email protected]
Combination of Test-Driven Development and Behavior-Driven Development for Improving Backend Testing Performance
Available online at www.sciencedirect.com
ScienceDirect Available online at www.sciencedirect.com Procedia Computer Science 00 (2019) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 157 (2019) 79–86
4th International Conference on Computer Science and Computational Intelligence 2019 (ICCSCI), 12–13 September 2019
Combination of Test-Driven Development and Behavior-Driven Development for Improving Backend Testing Performance Ida Bagus Kerthyayana Manuabaa,* a
Computer Science Program, Faculty of Computing and Media, Bina Nusantara University, Jakarta, Indonesia
Abstract In the process of software development, a testing phase is an important step for detecting the errors. In order to produce a reliable application, this paper discusses the utilization of a combination from two different testing methods, Test- Driven Development (TDD) and Behavior Driven Development (BDD). TDD testing method is a process with a high percentage of code which is tested automatically. However, it could lead to an error when the parameters of the features are changed. Meanwhile, the BDD testing method is a testing method that could perform successfully despite changes to the parameters of the features. This paper discusses the combination of TDD and BDD testing methods, which is called as T-BDD method. By implementing T-BDD into the Vixio backend system, this paper successfully shows that it could be performed to achieve a high percentage of test-coverage while it could also adapt when the parameters of the features were changed. This paper also shows that the T-BDD testing method performed better compared to using the TDD testing method only. © 2019 2019 The The Authors. Authors. Published Published by by Elsevier Elsevier B.V. B.V. © This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Peer-review under responsibility Computational Intelligence 2019. of the scientific committee of the 4th International Conference on Computer Science and Computational Intelligence 2019
Keywords: Unit Test; Test-Driven Development (TDD); Behavior-Driven Development(BDD); Backend; Web Application;
* Corresponding author. Tel.: +62-21-5345830. E-mail address: [email protected] 1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Computational Intelligence 2019
1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Computational Intelligence 2019. 10.1016/j.procs.2019.08.144
80 2
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
1. Introduction Nowadays, when developing a software system or software application, many software developers follow a number of specific stages known as software development methodologies1. There are a lot of examples of software development methodologies such as Agile2, Water Fall3, Lean Development4 and many more which have been applied as guidelines for software development process. For any software development methodology, a testing phase plays an important role in creating reliable software. Based on Gupta, et al5 in the book ‘Improvisation of Reusability Oriented Software Testing’, they stated that software testing plays an important role in evincing any errors that might occur during the development phases. In addition, software testing is also important to ensure the quality and performance of the product (software system/application)6. There are a lot of methods of software testing7. This paper discusses the usage and a combination of TDD and BDD testing methods to perform testing for backend system. As a continuation of previous work, these testing methods were applied into the development process of a backend system for the Vixio web application platform. Vixio is a platform that has been developed to create an environment for people to write and share an interactive fiction story8. In serving a fully functional code of the Vixio backend system, various things inside the system require tweaking, including the quality of the code and testing methods. For the unit test of the Vixio backend system, a TDD testing method has been applied. The testing steps were shown since the beginning of creating a new feature such as determining a goal, creating a unit test, and writing some code until the unit test showed no errors or failures. Although the test-coverage of the code was high, there was still a problem using TDD testing method. The problem occurred when the parameter values of the feature were changed, making the test function become error-prone, and the test function were needed to be updated or rewritten. For addressing this TDD problem in this backend testing scenario, a BDD testing method was applied into the test function. According to Shafiee9, a BDD testing method is a behavior tests of the program rather than the implementation. It means the input and output have become a dynamic shape according to the scenario of the program. Hence, by combining BDD and TDD testing methods, this hopefully could provide an evidence that these two testing methods could complement each other and produce in the best unit test result for the Vixio backend system. This paper will discuss in more detail about the implementation of the combination of TDD and BDD testing methods, which is referring to T-BDD testing, into the Vixio backend system. Further discussion is also presented in a theoretical foundation part, following by a system architecture part and a testing implementation and results part. Then the paper will continue with a discussion section and will close with a conclusion section. 2. Theoretical Foundation 2.1. Vixio backend system As mentioned previously, in order to apply a T-BDD testing method, the Vixio backend system was utilized for the testing object. Vixio is a web application platform that provide an interactive environment that allow users to write interactive stories and also share it with readers to be played10,11. The Vixio backend was developed using the Laravel framework. Laravel is an open source framework that uses PHP as programming language. The Laravel framework provides modules such as authentication, routing, sessions, and caching which are commonly used for a web applications12,13. Laravel also provides support for testing by using PHPUnit14,15. The testing includes JSON APIs for verifying the JSON and testing file uploads. PHPUnit is able to help a developer to test and debug each endpoint created in a form of unit test. The unit test is used to help the developer to design their code while maintaining the code safe from bugs and errors when making changes. 2.2. Test-Driven Development Test-Driven Development (TDD) is a testing method that obliges the developer to set the goals of the test before starting to write the code16. After the test is created, the developer is required to write the code to satisfy the test until no error or no failure produced. The stages of the TDD testing method can be seen in Figure 1 below:
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
81 3
Fig. 1. TDD testing method stage16
The process of using TDD are as follows: (1) set a goal for a new feature and implement it to a test function; (2) run the test that results in error or failure; (3) Write some code for that test function; (4) re-run the test function. If the result shows an error or failure, repeat the third step and refactor the code until there is no error or failure found; and (5) repeat the process of TDD for a new feature until there are no more features. According to Hartikainen17, the challenge in using the TDD testing method is when the developer needs to think what should be written in the test function when there is nothing to be tested. However, five tips that can help the developers using TDD testing method easier17, are (1) decide the inputs and output for that specific feature; (2) decide the parameters inside the test function; (3) divide the aspects of the feature and focus on the aspect one by one; (4) implement the test function by thinking about the behavior of the code rather than the implementation of the code; and (5) implement the code that makes the aspects pass one by one. 2.3. Behavior-Driven Development Behavior-Driven Development (BDD) is a testing method that focusing the test based on the scenarios or features (behavior) from the implemented code instead of the implementation process of the code9,18. BDD testing method provides advantages such as (1) easy to organize the test function; (2) more precise and effective for using the notation, sharing knowledge and organizing conversation between the developers; and (3) no change is required for test functions despite the requirements of the code changing. Figure 2 below shows the comparison between traditional sprint stages versus a fully behavior driven (BDD) sprint stages.
Fig. 2. a. Traditional sprint; b. BDD sprint
Based on those advantages, BDD is very convenient testing method that can be implemented in dynamic scenarios. It addresses the problem by showing how to do the testing. Hence, instead of thinking how the code is implemented,
82 4
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
the developer must think what scenarios the code will encounter. In order to understand the stages of BDD testing method, a scenario or behavior design could lead the developer to prepare a better test for BDD testing method19. 2.4. Test Behavior Driven Development (T-BDD) In order to improve the testing method for the developed backend system for Vixio, this work tried to combine TDD testing methods into BDD sprint called T-BDD. Based on previous explanations on TDD and BDD above, TDD supports the development process by defining when the testing stage is conducted. By combining this method into BDD, it was expected the development process still could be tested with different scenarios. Here, all pre-defined functions of unit test for TDD was included into the development process where the scenario and behavior function of the backend system had been defined and committed. In general, the combination design in forming T-BDD testing method could be seen in Figure 3.
Fig. 3 T-BDD design
3. System Architecture The Laravel framework was used to build the Vixio backend system, it includes a Content Management System (CMS) and an Application Programming Interface (API). Using Laravel routes, endpoints for API and CMS were created and used by the users. Figure 4 shows the Vixio backend architecture:
Fig. 4 Vixio’s backend architecture
The functions that are located in the controller with specific operations are triggered by calling an API endpoint. The functions performed queries using the Eloquent Library provided by Laravel. The performed queries are processed in the model connected to the database. The functions then returned the result and provide the response back to the users. The development of the Vixio CMS user interface utilizes blade templates that are provided by Laravel. This user interface is connected with a web route that enables the users to access registered endpoints created by the route directing the user’s browser to the specified blade pages. Before directing the user, a function from the accessed endpoint will be triggered. For the web-route, requesting a POST HTTP method required a cross-site request forgery
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
83 5
(CSRF) token. This request was forwarded to the controller when the token has been verified by “VerifyCsrfToken” middleware provided by Laravel. In this system architecture, the API provides endpoints that might be accessed by the users to generate the output data with JSON format. The output is provided from the function that are triggered by the accessed endpoint. The available endpoints are registered in the Laravel API route. Unlike the web routes that are already provided with CSRF token verification, the API route requires other methods of token verification in middleware. For authentication, the backend system uses JSON Web Token (JWT) to check whether the user had access to the system or not. The requests were being verified by the middleware and checked whether the user had the authorization or not. 4. Testing Implementation and Results The purpose of this research was to apply a T-BDD testing method including analyzing its performance compare to TDD testing method. Both of T-BDD and TDD test methods were implemented into the Vixio backend system development sprint. In implementing the test using the Laravel framework, the unit tests were conducted for the API and web routes. The unit tests were performed to check the capability of both routes by trying to request the endpoints on those routes and check the responses of the routes with the goal that has been set before using the TDD and T-BDD testing methods. For the implementation of the testing process, example features of the Vixio backend system for TDD and T-BDD methods were conducted in four main phases: 4.1. Phase I: create a unit test using TDD and T-BDD testing methods In the first phase, both testing methods, TDD and T-BDD were used to test a single unit test. For this phase, an example of initial data in Vixio databases was used as the default value. The initial data can be seen in Figure 5a below. The unit test was testing the ‘writer get story’ feature. This feature was used to get a list of stories that were created by the writer who was logged in into Vixio. Hence, to continue the testing process for TDD and T-BDD methods, some code required to be defined to get stories based on user id. Figure 5b shows the snippets of code of the ‘writer get story’ feature based on user id.
Fig. 5 a. Example of Initial data for testing purposes (left); b. Snippets code of writer get story feature based on user (right)
Next, in the Laravel framework, PHPUnit was used as the tool to conduct both testing methods. The unit test that using TDD and T-BDD testing methods respectively were tags by ‘TDD’ and ‘TBDD’ in the function’s name to differentiate the testing methods. Hence, a unit test ‘writer get story’ feature was set up for this purpose. Both TDD and T-BDD unit testing were created first before the code was written. The difference between the TDD and T-BDD unit test were located in the coding style. In TDD testing, the implementation of the code that was being tested were checked automatically, while for the T-BDD testing, the behavior of the code that has been committed were also checked together with the TDD unit test. The example unit test code can be seen in Figure 6a for the TDD test function and Figure 6b for the T-BDD test function.
84 6
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
Fig. 6 Snippets code of (a) TDD test function (left); (b) T-BDD test function (right)
4.2. Phase II: run the tests until no errors and failures are found In the second phase, the tests to check were performed and it showed that both testing methods from the first phase were resulting without errors or failures. If any errors or failures were found, the phase would be restarted from the first phase and the code would be changed to satisfy the unit tests. The test implementation from TDD and T-BDD unit tests were performed until both testing methods showing no error or failure. If one of the testing methods shows an error or failure, it was required to go back to the first phase and changed the code to satisfy both unit tests. Figure 7 shows the results of the test for the first run in both TDD and TBDD test functions.
Fig. 7 TDD Test Result First Run (left); T-BDD Test Result First Run (right)
4.3. Phase III: add data to the Vixio For the third phase, the data that involving the feature were added from the second phase into the database of Vixio. This phase was conducted to check whether the TDD and T-BDD testing methods still showed the satisfactory result after the data was changed. Some changes had been made in the database that included the data shown in the tested feature in the unit test. The purpose in doing some changes in the database was aiming to see whether the implementation and behavior test conducted in TDD and T-BDD testing methods still respectively valid after rerun the test in the fourth phase. Please refer to Figure 8 for data changes.
Fig. 8 Example of data after done some changes
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
85 7
4.4. Phase IV: rerunning the tests to check if there were errors or failures in the TDD and T-BDD testing methods In the fourth phase, the tests from the second phase were returned. The tests were still holding the same code while the data was already changed in the third phase. This phase existed to make sure the testing methods still resulted in no errors and failures. As expected, the TDD test method showed an error while for T-BDD it still performed well. Figure 9 shows the result for the TDD and T-BDD test after the data was changed.
Fig. 9 TDD test result after the data was changed – result: error (left); T-BDD test result after the data was changed – result: success (right)
5. Discussion Based on the testing implementation and result above, both TDD and T-BDD testing methods showed good results without any errors or failures for the first and second phases. However, when the third phase was conducted, which was changing data in the database, the fourth phase showed an error for the TDD test result. On the contrary, the TBDD test results still produced satisfactory results. The reason behind the error result of the TDD testing method was because the data used in this test function request had been deleted from the database which caused a different behavior for the code. For the T-BDD test function, the story id that became the parameter of the function was dynamic. Therefore, when executing the ‘when’ notation in the T-BDD test function (please refer to Figure 6b), the dynamic ‘storyId’ parameter from the ‘given’ notation was used. At the final notation, the ‘then’ notation, the test result was still showing no errors because it got the remaining ‘storyId’ that existed in the database after the data was changed. In addition, the result also showed that TDD and T-BDD testing methods each had its own advantages. By applying TDD logic in both testing methods, the test result improves the percentage of overall test-coverage in the unit test for Vixio, in this case for getting the story based on user id. Furthermore, the T-BDD test function is able to keep the unit test running smoothly even after the parameters were changed by changing the initial data inside the database of the Vixio backend system. By using T-BDD as the combination of TDD and BDD testing methods, this paper was able to provide evidence for an improvement of testing method compared with using TDD only. By using the T-BDD testing method, the testing phases were able to keep the overall test-coverage percentage high, while changes in the parameters of the feature do not affect the test result as long as the goal is still the same. 6. Conclusion This paper successfully implemented the testing method for the backend system by combining TDD and BDD into a T-BDD testing method. By implementing a simple scenario for the testing purposes, the result successfully showed that the T-BDD could perform the software testing better compared with TDD test only, especially when the parameters of the features were changed or expanded. By applying TDD and T-BDD for the Vixio backend system, this paper was also able to show that the theory16 that states that the use of a TDD testing method to get a high percentage of test-coverage is correctly proven. When the developed code that was tested did not contain any bugs or errors and the behavior was not changing, the test method worked flawlessly. In addition, when the environment was changed, for example when the dummy data was regenerated differently, the unit test for TDD resulted an error which had to be refactored. Here, the BDD scenario in the TBDD testing method showed an alternative method to overcome the error. In using the T-BDD testing method, the
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
86 8
implementation of the test function shifted automatically from implementation-based into behavior-based. For future research, to ensure T-BDD could perform well in any features for Vixio backend system testing (or any other system testing), further enhancement of T-BDD function test need to be performed. Further test scenarios with different features are also required to accommodate more complex environment. Acknowledgments This work has been supported by Bina Nusantara University. I personally would like to thank my former student Ieuan Ignatius for his work and contribution in building and testing the backend system. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19.
Vijayasarathy LR, Butler CW. Choice of Software Development Methodologies: Do Organizational, Project, and Team Characteristi cs Matter? IEEE Software. 2016 ;33(5):86–94. Abrahamsson P, Salo O, Ronkainen J, Warsta J. Agile Software Development Methods: Review and Analysis. 2017 Banica L, Radulescu M, Rosca D, Hagiu A. Is DevOps another Project Management Methodology? Inform Econ. 2017 ;21(3). Nidagundi P, Novickis L. Introducing Lean Canvas Model Adaptation in the Scrum Software Testing. Procedia Computer Science. 2017; 104:97–103. Gupta V, Mehta A, Gupta C. Improvisation of Reusability Oriented Software Testing. In Springer, Singapore; 2019 ; p. 139–45. Kumar N. Software Testing Article - Why is software testing necessary? [internet]. 2019. Available from: https://www.utest.com/articles/whyis-software-testing-necessary SM R. What Is Software Testing - Definition, Types, Methods, Approaches. [internet]. 2019. Available from: https://www.softwaretestingmaterial.com/software-testing/ Darmawan A, Ignatius I, Valdo S, Manuaba IBK. A Comparative Study Between Narrative Fiction and Interactive Fiction to Enhanc e Youth Literacy in Indonesia. In: 11th Annual International Conference on Computer Games Multimedia and Allied Technologies (CGAT 2018). 2018. Shafiee S, Hvam L, Haug A, Wautelet Y. Behavior-Driven Development in Product Configuration Systems. CEUR-WS; 2018 p. 49–52. Darmawan A. Vixio: Interactive Fiction Platform Implementation of Single Page Application with Flux Architecture Generate Story Recommendations. Thesis Report. Bina Nusantara University; 2018 Ignatius I. Vixio: Interactive Fiction Platform Back-End Design with Ink Plugin. Thesis Report, Bina Nusantara University; 2018 He RY. Design and Implementation of Web Based on Laravel Framework. In: Proceedings of the 2014 International Conference on C omputer Science and Electronic Technology. Paris, France: Atlantis Press; 2015 Stauffer M. Laravel: up and running: a framework for building modern PHP apps. O'Reilly Media, Inc., 2016 p. 454 Bean M. Laravel 5 Essentials. 1st ed. Vol. Laravel 5, Packt Publishing. Brimingham -UK: Packt Publishing; 2015 Sun Y. Unit Testing. In: Practical Application Development with AppRun. Berkeley, CA: Apress Publishing; 2019. p. 247–64. Ambler S. Introduction to Test Driven Development (TDD) [Internet]. 2019 [cited 2019 Mar 29]. Available from: http://agiledata.org/essays/tdd.html Hartikainen J. 5 step method to make test-driven development and unit testing easy | Code Utopia [Internet]. 2016 [cited 2019 Mar 29]. Available from: https://codeutopia.net/blog/2016/10/10/5-step-method-to-make-test-driven-development-and-unit-testing-easy/ Bonfanti S, Gargantini A, Mashkoor A. Generation of Behavior-Driven Development C++ Tests from Abstract State Machine Scenarios. In Springer, Cham; 2018. p. 146–52. CodeUtopia. What’s the difference between Unit Testing, TDD and BDD? | CodeUtopia [Internet]. 2019 [cited 2019 May 6]. Availa ble from: https://codeutopia.net/blog/2015/03/01/unit-testing-tdd-and-bdd/
ScienceDirect Available online at www.sciencedirect.com Procedia Computer Science 00 (2019) 000–000
ScienceDirect
www.elsevier.com/locate/procedia
Procedia Computer Science 157 (2019) 79–86
4th International Conference on Computer Science and Computational Intelligence 2019 (ICCSCI), 12–13 September 2019
Combination of Test-Driven Development and Behavior-Driven Development for Improving Backend Testing Performance Ida Bagus Kerthyayana Manuabaa,* a
Computer Science Program, Faculty of Computing and Media, Bina Nusantara University, Jakarta, Indonesia
Abstract In the process of software development, a testing phase is an important step for detecting the errors. In order to produce a reliable application, this paper discusses the utilization of a combination from two different testing methods, Test- Driven Development (TDD) and Behavior Driven Development (BDD). TDD testing method is a process with a high percentage of code which is tested automatically. However, it could lead to an error when the parameters of the features are changed. Meanwhile, the BDD testing method is a testing method that could perform successfully despite changes to the parameters of the features. This paper discusses the combination of TDD and BDD testing methods, which is called as T-BDD method. By implementing T-BDD into the Vixio backend system, this paper successfully shows that it could be performed to achieve a high percentage of test-coverage while it could also adapt when the parameters of the features were changed. This paper also shows that the T-BDD testing method performed better compared to using the TDD testing method only. © 2019 2019 The The Authors. Authors. Published Published by by Elsevier Elsevier B.V. B.V. © This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Peer-review under responsibility Computational Intelligence 2019. of the scientific committee of the 4th International Conference on Computer Science and Computational Intelligence 2019
Keywords: Unit Test; Test-Driven Development (TDD); Behavior-Driven Development(BDD); Backend; Web Application;
* Corresponding author. Tel.: +62-21-5345830. E-mail address: [email protected] 1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Computational Intelligence 2019
1877-0509 © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/) Peer-review under responsibility of the scientific committee of the 4th International Conference on Computer Science and Computational Intelligence 2019. 10.1016/j.procs.2019.08.144
80 2
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
1. Introduction Nowadays, when developing a software system or software application, many software developers follow a number of specific stages known as software development methodologies1. There are a lot of examples of software development methodologies such as Agile2, Water Fall3, Lean Development4 and many more which have been applied as guidelines for software development process. For any software development methodology, a testing phase plays an important role in creating reliable software. Based on Gupta, et al5 in the book ‘Improvisation of Reusability Oriented Software Testing’, they stated that software testing plays an important role in evincing any errors that might occur during the development phases. In addition, software testing is also important to ensure the quality and performance of the product (software system/application)6. There are a lot of methods of software testing7. This paper discusses the usage and a combination of TDD and BDD testing methods to perform testing for backend system. As a continuation of previous work, these testing methods were applied into the development process of a backend system for the Vixio web application platform. Vixio is a platform that has been developed to create an environment for people to write and share an interactive fiction story8. In serving a fully functional code of the Vixio backend system, various things inside the system require tweaking, including the quality of the code and testing methods. For the unit test of the Vixio backend system, a TDD testing method has been applied. The testing steps were shown since the beginning of creating a new feature such as determining a goal, creating a unit test, and writing some code until the unit test showed no errors or failures. Although the test-coverage of the code was high, there was still a problem using TDD testing method. The problem occurred when the parameter values of the feature were changed, making the test function become error-prone, and the test function were needed to be updated or rewritten. For addressing this TDD problem in this backend testing scenario, a BDD testing method was applied into the test function. According to Shafiee9, a BDD testing method is a behavior tests of the program rather than the implementation. It means the input and output have become a dynamic shape according to the scenario of the program. Hence, by combining BDD and TDD testing methods, this hopefully could provide an evidence that these two testing methods could complement each other and produce in the best unit test result for the Vixio backend system. This paper will discuss in more detail about the implementation of the combination of TDD and BDD testing methods, which is referring to T-BDD testing, into the Vixio backend system. Further discussion is also presented in a theoretical foundation part, following by a system architecture part and a testing implementation and results part. Then the paper will continue with a discussion section and will close with a conclusion section. 2. Theoretical Foundation 2.1. Vixio backend system As mentioned previously, in order to apply a T-BDD testing method, the Vixio backend system was utilized for the testing object. Vixio is a web application platform that provide an interactive environment that allow users to write interactive stories and also share it with readers to be played10,11. The Vixio backend was developed using the Laravel framework. Laravel is an open source framework that uses PHP as programming language. The Laravel framework provides modules such as authentication, routing, sessions, and caching which are commonly used for a web applications12,13. Laravel also provides support for testing by using PHPUnit14,15. The testing includes JSON APIs for verifying the JSON and testing file uploads. PHPUnit is able to help a developer to test and debug each endpoint created in a form of unit test. The unit test is used to help the developer to design their code while maintaining the code safe from bugs and errors when making changes. 2.2. Test-Driven Development Test-Driven Development (TDD) is a testing method that obliges the developer to set the goals of the test before starting to write the code16. After the test is created, the developer is required to write the code to satisfy the test until no error or no failure produced. The stages of the TDD testing method can be seen in Figure 1 below:
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
81 3
Fig. 1. TDD testing method stage16
The process of using TDD are as follows: (1) set a goal for a new feature and implement it to a test function; (2) run the test that results in error or failure; (3) Write some code for that test function; (4) re-run the test function. If the result shows an error or failure, repeat the third step and refactor the code until there is no error or failure found; and (5) repeat the process of TDD for a new feature until there are no more features. According to Hartikainen17, the challenge in using the TDD testing method is when the developer needs to think what should be written in the test function when there is nothing to be tested. However, five tips that can help the developers using TDD testing method easier17, are (1) decide the inputs and output for that specific feature; (2) decide the parameters inside the test function; (3) divide the aspects of the feature and focus on the aspect one by one; (4) implement the test function by thinking about the behavior of the code rather than the implementation of the code; and (5) implement the code that makes the aspects pass one by one. 2.3. Behavior-Driven Development Behavior-Driven Development (BDD) is a testing method that focusing the test based on the scenarios or features (behavior) from the implemented code instead of the implementation process of the code9,18. BDD testing method provides advantages such as (1) easy to organize the test function; (2) more precise and effective for using the notation, sharing knowledge and organizing conversation between the developers; and (3) no change is required for test functions despite the requirements of the code changing. Figure 2 below shows the comparison between traditional sprint stages versus a fully behavior driven (BDD) sprint stages.
Fig. 2. a. Traditional sprint; b. BDD sprint
Based on those advantages, BDD is very convenient testing method that can be implemented in dynamic scenarios. It addresses the problem by showing how to do the testing. Hence, instead of thinking how the code is implemented,
82 4
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
the developer must think what scenarios the code will encounter. In order to understand the stages of BDD testing method, a scenario or behavior design could lead the developer to prepare a better test for BDD testing method19. 2.4. Test Behavior Driven Development (T-BDD) In order to improve the testing method for the developed backend system for Vixio, this work tried to combine TDD testing methods into BDD sprint called T-BDD. Based on previous explanations on TDD and BDD above, TDD supports the development process by defining when the testing stage is conducted. By combining this method into BDD, it was expected the development process still could be tested with different scenarios. Here, all pre-defined functions of unit test for TDD was included into the development process where the scenario and behavior function of the backend system had been defined and committed. In general, the combination design in forming T-BDD testing method could be seen in Figure 3.
Fig. 3 T-BDD design
3. System Architecture The Laravel framework was used to build the Vixio backend system, it includes a Content Management System (CMS) and an Application Programming Interface (API). Using Laravel routes, endpoints for API and CMS were created and used by the users. Figure 4 shows the Vixio backend architecture:
Fig. 4 Vixio’s backend architecture
The functions that are located in the controller with specific operations are triggered by calling an API endpoint. The functions performed queries using the Eloquent Library provided by Laravel. The performed queries are processed in the model connected to the database. The functions then returned the result and provide the response back to the users. The development of the Vixio CMS user interface utilizes blade templates that are provided by Laravel. This user interface is connected with a web route that enables the users to access registered endpoints created by the route directing the user’s browser to the specified blade pages. Before directing the user, a function from the accessed endpoint will be triggered. For the web-route, requesting a POST HTTP method required a cross-site request forgery
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
83 5
(CSRF) token. This request was forwarded to the controller when the token has been verified by “VerifyCsrfToken” middleware provided by Laravel. In this system architecture, the API provides endpoints that might be accessed by the users to generate the output data with JSON format. The output is provided from the function that are triggered by the accessed endpoint. The available endpoints are registered in the Laravel API route. Unlike the web routes that are already provided with CSRF token verification, the API route requires other methods of token verification in middleware. For authentication, the backend system uses JSON Web Token (JWT) to check whether the user had access to the system or not. The requests were being verified by the middleware and checked whether the user had the authorization or not. 4. Testing Implementation and Results The purpose of this research was to apply a T-BDD testing method including analyzing its performance compare to TDD testing method. Both of T-BDD and TDD test methods were implemented into the Vixio backend system development sprint. In implementing the test using the Laravel framework, the unit tests were conducted for the API and web routes. The unit tests were performed to check the capability of both routes by trying to request the endpoints on those routes and check the responses of the routes with the goal that has been set before using the TDD and T-BDD testing methods. For the implementation of the testing process, example features of the Vixio backend system for TDD and T-BDD methods were conducted in four main phases: 4.1. Phase I: create a unit test using TDD and T-BDD testing methods In the first phase, both testing methods, TDD and T-BDD were used to test a single unit test. For this phase, an example of initial data in Vixio databases was used as the default value. The initial data can be seen in Figure 5a below. The unit test was testing the ‘writer get story’ feature. This feature was used to get a list of stories that were created by the writer who was logged in into Vixio. Hence, to continue the testing process for TDD and T-BDD methods, some code required to be defined to get stories based on user id. Figure 5b shows the snippets of code of the ‘writer get story’ feature based on user id.
Fig. 5 a. Example of Initial data for testing purposes (left); b. Snippets code of writer get story feature based on user (right)
Next, in the Laravel framework, PHPUnit was used as the tool to conduct both testing methods. The unit test that using TDD and T-BDD testing methods respectively were tags by ‘TDD’ and ‘TBDD’ in the function’s name to differentiate the testing methods. Hence, a unit test ‘writer get story’ feature was set up for this purpose. Both TDD and T-BDD unit testing were created first before the code was written. The difference between the TDD and T-BDD unit test were located in the coding style. In TDD testing, the implementation of the code that was being tested were checked automatically, while for the T-BDD testing, the behavior of the code that has been committed were also checked together with the TDD unit test. The example unit test code can be seen in Figure 6a for the TDD test function and Figure 6b for the T-BDD test function.
84 6
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
Fig. 6 Snippets code of (a) TDD test function (left); (b) T-BDD test function (right)
4.2. Phase II: run the tests until no errors and failures are found In the second phase, the tests to check were performed and it showed that both testing methods from the first phase were resulting without errors or failures. If any errors or failures were found, the phase would be restarted from the first phase and the code would be changed to satisfy the unit tests. The test implementation from TDD and T-BDD unit tests were performed until both testing methods showing no error or failure. If one of the testing methods shows an error or failure, it was required to go back to the first phase and changed the code to satisfy both unit tests. Figure 7 shows the results of the test for the first run in both TDD and TBDD test functions.
Fig. 7 TDD Test Result First Run (left); T-BDD Test Result First Run (right)
4.3. Phase III: add data to the Vixio For the third phase, the data that involving the feature were added from the second phase into the database of Vixio. This phase was conducted to check whether the TDD and T-BDD testing methods still showed the satisfactory result after the data was changed. Some changes had been made in the database that included the data shown in the tested feature in the unit test. The purpose in doing some changes in the database was aiming to see whether the implementation and behavior test conducted in TDD and T-BDD testing methods still respectively valid after rerun the test in the fourth phase. Please refer to Figure 8 for data changes.
Fig. 8 Example of data after done some changes
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
85 7
4.4. Phase IV: rerunning the tests to check if there were errors or failures in the TDD and T-BDD testing methods In the fourth phase, the tests from the second phase were returned. The tests were still holding the same code while the data was already changed in the third phase. This phase existed to make sure the testing methods still resulted in no errors and failures. As expected, the TDD test method showed an error while for T-BDD it still performed well. Figure 9 shows the result for the TDD and T-BDD test after the data was changed.
Fig. 9 TDD test result after the data was changed – result: error (left); T-BDD test result after the data was changed – result: success (right)
5. Discussion Based on the testing implementation and result above, both TDD and T-BDD testing methods showed good results without any errors or failures for the first and second phases. However, when the third phase was conducted, which was changing data in the database, the fourth phase showed an error for the TDD test result. On the contrary, the TBDD test results still produced satisfactory results. The reason behind the error result of the TDD testing method was because the data used in this test function request had been deleted from the database which caused a different behavior for the code. For the T-BDD test function, the story id that became the parameter of the function was dynamic. Therefore, when executing the ‘when’ notation in the T-BDD test function (please refer to Figure 6b), the dynamic ‘storyId’ parameter from the ‘given’ notation was used. At the final notation, the ‘then’ notation, the test result was still showing no errors because it got the remaining ‘storyId’ that existed in the database after the data was changed. In addition, the result also showed that TDD and T-BDD testing methods each had its own advantages. By applying TDD logic in both testing methods, the test result improves the percentage of overall test-coverage in the unit test for Vixio, in this case for getting the story based on user id. Furthermore, the T-BDD test function is able to keep the unit test running smoothly even after the parameters were changed by changing the initial data inside the database of the Vixio backend system. By using T-BDD as the combination of TDD and BDD testing methods, this paper was able to provide evidence for an improvement of testing method compared with using TDD only. By using the T-BDD testing method, the testing phases were able to keep the overall test-coverage percentage high, while changes in the parameters of the feature do not affect the test result as long as the goal is still the same. 6. Conclusion This paper successfully implemented the testing method for the backend system by combining TDD and BDD into a T-BDD testing method. By implementing a simple scenario for the testing purposes, the result successfully showed that the T-BDD could perform the software testing better compared with TDD test only, especially when the parameters of the features were changed or expanded. By applying TDD and T-BDD for the Vixio backend system, this paper was also able to show that the theory16 that states that the use of a TDD testing method to get a high percentage of test-coverage is correctly proven. When the developed code that was tested did not contain any bugs or errors and the behavior was not changing, the test method worked flawlessly. In addition, when the environment was changed, for example when the dummy data was regenerated differently, the unit test for TDD resulted an error which had to be refactored. Here, the BDD scenario in the TBDD testing method showed an alternative method to overcome the error. In using the T-BDD testing method, the
Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 157 (2019) 79–86 Ida Bagus Kerthyayana Manuaba / Procedia Computer Science 00 (2019) 000–000
86 8
implementation of the test function shifted automatically from implementation-based into behavior-based. For future research, to ensure T-BDD could perform well in any features for Vixio backend system testing (or any other system testing), further enhancement of T-BDD function test need to be performed. Further test scenarios with different features are also required to accommodate more complex environment. Acknowledgments This work has been supported by Bina Nusantara University. I personally would like to thank my former student Ieuan Ignatius for his work and contribution in building and testing the backend system. References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 18. 19.
Vijayasarathy LR, Butler CW. Choice of Software Development Methodologies: Do Organizational, Project, and Team Characteristi cs Matter? IEEE Software. 2016 ;33(5):86–94. Abrahamsson P, Salo O, Ronkainen J, Warsta J. Agile Software Development Methods: Review and Analysis. 2017 Banica L, Radulescu M, Rosca D, Hagiu A. Is DevOps another Project Management Methodology? Inform Econ. 2017 ;21(3). Nidagundi P, Novickis L. Introducing Lean Canvas Model Adaptation in the Scrum Software Testing. Procedia Computer Science. 2017; 104:97–103. Gupta V, Mehta A, Gupta C. Improvisation of Reusability Oriented Software Testing. In Springer, Singapore; 2019 ; p. 139–45. Kumar N. Software Testing Article - Why is software testing necessary? [internet]. 2019. Available from: https://www.utest.com/articles/whyis-software-testing-necessary SM R. What Is Software Testing - Definition, Types, Methods, Approaches. [internet]. 2019. Available from: https://www.softwaretestingmaterial.com/software-testing/ Darmawan A, Ignatius I, Valdo S, Manuaba IBK. A Comparative Study Between Narrative Fiction and Interactive Fiction to Enhanc e Youth Literacy in Indonesia. In: 11th Annual International Conference on Computer Games Multimedia and Allied Technologies (CGAT 2018). 2018. Shafiee S, Hvam L, Haug A, Wautelet Y. Behavior-Driven Development in Product Configuration Systems. CEUR-WS; 2018 p. 49–52. Darmawan A. Vixio: Interactive Fiction Platform Implementation of Single Page Application with Flux Architecture Generate Story Recommendations. Thesis Report. Bina Nusantara University; 2018 Ignatius I. Vixio: Interactive Fiction Platform Back-End Design with Ink Plugin. Thesis Report, Bina Nusantara University; 2018 He RY. Design and Implementation of Web Based on Laravel Framework. In: Proceedings of the 2014 International Conference on C omputer Science and Electronic Technology. Paris, France: Atlantis Press; 2015 Stauffer M. Laravel: up and running: a framework for building modern PHP apps. O'Reilly Media, Inc., 2016 p. 454 Bean M. Laravel 5 Essentials. 1st ed. Vol. Laravel 5, Packt Publishing. Brimingham -UK: Packt Publishing; 2015 Sun Y. Unit Testing. In: Practical Application Development with AppRun. Berkeley, CA: Apress Publishing; 2019. p. 247–64. Ambler S. Introduction to Test Driven Development (TDD) [Internet]. 2019 [cited 2019 Mar 29]. Available from: http://agiledata.org/essays/tdd.html Hartikainen J. 5 step method to make test-driven development and unit testing easy | Code Utopia [Internet]. 2016 [cited 2019 Mar 29]. Available from: https://codeutopia.net/blog/2016/10/10/5-step-method-to-make-test-driven-development-and-unit-testing-easy/ Bonfanti S, Gargantini A, Mashkoor A. Generation of Behavior-Driven Development C++ Tests from Abstract State Machine Scenarios. In Springer, Cham; 2018. p. 146–52. CodeUtopia. What’s the difference between Unit Testing, TDD and BDD? | CodeUtopia [Internet]. 2019 [cited 2019 May 6]. Availa ble from: https://codeutopia.net/blog/2015/03/01/unit-testing-tdd-and-bdd/