Link Search Menu Expand Document
Software testing (TDDD04)

Lab 4: Model-based testing

Purpose

So far in the course, we have seen several techniques that may be useful for testing units of an application, integrating components of an application and generating regression tests for verifying the correct behavior of applications. Testing an application through the use of either static test scripts or exploratory testing may prove insufficient though. Exploratory testing allows us to test all functionality that is possible to reach from the main interface of an application, and has the potential of revealing new bugs, but is time-consuming and costly. Script-based testing is of course much cheaper, but also much more restricted in what it tests, and by their definition, scripts only ever follow a certain path through an application, and sometimes only a restricted part of an application. If we want to explore how an application behaves under all conditions in which we have stated that it should work, and efficiently find sources of errors during testing, we may have to explore other options as well. One option is to use models of how a certain piece of software behaves, and then use such a model to generate possible interaction patterns with the software, and ascertain certain invariants or facts about our system during such generated interactions. A model may be a UML state diagram , or some other formalism to describe sets of interactions that are possible, along with their prerequisites and consequences in our system. The general idea is to ensure that there is an abstract model of your software system that you keep in sync with your implementation, and whenever the software changes, your model should reflect these changes and enable you to test many different paths along your application, given that you have defined how to move from one state to the next, such as moving from the “Logged out” state to the “Logged in” state by, well, logging in. In this lab assignment, you will use the Model-based testing tool GraphWalker developed by people at Spotify in order to run tests and verify the behavior of a system, create changes to your application and ensure that GraphWalker can detect those changes, share your changes with your peers, and work together to resolve the issues that arise.

  • Due to some issues please use this GraphWalker clone instead for running pet clinic tests.

Preparation

For this lab you will work separately for part 2 - you should include individual reflections on how you detected the changes. You also need to have Maven installed. Maven is a tool for building applications, as well as maintaining dependencies and a build environment for you, and possibly generating boilerplate code, which is what the example above will help you with. You will also need to have Firefox installed. Start by reading how GraphWalker and the editor GraphWalker Studio work.

Part 1

Start by cloning a small starter project. This project contains a small example of how to exercise the edges and the vertices in this model in the file src/main/java/com/company/SomeSmallTest.java.

Download GraphWalker Studio and start it with java -jar graphwalker-studio-4.3.1.jar.

Then open a browser and use the following url: http://localhost:9090/studio.html.

You can use the Graphwalker editor to open the json file describing the model of the code. The model is described as a state machine and the test code exercises the code by going through the possible paths described by the state machine.

Introduce a change in the test code (SomeSmallTest.java) so that, when executing one of the edges twice, it will invalidate an assertion in the test program (you can use Junit assertions). GraphWalker may not detect this, so you may need to modify the annotation of the test class so that the generator and stop condition allow you to traverse each edge of the graph more than once. Does GraphWalker detect the error now?

Given that you have described all different paths through an application in your model, what would happen if some paths are invalidated by further development of your application, where new conditions are introduced for transitions? In the next part, you will explore what happens when such changes take place, when working with your team to share code changes.

Part 2

In this assignment, you will work with the PetClinic example which is a somewhat larger, but still small web application. In the tests, you will use a web browser automation tool called Selenium, which enables you to automate browser interactions with a web application, such as clicking on links, filling out forms, etcetera. It will not help you assess the visual appearance of an application, but you can traverse the application from the perspective of the web browser. The tests that GraphWalker uses in this case include browser interactions and verification of the outcome of such interactions. The edges are actions, and the nodes are where you verify the outcome of your actions, and the validity of the current application state. The first step is to start up the Pet Clinic application:

git clone https://github.com/spring-projects/spring-petclinic
cd spring-petclinic
git reset --hard e2fbc561309d03d92a0958f3cf59219b1fc0d985
./mvnw spring-boot:run

To check that the applicaiton is running, open it http://localhost:8080 . It should show: Pet clinic

Now install the graphwalker example:

git clone https://gitlab.liu.se/TDDD04-course/labs/tddd04-graphwalker-2023.git
cd petclinic-graphwalker/java-petclinic

Then start GraphWalker:

./mvnw graphwalker:test
  1. You will introduce changes in the application that will require your model to change. Either, you may add constraints to the domain model, as in the pets, owners and veterinarians, such as that names should have a certain limited length, or that the age of owners should be in a given range. Also ensure that the tests fail after the introduction of your changes unless you change the tests. For instance, if you change the required length of a telephone number for a pet owner, your model should fail to verify the current model of the PetClinic application state in which you add a new owner. Introduce changes to the PetClinic application that in turn require changes to either:
  • the model edges, including guards and actions
  • the model states,
  • the model step definitions
  • the generator or stop conditions to be detected

Introduce two such changes to your local PetClinic application.

  1. After you have introduced these changes, you shall share them with your partner. Use Gitlab for this purpose, and share through a common repository of your modified PetClinic applications. After you have cloned the PetClinic repository to your own machine and local repository, add a new remote repository on Gitlab. Start by creating a bare git repository for your shared code, and then pushing your application. We suggest you use separate branches for your pairs, so you can switch between them.

  2. Make sure you can detect each others’ modifications to the application in your Graphwalker tests, and resolve the issues you encounter.

Part 3 - Optional

For this part you can use the example in Part 1 as a starting point. Modify the example and the test cases as necessary to illustrate your answer to the following questions:

  1. What is the relationship between the model coverage, as in edge coverage, vertex coverage or path coverage when GraphWalker executes the tests that implement the model steps, and code coverage of the application code? Illustrate your answer by running Graphwalker with different coverage criteria and discussing the results.

  2. Does 100% edge coverage imply 100% code coverage? Does it imply 100% branch coverage? Motivate your answer.

Reporting your results:

See the general instructions for successfully completing the lab assignment. Your shall submit a report where you describe:

  • answers to the questions in part 1
  • your changes to the PetClinic application (refer to commits and describe them)
  • the changes that you received from your partner, how your model-based tests detected them, and how you could revise your model to correctly reflect the new functionality introduced
  • answers to the questions in part 3 together with any modifications to the code of the example in Part 1