Your test scenarios have become quite numerous and you are about to lose the overview? You are looking for a clever way to keep them well organized and want to increase the maintainability?

TAGS

One way is to add tags to the tests using annotations. There are multiple annotations available @WithTag, @WithTags and @WithTagValuesOf. As we want to add multiple tags to each test I will go on with @WithTagValuesOf. An explanation of the other tag annotations can be found in the Serenity Reference Manual.

The annotation @WithTagValuesOf can be used to assign multiple tags to the same test and thus creating categories which can later be used to run only specific tests. How it can look like in actual test:

@WithTagValuesOf({"level:tutorial", "feature:battle", "story:battle start"})

The colon separates the tag type from the tag name which will appear in the reports. This allows already to run tests which are e.g. only tagged for tutorial (specifying multiple filters is also possible):

mvn clean verify -Dtags="level:tutorial"

Tagging is a great way to organize your tests and also the report will give better insights on test categories and how many were run in each.

CONSTANTS

Tags are great but is it really necessary to type them in each single test again? It might be good for debugging but when adding tests to hundreds of tests we should really not use Strings anymore but look for an alternative. This comes in form of constants and further constant classes.

Meaning for each type creating a class which contains all tag names for this type.

public class Level {
    private static final String BASE = "level:";

    public static final String TUTORIAL = BASE + "tutorial";
    public static final String PAYMENT = BASE + "payment";
    public static final String HOMEPAGE = BASE + "homepage";
    public static final String SETTINGS = BASE + "settings";
}

These can now be used within the same annotation:

@WithTagValuesOf({Level.TUTORIAL, Feature.BATTLE, Story.BATTLE_START"})

Just imagine how much time you have saved now when only one of the strings need to be changed. Instead of changing it in hundreds of tests it only needs to be changed in a single place.

That’s not the only advantage as it is also much easier to find the usages with the IDE.

REFLECTION

But there is more to it. You may want to get an overview of all the defined tags then Reflection is your friend. Java Reflection makes it possible to inspect classes, interfaces, fields and methods at runtime, without knowing the names of the classes, methods etc. at compile time. Sounds awesome? Let’s see how we can use it.

All the work can be done with the following method which will print all the constants for us:

public static void getValuesFromClass(Object object) {
    String name = object.getClass().getSimpleName();
    String pack = object.getClass().getPackage().getName();
    try {
        Class c = Class.forName(pack + "." + name);
        Field[] fields = c.getFields();

        for (Field f : fields) {
            f.setAccessible(true);
            System.out.println(f.get(null));
        }
    } catch (Exception e) {
        System.out.println("Exception: " + e.toString());
    }
}

Now we only need to specify for which class we want to get the constants and as result we get a nice list with all the tags:

public static void main(String args[]) {
    Object level = new Level();
    Object feature = new Feature();
    Object story = new Story();

    TagValues.getValuesFromClass(level);
    TagValues.getValuesFromClass(feature);
    TagValues.getValuesFromClass(story);
}

And the result:

level:tutorial
level:payment
level:homepage
level:settings
feature:first session
feature:battle
story:welcome
story:scout province
story:click on campaign map
story:end
story:enter province
story:select sector
story:attack sector
story:start battle

This data could also be stored somewhere in a file and accessed by other tools such as Jenkins.

JENKINS

As part of our continuous integration we use Jenkins to run our tests on a scheduled basis but also manually when needed. While we were still using the strings as tags we had no other was as to use the same strings also in Jenkins.

mvn clean verify -Dtags="level:tutorial"

There are many plugins for Jenkins available which will help us to develop a somehow dynamic system for providing the tags. In a first step we need to make the maven command parameterized so the string can be easily exchanged. By defining a parameterized build with the string parameter which we will call TAG_VALUE and changing the command to use it we can simply enter a string on running the build:

mvn clean verify -Dtags="${TAG_VALUE}"

We would still have to recall all the possible tags to be able to run the build as we want. You remember that we could get a list of tag values and store them in a file? We can now use this file to create a choice parameter instead of the string parameter. To be more precise we are going to use a dynamic choice parameter which will provide all the entries of our file as choices for the Jenkins build.

The dynamic choice parameter accepts a script which can fetch a file from a repository and use it’s content:

def list = ['/bin/bash','-c',"git archive --format tar --remote=git@gitlab.tigerteufel.de:tests-browser.git 
HEAD:src/main/resources/statistics tagValues.txt | tar xf - -O"].execute().text.readLines()

When now starting the build Jenkins will provide us with a nice dropdown list which contains all our tags we have defined in the constant classes.

CONCLUSION

Tags can be used to organize numerous scenarios. Constants can be used for easier maintaining of the defined tags. Reflection can provide us with a list of all defined tag constants which can be stored in a file. In the end this can be used in Jenkins to populate a dropdown with all the defined tags.

Overall these changes reduced human errors by e.g. typing the wrong tag type or name in the test resulting in the test never being executed. But it also allows other people who are less experienced with the whole test environment to run and execute tests easily.