Using JUnit5 – Part 4 – Filtering Tests

java, junit5

Published: 2019-07 (July 2019)
Relevant for: JUnit 5.5.0

JUnit5 Blog Series

Part 1 – Introduction
Part 2 – Test Basics
Part 3 – Display Names
Part 4 – Filtering tests

In Part 1 of the blog series, we looked at several annotations used in JUnit5. We covered test methods as well as lifecycle methods.

In Part 2 of the blog series, we covered the basics of testing using JUnit5. We covered test annotations such as marking a test method and asserting. We saw how a test method could be tagged and how assumptions can be used. We finally wrapped up with test execution ordering mechanisms.

In Part 3 of the blog series, we detailed ways in which JUnit test classes and test methods could be customized with readable and meaningful names rather than the standard class and method names.

This post will cover filtering JUnit tests for execution. There are several reasons for running a subset of your comprehensive suite of tests. Some example usages include:

  • You may have unit and integration tests and wish to only run unit tests.
  • You may want a sanity check with the core features alone being tested.
  • You may wish to bypass a few tests to test a specific situation.
  • You may wish to skip a few failing tests, since you know they fail.

Filtration Types

Dynamic filtration

In Part 2 of the blog series, we covered what I term, dynamic filtration. This was done using Assumptions (https://junit.org/junit5/docs/5.5.0/user-guide/#writing-tests-assumptions). Assumptions are conditions that when not met, cause the test to be aborted rather than fail. Assumptions are a great feature to dynamically filter unit tests.

Assumptions are evaluated during execution of the test class, and are embedded within the code block of the test method. Repeating the example we covered in that blog:

class TestWithAssumptions {

    @Test
    void testOnlyOnHost123() {
        assumeTrue("host123".equals(System.getenv("HOSTNAME")));
        // remainder of test
    }

    @Test
    void testOnHost123OrAbortWithMessage() {
        assumeTrue("host123".equals(System.getenv("HOSTNAME")),
            () -> "Aborting test: not on host123");
        // remainder of test
    }

}

Static filtration

In Part 2 of the blog series, we covered the @Tag (https://junit.org/junit5/docs/5.5.0/api/org/junit/jupiter/api/Tag.html) annotation on the unit test method. This annotation can be used to statically label classes and methods. The benefit of such an identification (or label or category) is that a predicate check can be made of such.

Filtering tests based on static labels is what I term static filtration. Since the Tag is an annotation, it is not a part of the code block that executes the unit test. A Tag is a @Repeatable (https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/annotation/Repeatable.html) annotation, which implies a class or method can have several tags annotating them.

How Static Filtration works

Code Katas are means to teach a concept, a feature or a functionality. In some Code Katas we deliberately remove logic to make tests fail. The mission for the student is to “fix” these tests. Since this is a learning process, there is a known solution also provided. In order to verify that the solutions are not failing, we can run a build that executes only the solutions (and other tests that are expected to pass). A way of doing that is to leverage Tags.

In this above example, we can thus tag known passing tests as “PASSING” and the kata tests as “TODO”.

See PublicMethodInvocationTest (https://github.com/c-guntur/java-katas/blob/baseline/java-handles/src/test/java/none/cvg/methods/PublicMethodInvocationTest.java).

  • On line 37, the test method is tagged as PASSING. This is a test that is expected to always pass, since it is already solved.
  • On line 63, the other test method is tagged as TODO, since it is what needs to be fixed (and is expected to fail).

We will now look at how we can filter tests both in an IDE (IntelliJ IDEA) as well as during a build process (Apache Maven).

Filtering tests in IDE

When we run all tests in the IDE, screen shot below, ALL tests found are executed, which is not the ideal outcome since several tests marked TODO will fail.

JUNitRunAllTests

Run All Tests (in IntelliJ IDEA)

The Runner configuration can be edited. Select a Test Kind, with value Tags. In this example we used PASSING and TODO as the tags. We are trying to only run the PASSING tests, thus the Tag expression we use is PASSING.

JUNitRunConfiguration

Configure to Run tests with a specific Tag (in IntelliJ IDEA)

When this run configuration is executed, any test tagged as PASSING is included and executed. Tests without this tag (or with the TODO tag, in this example), are filtered out and ignored.

Filtering tests in a build

It is great to setup the IDE on a desktop to work as needed. Repeatability and automation drives using a build tool to do the same. We can also filter tests based on Tags using a build tool such as maven.

JUnit tests are executed in the test phase of the maven lifecycle. A common (and default) plugin to run unit tests is the maven-surefire-plugin. The plugin can be configured to include and exclude groups (generic name corresponding to tags in JUnit5, other unit test frameworks may have other names).

Example: pom.xml (https://github.com/c-guntur/java-katas/blob/baseline/pom.xml#L59-L67)

                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <version>2.22.2</version>
                    <configuration>
                        <groups>PASSING</groups>
                        <excludedGroups>TODO</excludedGroups>
                    </configuration>
                </plugin>

In this example, when the maven build is run, any test with a Tag of PASSING is included and any test with a Tag of TODO is excluded.

Summary

In this blog, we saw how we could filter test class and test method names both dynamically using assumptions and more statically using Tags.

Hope this was helpful !

3 thoughts on “Using JUnit5 – Part 4 – Filtering Tests

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s