Integration testing with Docker Neo4j image and Testcontainers

· 8 min read

Automated testing is the cornerstone of any successful software project. Applications using the Neo4j database are no exception. This blog post shows how to use the Neo4j Docker image and the Testcontainers library for integration testing in Java using JUnit.

This blog post shows examples in Java. Testcontainers library has been ported to many other languages so the same approach and principles can be applied. Check out the Testcontainers github page.

Motivation

Neo4j already provides a testing harness to start a temporary database within tests, either manually or through a JUnit rule. To use this harness one must include the neo4j-harness maven artifact, together with whole Neo4j database as a test dependency to the project. This inevitably pollutes the test classpath leading to various issues, for example

  • having conflicting versions of libraries on the classpath (the culprits are usually Lucene, Jetty or Scala), see for example this issue

  • Spring auto-configuration is affected by presence of certain classes on classpath

  • non-determinism - your code either sometimes fails, or fails only in certain environments

To avoid these issues we will use the Testcontainers library with the official Neo4j Docker image to start the database for our tests.

Testcontainers is a Java library that supports JUnit tests, providing lightweight, throwaway instances of common databases, Selenium web browsers, or anything else that can run in a Docker container.

https://www.testcontainers.org/

How to use Testcontainers

Let’s start with a simplest possible example of an application using the Neo4j Java driver to connect to a Neo4j database running in server mode.

First you need Docker installed on your machine. If you don’t have Docker installed already follow the official Docker documentation.

Then in your project, add the Testcontainers dependency. Use test scope so you don’t distribute the dependency with your application.

        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>testcontainers</artifactId>
            <version>1.10.2</version>
            <scope>test</scope>
        </dependency>

and the Neo4j Java driver

        <dependency>
            <groupId>org.neo4j.driver</groupId>
            <artifactId>neo4j-java-driver</artifactId>
            <version>1.7.2</version>
        </dependency>

Note that there are no other Neo4j dependencies required in the project.

A short while ago there has been a merged PR to the Testcontainers library with special support for Neo4j in form of a Neo4jContainer class. It provides some of the features we cover in this blog post - namely authentication configuration, getting the bolt url and configuring the enterprise edition. We are looking forward to trying it out once it is released!.

Then in your JUnit tests, use the Testcontainers library to start a container with the Neo4j database

public class ApplicationIT {

   @ClassRule
   public static GenericContainer neo4j = new GenericContainer("neo4j:3.5.0")
           .withExposedPorts(7687);

   @Test
   public void shouldAnswerWithOne() {
       String uri = "bolt://" + neo4j.getContainerIpAddress() + ":" + neo4j.getMappedPort(7687);

       try (Driver driver = GraphDatabase.driver(uri, AuthTokens.basic("neo4j", "neo4j"))) {

           try (Session session = driver.session()) {
               StatementResult result = session.run("RETURN 1 AS value");
               int value = result.single().get("value").asInt();
               assertThat(value).isEqualTo(1);
           }

       }
   }

}

The rule GenericContainer starts the given docker image before all tests and shuts it down after all tests finish. The setting withExposedPorts exposes a given port from the inside of the container on a random port available from the outside. Then neo4j.getMappedPort() returns this random port.

Once the bolt uri is constructed, the test creates an instance of Neo4j Java driver as usual. See full example on github.

You can also run your tests against enterprise version of Neo4j provided you have a license. Just change the docker image version to x.y.z-enterprise. Of course, you need to have a valid enterprise license

Configuring Neo4j

Neo4j Docker image provides a way to change the Neo4j configuration through environment variables. For details of the naming convention see the official Neo4j documentation. Testcontainers provides the withEnv method to pass a variable to the container, for example, the usual way of disabling authentication in neo4j.conf is

dbms.security.auth_enabled=false

With Testcontainers this can be done with

new GenericContainer("neo4j:3.5.0")
	.withEnv("NEO4J_dbms_security_auth__enabled", "false")

Turning off authentication isn’t a best practice. Fortunately the Neo4j Docker image supports setting password via a special environment variable (this is specific to the image, not Neo4j).

new GenericContainer("neo4j:3.5.0")
	.withEnv("NEO4J_AUTH", "neo4j/Password123")

The GenericContainer class from Testcontainers library has also few configuration options. For example to reduce timeout during container startup to make debugging faster you can use withStartupTimeout.

new GenericContainer("neo4j:3.5.0")
	.withStartupTimeout(ofSeconds(5))

Container scope

Using the @ClassRule annotation specifies that the container is started before the test class and is shut down after all tests are executed. By using the @Rule annotation instead, it is possible to limit the scope to a single test method only. This can be useful for tests which leave the database in unusable state when finished.

In most cases, however, you will likely want to extend the scope of the container beyond a single test class, for example to avoid repeated start of the container and hence reducing the test execution time. Currently there is no native support in the Testcontainers library for this, but it is easily achievable by the following snippet of custom code.

public class Neo4jContainerSupport {

   private static GenericContainer neo4j = new GenericContainer("neo4j:3.5.0")
           .withEnv("NEO4J_dbms_security_auth__enabled", "false")
           .withExposedPorts(7687);


   public static void start() {
       if (!neo4j.isRunning()) {
           neo4j.start();
       }
   }

   public static String uri() {
       return "bolt://" + neo4j.getContainerIpAddress() + ":" + neo4j.getMappedPort(7687);
   }
}

And in your tests ensure that the container is started.

@BeforeClass
public static void setUpClass() {
   Neo4jContainerSupport.start();
}

Because a single instance of the database is used for all tests, you must perform a cleanup between tests manually to have the tests independent. This might be tricky if indexes and/or constraints are involved and unless you have a strong requirements about performance using per-class or per-method container might be easier.

Pro tip: clean up your database in @Before method, not @After - you will be able to inspect the state of your database after running a single test.

Notice there is no need to stop the container anywhere, it will be destroyed at JVM exit by the Testcontainers library.

Custom procedure or plugin

Sometimes an application needs a custom Cypher procedure or a Neo4j plugin to run. The Neo4j Docker image loads plugins from the /plugins directory inside the container. This can be achieved with binding the jar file using withFileSystemBind method on the GenericContainer.

Plugins usually fall into one of following categories, which further complicates things

  • no dependencies

  • has dependencies, can be packaged as fat jar

  • has dependencies, cannot be packaged as fat jar

No dependencies

This is the simplest case. Your fully functioning plugin jar is prepared by the time the integration test phase runs. Simply bind the final jar to the plugins folder. See the example on github.

Fat jar

Similar to previous case, just bind the fat jar, instead of the regular jar. See the example on github.

Jar with dependencies

While Neo4j itself shows in the manual how to package a plugin jar with dependencies, there are reasons why you might not want to do that (for example you may simply loathe fat jars or you have been burned with classpath issues). In such cases, you can copy all dependencies to a folder and bind the folder to the plugins. See the example on github.

In any case, don’t forget to whitelist your procedures if required.

Existing docker compose

Do you already have a docker compose file describing your Neo4j container with all plugins and configuration? By using DockerComposeContainer you can take advantage of it and start Neo4j (and others services) in the same way as you do in your development and production environments.

Summary

Testcontainers works really well together with Neo4j and is a handy tool to have in one’s toolbox.

It provides several advantages

  • easy to use

  • cleans up your dependencies, removes conflicts

  • tests execution is closer/identical to the production environment

and has some disadvantages

  • might have worse performance

  • need to have Docker installed on your local and CI machines, with all it’s caveats.

František Hartman