This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Core Concepts

Understand the fundamental concepts that power Verifyica

1: Arguments
2: Lifecycle
3: Contexts
4: Interceptors
5: Execution Model

This section explores the core concepts that make Verifyica a powerful testing framework.

Key Concepts

Arguments

At the heart of Verifyica is the concept of arguments. Instead of writing separate test methods for different scenarios, you write one test that executes multiple times with different arguments. The Argument<T> interface provides type-safe, named test data containers.

Learn more: Arguments →

Lifecycle

Verifyica provides a complete test lifecycle that goes far beyond traditional setUp/tearDown patterns. Each argument gets its own lifecycle with Prepare, BeforeAll, BeforeEach, Test, AfterEach, AfterAll, and Conclude phases.

Learn more: Lifecycle →

Contexts

Context objects provide access to test execution state and metadata. Three context types (EngineContext, ClassContext, ArgumentContext) give you control over test execution at different levels.

Learn more: Contexts →

Interceptors

Interceptors allow you to hook into the test lifecycle for cross-cutting concerns like logging, metrics, or resource management. ClassInterceptor and EngineInterceptor provide flexible interception points.

Learn more: Interceptors →

Execution Model

Understanding how Verifyica executes tests is crucial for writing efficient, parallel-friendly tests. The execution model defines how arguments are processed, how parallelism works, and how test methods are invoked.

Learn more: Execution Model →

Why These Concepts Matter

Traditional parameterized testing frameworks often:

Lack proper lifecycle management per parameter
Don’t support complex argument types well
Have limited parallelism control
Provide no interception or extension points

Verifyica addresses all these limitations through its core concepts, enabling you to:

Write cleaner, more maintainable tests
Test complex scenarios with minimal code
Control execution parallelism at multiple levels
Extend test behavior without modifying test code

Next Steps

Start with Arguments to understand how test data flows through Verifyica, then progress through the other concepts to build a complete mental model of the framework.

1 - Arguments

Understanding the Argument interface and argument suppliers

Arguments are the foundation of Verifyica testing. They define the test data that your tests will execute against.

The Argument<T> Interface

The Argument<T> interface is a type-safe container that associates a name with a payload:

public interface Argument<T> {
    String getName();         // Display name for test reporting
    T getPayload();          // The actual test data
    boolean hasPayload();    // Check if payload is non-null
    <V> V getPayloadAs(Class<V> type); // Cast payload to specific type
}

Why Use Argument<T>?

While you can use simple types like String or Integer directly, Argument<T> provides:

Named arguments - Better test reporting with descriptive names
Type safety - Compile-time type checking
Complex payloads - Wrap any type of test data
Better debugging - Clear argument identification in logs and reports

Creating Arguments

Use the static factory methods:

Generic Arguments

Argument<String> arg = Argument.of("test-name", "payload-value");
Argument<Config> config = Argument.of("prod-config", new DatabaseConfig());

Primitive Type Arguments

Verifyica provides convenience methods for primitives:

Argument<Boolean> bool = Argument.ofBoolean(true);        // "true"
Argument<Integer> num = Argument.ofInt(42);               // "42"
Argument<Long> lng = Argument.ofLong(100L);               // "100"
Argument<Double> dbl = Argument.ofDouble(3.14);           // "3.14"
Argument<String> str = Argument.ofString("hello");        // "hello"

Special Handling

String arguments have special null/empty handling:

Argument.ofString(null);      // Name: "String=/null/", Payload: null
Argument.ofString("");        // Name: "String=/empty/", Payload: ""
Argument.ofString("value");   // Name: "value", Payload: "value"

BigInteger and BigDecimal

Argument<BigInteger> big = Argument.ofBigInteger("12345678901234567890");
Argument<BigDecimal> dec = Argument.ofBigDecimal("3.141592653589793");

Argument Suppliers

The @ArgumentSupplier method provides arguments to your tests.

Basic Structure

@Verifyica.ArgumentSupplier
public static Object arguments() {
    return /* Collection, array, Stream, or Argument<T> instances */;
}

Requirements

Must be static
Must be public
Return type must be one of:
- Collection<?>
- Array (e.g., String[], Object[])
- Stream<?>
- Single or multiple Argument<T> objects

Return Type Examples

Collection (Most Common)

@Verifyica.ArgumentSupplier
public static Collection<String> arguments() {
    return Arrays.asList("test1", "test2", "test3");
}

Argument<T> Collection

@Verifyica.ArgumentSupplier
public static Collection<Argument<DatabaseConfig>> arguments() {
    return Arrays.asList(
        Argument.of("h2-memory", new DatabaseConfig("jdbc:h2:mem:test")),
        Argument.of("postgresql", new DatabaseConfig("jdbc:postgresql://localhost/test")),
        Argument.of("mysql", new DatabaseConfig("jdbc:mysql://localhost/test"))
    );
}

Array

@Verifyica.ArgumentSupplier
public static String[] arguments() {
    return new String[] {"arg1", "arg2", "arg3"};
}

Stream (For Large Datasets)

@Verifyica.ArgumentSupplier
public static Stream<Integer> arguments() {
    return IntStream.range(0, 1000).boxed();
}

Streams are useful for:

Large datasets that don’t fit in memory
Lazy evaluation of arguments
Dynamic argument generation

Parallelism Configuration

Control how many arguments execute concurrently:

@Verifyica.ArgumentSupplier(parallelism = 4)
public static Collection<String> arguments() {
    return generateArguments();
}

parallelism = 1 (default) - Sequential execution
parallelism = 2+ - Number of arguments executing in parallel
parallelism = 0 - Uses configured default parallelism

See Configuration → Parallelism for more details.

Using Arguments in Tests

Direct Type Usage

When your argument supplier returns simple types:

@Verifyica.ArgumentSupplier
public static Collection<String> arguments() {
    return Arrays.asList("test1", "test2");
}

@Verifyica.Test
public void test(String argument) {
    // argument is directly the String value
    System.out.println("Testing: " + argument);
}

Argument<T> Usage

When using Argument<T>:

@Verifyica.ArgumentSupplier
public static Collection<Argument<Config>> arguments() {
    return Arrays.asList(
        Argument.of("dev", new Config("dev")),
        Argument.of("prod", new Config("prod"))
    );
}

@Verifyica.Test
public void test(Config config) {
    // config is the unwrapped payload
    System.out.println("Testing config: " + config.getName());
}

Verifyica automatically unwraps Argument<T> to provide the payload to your test methods.

Accessing Argument Name

To access the argument name in your test, use the ArgumentContext:

@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    String name = argumentContext.getTestArgument().getName();
    System.out.println("Argument name: " + name);
    System.out.println("Config: " + config);
}

Complex Argument Patterns

Multiple Argument Types

Create a wrapper class for multiple parameters:

public class TestData {

    private final String name;
    private final int port;
    private final boolean ssl;

    public TestData(String name, int port, boolean ssl) {
        this.name = name;
        this.port = port;
        this.ssl = ssl;
    }

    // Getters...
}

@Verifyica.ArgumentSupplier
public static Collection<Argument<TestData>> arguments() {
    return Arrays.asList(
        Argument.of("http-local", new TestData("localhost", 8080, false)),
        Argument.of("https-local", new TestData("localhost", 8443, true)),
        Argument.of("http-remote", new TestData("remote.com", 80, false))
    );
}

@Verifyica.Test
public void test(TestData data) {
    Connection conn = new Connection(data.getName(), data.getPort(), data.isSsl());
    // Test logic...
}

Resource-Based Arguments

Create arguments that manage their own resources:

public class DatabaseArgument implements AutoCloseable {
    private final String name;
    private final Connection connection;

    public DatabaseArgument(String url) {
        this.name = extractName(url);
        this.connection = DriverManager.getConnection(url);
    }

    public Connection getConnection() {
        return connection;
    }

    @Override
    public void close() throws Exception {
        connection.close();
    }
}

@Verifyica.ArgumentSupplier
public static Collection<Argument<DatabaseArgument>> arguments() {
    return Arrays.asList(
        Argument.of("h2", new DatabaseArgument("jdbc:h2:mem:test")),
        Argument.of("postgres", new DatabaseArgument("jdbc:postgresql://localhost/test"))
    );
}

@Verifyica.AfterAll
public void afterAll(DatabaseArgument db) throws Exception {
    db.close(); // Clean up per-argument resources
}

Dynamic Argument Generation

Generate arguments based on configuration files, environment variables, or external sources:

@Verifyica.ArgumentSupplier
public static Collection<Argument<ServerConfig>> arguments() throws IOException {
    List<Argument<ServerConfig>> args = new ArrayList<>();

    // Load from file
    Path configFile = Paths.get("test-configs.json");
    if (Files.exists(configFile)) {
        List<ServerConfig> configs = loadConfigsFromFile(configFile);
        for (ServerConfig config : configs) {
            args.add(Argument.of(config.getName(), config));
        }
    }

    // Add environment-specific configs
    String env = System.getenv("TEST_ENV");
    if ("full".equals(env)) {
        args.addAll(loadFullTestSuite());
    }

    return args;
}

Conditional Arguments

Filter arguments based on runtime conditions:

@Verifyica.ArgumentSupplier
public static Collection<Argument<Database>> arguments() {
    List<Argument<Database>> all = Arrays.asList(
        Argument.of("h2", Database.H2),
        Argument.of("postgres", Database.POSTGRES),
        Argument.of("mysql", Database.MYSQL),
        Argument.of("oracle", Database.ORACLE)
    );

    // Filter based on available drivers
    return all.stream()
        .filter(arg -> isDriverAvailable(arg.getPayload()))
        .collect(Collectors.toList());
}

private static boolean isDriverAvailable(Database db) {
    try {
        Class.forName(db.getDriverClass());
        return true;
    } catch (ClassNotFoundException e) {
        return false;
    }
}

Best Practices

Use Meaningful Names

// Good: Descriptive names
Argument.of("prod-database-ssl-enabled", config)
Argument.of("dev-database-no-ssl", config)

// Bad: Generic names
Argument.of("config1", config)
Argument.of("config2", config)

Keep Arguments Immutable

Arguments should be immutable to avoid side effects between tests:

// Good: Immutable argument
public class Config {

    private final String url;
    private final int timeout;

    public Config(String url, int timeout) {
        this.url = url;
        this.timeout = timeout;
    }

    // Only getters, no setters
}

// Bad: Mutable argument
public class Config {

    private String url;
    private int timeout;

    // Setters allow modification
    public void setUrl(String url) { this.url = url; }
}

Validate Arguments Early

Validate in the argument supplier to fail fast:

@Verifyica.ArgumentSupplier
public static Collection<Argument<Config>> arguments() {
    List<Config> configs = loadConfigs();

    // Validate all configs before returning
    for (Config config : configs) {
        if (config.getUrl() == null) {
            throw new IllegalStateException("Config URL cannot be null");
        }
    }

    return configs.stream()
        .map(c -> Argument.of(c.getName(), c))
        .collect(Collectors.toList());
}

Limit Argument Count

Too many arguments can make tests slow:

// Good: Focused test scope
@Verifyica.ArgumentSupplier
public static Collection<String> arguments() {
    return Arrays.asList("critical-case-1", "critical-case-2", "edge-case");
}

// Bad: Excessive arguments
@Verifyica.ArgumentSupplier
public static Collection<Integer> arguments() {
    return IntStream.range(0, 10000).boxed().collect(Collectors.toList());
}

For large datasets, consider:

Using filters to run subsets
Splitting into multiple test classes
Using parallelism to speed up execution

Next Steps

Lifecycle - Understand how arguments flow through the test lifecycle
Contexts - Access argument metadata via ArgumentContext
Configuration → Parallelism - Configure parallel argument execution

2 - Lifecycle

Complete test lifecycle management with Prepare, BeforeAll, BeforeEach, Test, AfterEach, AfterAll, and Conclude

Verifyica provides a comprehensive test lifecycle that gives you fine-grained control over setup and teardown at multiple levels.

Lifecycle Overview

Unlike traditional testing frameworks that only provide before/after hooks, Verifyica offers a complete lifecycle with five distinct phases:

Prepare - One-time setup before any arguments are processed
BeforeAll - Setup for each argument (before its tests)
BeforeEach - Setup before each individual test method
Test - The actual test execution
AfterEach - Cleanup after each individual test method
AfterAll - Cleanup for each argument (after its tests)
Conclude - One-time cleanup after all arguments are processed

Lifecycle Diagram

sequenceDiagram
    participant Engine as Test Engine
    participant TestClass as Test Class
    participant Interceptors as Interceptors

    Engine->>TestClass: @Prepare
    Engine->>TestClass: @ArgumentSupplier

    loop For Each Argument
        Interceptors->>TestClass: preBeforeAll
        Engine->>TestClass: @BeforeAll(argument)

        loop For Each @Test Method
            Engine->>TestClass: @BeforeEach(argument)
            Engine->>TestClass: @Test(argument)
            Engine->>TestClass: @AfterEach(argument)
        end

        Engine->>TestClass: @AfterAll(argument)
        Interceptors->>TestClass: postAfterAll
    end

    Engine->>TestClass: @Conclude

Lifecycle Annotations

@Prepare

Executes once before any arguments are processed.

@Verifyica.Prepare
public void prepare() {
    // Called once before all arguments
    // Use for: Starting services, initializing shared resources
}

When to use:

Starting external services (databases, web servers)
Loading shared configuration
Initializing expensive resources used by all arguments
Setting up test environment

Example:

@Verifyica.Prepare
public void prepare() {
    System.setProperty("test.mode", "true");
    SharedResourcePool.initialize();
    TestDatabase.start();
}

@ArgumentSupplier

Provides the test arguments (covered in detail in Arguments).

@Verifyica.ArgumentSupplier(parallelism = 2)
public static Object arguments() {
    return Arrays.asList("arg1", "arg2", "arg3");
}

@BeforeAll

Executes once per argument, before any test methods run for that argument.

@Verifyica.BeforeAll
public void beforeAll(String argument) {
    // Called once per argument before its tests
    // Use for: Creating connections, setting up argument-specific state
}

When to use:

Creating database connections for this argument
Setting up argument-specific resources
Initializing state shared across test methods for this argument
Performing expensive setup that can be reused

Example:

private DatabaseConnection connection;

@Verifyica.BeforeAll
public void beforeAll(DatabaseConfig config) {
    connection = new DatabaseConnection(config);
    connection.connect();
    connection.createSchema();
    connection.loadTestData();
}

Note: This simple example assumes sequential argument execution (parallelism = 1). For parallel arguments, use a Map<String, Connection> to isolate connections per argument. See Managing State with Instance Variables below.

@BeforeEach

Executes before each test method for each argument.

@Verifyica.BeforeEach
public void beforeEach(String argument) {
    // Called before each test method
    // Use for: Resetting state, clearing caches, preparing test data
}

When to use:

Resetting shared state between tests
Clearing caches or temporary data
Creating test-specific resources
Ensuring test isolation

Example:

@Verifyica.BeforeEach
public void beforeEach(DatabaseConfig config) {
    connection.clearTables();
    connection.resetSequences();
    testDataBuilder = new TestDataBuilder();
}

@Test

The actual test method that contains your test logic.

@Verifyica.Test
public void testSomething(String argument) {
    // Test logic
    // Executes once per argument
}

Multiple test methods:

@Verifyica.Test
public void testCreation(Config config) {
    Service service = new Service(config);
    assert service != null;
}

@Verifyica.Test
public void testOperation(Config config) {
    Service service = new Service(config);
    Result result = service.execute();
    assert result.isSuccess();
}

Each @Test method executes for every argument, with @BeforeEach and @AfterEach surrounding each execution.

@AfterEach

Executes after each test method for each argument.

@Verifyica.AfterEach
public void afterEach(String argument) {
    // Called after each test method
    // Use for: Cleaning up test-specific resources
}

When to use:

Cleaning up test-specific resources
Removing temporary files or data
Resetting mocks or stubs
Capturing test metrics or logs

Example:

@Verifyica.AfterEach
public void afterEach(DatabaseConfig config) {
    connection.rollback();
    tempFileManager.cleanupTestFiles();
    metricCollector.recordTestExecution();
}

@AfterAll

Executes once per argument, after all test methods run for that argument.

@Verifyica.AfterAll
public void afterAll(String argument) {
    // Called once per argument after its tests
    // Use for: Closing connections, releasing argument-specific resources
}

When to use:

Closing database connections
Releasing argument-specific resources
Saving test reports or metrics
Performing final cleanup for this argument

Example:

@Verifyica.AfterAll
public void afterAll(DatabaseConfig config) {
    connection.dropSchema();
    connection.disconnect();
    reportGenerator.saveReport(config.getName());
}

@Conclude

Executes once after all arguments are processed.

@Verifyica.Conclude
public void conclude() {
    // Called once after all arguments
    // Use for: Stopping services, generating aggregate reports
}

When to use:

Stopping external services
Generating aggregate test reports
Cleaning up shared resources
Performing final validation

Example:

@Verifyica.Conclude
public void conclude() {
    TestDatabase.stop();
    SharedResourcePool.shutdown();
    AggregateReportGenerator.generateReport();
}

Complete Lifecycle Example

package com.example.tests;

import org.verifyica.api.Argument;
import org.verifyica.api.Verifyica;
import java.util.Arrays;
import java.util.Collection;

public class CompleteLifecycleTest {

    private static TestServer server;
    private HttpClient client;

    @Verifyica.Prepare
    public void prepare() {
        System.out.println("1. Prepare: Starting test server");
        server = new TestServer();
        server.start();
    }

    @Verifyica.ArgumentSupplier(parallelism = 2)
    public static Collection<Argument<Config>> arguments() {
        System.out.println("2. ArgumentSupplier: Generating arguments");
        return Arrays.asList(
            Argument.of("http-config", new Config("http", 8080)),
            Argument.of("https-config", new Config("https", 8443))
        );
    }

    @Verifyica.BeforeAll
    public void beforeAll(Config config) {
        System.out.println("3. BeforeAll: Setting up for " + config);
        client = new HttpClient(config);
        client.connect();
    }

    @Verifyica.BeforeEach
    public void beforeEach(Config config) {
        System.out.println("4. BeforeEach: Preparing for test with " + config);
        client.clearCookies();
        server.resetState();
    }

    @Verifyica.Test
    public void test1(Config config) {
        System.out.println("5. Test1: Testing with " + config);
        Response response = client.get("/api/test");
        assert response.getStatus() == 200;
    }

    @Verifyica.Test
    public void test2(Config config) {
        System.out.println("5. Test2: Testing with " + config);
        Response response = client.post("/api/data", "payload");
        assert response.getStatus() == 201;
    }

    @Verifyica.AfterEach
    public void afterEach(Config config) {
        System.out.println("6. AfterEach: Cleaning up after test with " + config);
        server.clearTestData();
    }

    @Verifyica.AfterAll
    public void afterAll(Config config) {
        System.out.println("7. AfterAll: Tearing down for " + config);
        client.disconnect();
    }

    @Verifyica.Conclude
    public void conclude() {
        System.out.println("8. Conclude: Stopping test server");
        server.stop();
    }
}

Execution Flow

With 2 arguments and 2 test methods, the execution is:

1. Prepare
2. ArgumentSupplier
   --- Argument: http-config ---
   3. BeforeAll (http-config)
      4. BeforeEach (http-config)
      5. Test1 (http-config)
      6. AfterEach (http-config)
      4. BeforeEach (http-config)
      5. Test2 (http-config)
      6. AfterEach (http-config)
   7. AfterAll (http-config)
   --- Argument: https-config ---
   3. BeforeAll (https-config)
      4. BeforeEach (https-config)
      5. Test1 (https-config)
      6. AfterEach (https-config)
      4. BeforeEach (https-config)
      5. Test2 (https-config)
      6. AfterEach (https-config)
   7. AfterAll (https-config)
8. Conclude

With parallelism = 2, both arguments may execute concurrently (http-config and https-config in parallel).

Error Handling in Lifecycle

Exceptions in @Prepare

If @Prepare throws an exception, test execution stops immediately:

@Verifyica.Prepare
public void prepare() throws Exception {
    if (!canStartTestServer()) {
        throw new RuntimeException("Cannot start test server");
    }
    server.start(); // Not executed if exception thrown
}

Result: No arguments are processed, all tests are skipped.

Exceptions in @BeforeAll

If @BeforeAll throws an exception for an argument, all tests for that argument are skipped:

@Verifyica.BeforeAll
public void beforeAll(Config config) throws Exception {
    connection = database.connect(config);
    if (connection == null) {
        throw new RuntimeException("Failed to connect to " + config);
    }
}

Result: Tests for this argument are skipped, other arguments continue.

Exceptions in @BeforeEach

If @BeforeEach throws an exception, the test method is skipped but other tests continue:

@Verifyica.BeforeEach
public void beforeEach(Config config) {
    if (!prerequisitesMet()) {
        throw new TestSkippedException("Prerequisites not met");
    }
}

Result: Current test is skipped, next test method proceeds.

Exceptions in @Test

Test failures are reported normally:

@Verifyica.Test
public void test(Config config) {
    Result result = service.execute(config);
    if (!result.isSuccess()) {
        throw new AssertionError("Test failed: " + result.getError());
    }
}

Result: Test marked as failed, lifecycle continues (@AfterEach, other tests).

@AfterEach and @AfterAll Always Execute

Even if tests fail, cleanup methods execute:

@Verifyica.Test
public void test(Config config) {
    throw new RuntimeException("Test failed!");
}

@Verifyica.AfterEach
public void afterEach(Config config) {
    // Still executes even though test failed
    cleanup();
}

@Verifyica.AfterAll
public void afterAll(Config config) {
    // Still executes even if tests failed
    connection.close();
}

This ensures proper resource cleanup regardless of test outcomes.

@Conclude Always Executes

@Conclude executes even if all tests fail:

@Verifyica.Conclude
public void conclude() {
    // Always executes at the end
    server.stop();
    cleanupGlobalResources();
}

Lifecycle Best Practices

Use the Right Hook for the Right Job

// Good: Expensive setup in @BeforeAll
@Verifyica.BeforeAll
public void beforeAll(Config config) {
    connection = database.connect(config); // Expensive, do once per argument
}

// Bad: Expensive setup in @BeforeEach
@Verifyica.BeforeEach
public void beforeEach(Config config) {
    connection = database.connect(config); // Wasteful, reconnects for every test!
}

Note: This example shows efficiency patterns. For thread-safe state management with parallel arguments, use context classes stored in the ArgumentContext map. See Managing State with Instance Variables.

Keep @Prepare and @Conclude Fast

These methods block all test execution:

// Good: Only essential global setup
@Verifyica.Prepare
public void prepare() {
    System.setProperty("test.mode", "true");
}

// Bad: Slow operations in @Prepare
@Verifyica.Prepare
public void prepare() throws InterruptedException {
    Thread.sleep(10000); // Blocks everything for 10 seconds!
}

Always Clean Up in @AfterAll

Even if tests fail, clean up resources:

@Verifyica.AfterAll
public void afterAll(Config config) {
    try {
        if (connection != null) {
            connection.close();
        }
    } catch (Exception e) {
        // Log but don't throw - don't cascade failures
        logger.warn("Failed to close connection", e);
    }
}

Managing State with Instance Variables

Important: A single test class instance is shared across all arguments, so instance variables must be managed carefully, especially with parallel execution.

Safe Pattern: Use Context Classes for Per-Argument State

public class DatabaseTest {
    // Define a context class to encapsulate per-argument state
    public static class TestContext {
        private final Connection connection;

        public TestContext(Connection connection) {
            this.connection = connection;
        }

        public Connection getConnection() {
            return connection;
        }
    }

    @Verifyica.BeforeAll
    public void beforeAll(ArgumentContext argumentContext) {
        Config config = argumentContext.getArgument().getPayloadAs(Config.class);
        Connection conn = database.connect(config);

        // Store context in ArgumentContext map (thread-safe)
        TestContext context = new TestContext(conn);
        argumentContext.getMap().put("testContext", context);
    }

    @Verifyica.Test
    public void test1(ArgumentContext argumentContext) {
        TestContext context = (TestContext) argumentContext.getMap().get("testContext");
        context.getConnection().execute("SELECT 1"); // Uses correct connection for this argument
    }

    @Verifyica.Test
    public void test2(ArgumentContext argumentContext) {
        TestContext context = (TestContext) argumentContext.getMap().get("testContext");
        context.getConnection().execute("SELECT 2"); // Uses same argument's connection
    }

    @Verifyica.AfterAll
    public void afterAll(ArgumentContext argumentContext) {
        TestContext context = (TestContext) argumentContext.getMap().get("testContext");
        if (context != null && context.getConnection() != null) {
            context.getConnection().close();
        }
    }
}

Note: Using context classes provides clean encapsulation and works safely with any parallelism configuration.

Next Steps

Contexts - Access lifecycle execution state via context objects
Interceptors - Hook into lifecycle execution
Execution Model - Understand how lifecycle phases execute

3 - Contexts

Understanding EngineContext, ClassContext, and ArgumentContext

Verifyica provides three context objects that give you access to test execution state and metadata at different levels.

Context Hierarchy

EngineContext (Engine level)
  └── ClassContext (Test class level)
        └── ArgumentContext (Argument level)

Each context provides access to:

Its parent context
Configuration and properties
Test execution metadata
Lifecycle state

Context Types

ArgumentContext

The most commonly used context, providing access to the current argument and test execution state.

Interface

public interface ArgumentContext extends Context {
    ClassContext getClassContext();
    int getArgumentIndex();
    Argument<?> getArgument();
    <V> Argument<V> getArgumentAs(Class<V> type);
}

Usage in Test Methods

Inject ArgumentContext as a parameter to any lifecycle method:

@Verifyica.BeforeAll
public void beforeAll(ArgumentContext argumentContext) {
    // Access the Argument object
    Argument<?> argument = argumentContext.getArgument();
    System.out.println("Argument name: " + argument.getName());
    System.out.println("Argument index: " + argumentContext.getArgumentIndex());

    // Access payload if needed
    String value = (String) argument.getPayload();
}

@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    // Access argument metadata
    String name = argumentContext.getArgument().getName();
    int index = argumentContext.getArgumentIndex();

    // Access class context
    ClassContext classContext = argumentContext.getClassContext();
    Class<?> testClass = classContext.getTestClass();
}

Key Methods

Method	Description
`getArgument()`	Returns the current `Argument<?>`
`getArgumentAs(Class<V>)`	Returns the argument cast to specific type
`getArgumentIndex()`	Returns the argument’s index (0-based)
`getClassContext()`	Returns the parent ClassContext

Example: Using Argument Name

@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    String argumentName = argumentContext.getArgument().getName();

    // Use argument name for logging, reporting, or conditional logic
    logger.info("Testing: " + argumentName);

    if (argumentName.contains("production")) {
        // Special handling for production arguments
        enableProductionSafetyChecks();
    }
}

Example: Conditional Test Execution

@Verifyica.BeforeAll
public void beforeAll(ArgumentContext argumentContext) {
    int index = argumentContext.getArgumentIndex();

    // Only run expensive setup for first argument
    if (index == 0) {
        expensiveGlobalSetup();
    }
}

ClassContext

Provides access to test class-level information and state.

Interface

public interface ClassContext extends Context {
    EngineContext getEngineContext();
    Class<?> getTestClass();
    Object getTestInstance();
    // Additional methods...
}

Usage

@Verifyica.Prepare
public void prepare(ClassContext classContext) {
    Class<?> testClass = classContext.getTestClass();
    System.out.println("Preparing test class: " + testClass.getName());

    // Access engine context
    EngineContext engineContext = classContext.getEngineContext();
}

Key Methods

Method	Description
`getTestClass()`	Returns the test class (`Class<?>`)
`getTestInstance()`	Returns the test instance object
`getEngineContext()`	Returns the parent EngineContext

Example: Class-Level State

@Verifyica.Prepare
public void prepare(ClassContext classContext) {
    Class<?> testClass = classContext.getTestClass();

    // Check for class-level annotations
    if (testClass.isAnnotationPresent(RequiresDatabase.class)) {
        databaseManager.startDatabase();
    }

    // Store class-level metadata
    String testClassName = testClass.getSimpleName();
    reportGenerator.startTestClass(testClassName);
}

EngineContext

Provides access to engine-level configuration and global state.

Interface

public interface EngineContext extends Context {
    Configuration getConfiguration();
    // Additional methods...
}

Usage

@Verifyica.Prepare
public void prepare(EngineContext engineContext) {
    Configuration config = engineContext.getConfiguration();

    // Access configuration properties
    String property = config.getProperty("custom.property");
    System.out.println("Property value: " + property);
}

Example: Global Configuration

@Verifyica.Prepare
public void prepare(EngineContext engineContext) {
    Configuration config = engineContext.getConfiguration();

    // Check if running in CI environment
    boolean isCi = Boolean.parseBoolean(config.getProperty("verifyica.ci.mode", "false"));

    if (isCi) {
        // Adjust test behavior for CI
        timeouts.setMultiplier(2.0);
        logger.setLevel(Level.DEBUG);
    }
}

Context Parameter Injection

Verifyica automatically injects context parameters into lifecycle methods based on their type.

Supported Parameter Combinations

Based on the lifecycle annotation, different parameter patterns are valid:

For @Prepare and @Conclude

These annotations only support no parameters OR ClassContext:

// No parameters
@Verifyica.Prepare
public void prepare() {
}

// ClassContext only
@Verifyica.Prepare
public void prepare(ClassContext classContext) {
    classContext.getMap().put("key", "value");
}

@Verifyica.Conclude
public void conclude(ClassContext classContext) {
    // Access class-level state
}

For @BeforeAll, @AfterAll, @BeforeEach, @AfterEach, @Test

These annotations support unwrapped argument OR ArgumentContext:

// Pattern 1: Unwrapped argument (most common)
@Verifyica.Test
public void test(String argument) {
    // Use the argument directly
}

// Pattern 2: ArgumentContext (when you need metadata)
@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    // Access the Argument object
    Argument<?> argument = argumentContext.getArgument();
    String name = argument.getName();
    Object payload = argument.getPayload();
}

Important Rules:

✅ @Prepare/@Conclude: No params OR ClassContext only
✅ @BeforeAll/@AfterAll/@BeforeEach/@AfterEach/@Test: Unwrapped argument OR ArgumentContext
❌ NEVER use EngineContext (not supported)
❌ NEVER mix context with unwrapped argument: test(ArgumentContext ctx, String arg)
❌ NEVER use multiple contexts: test(ArgumentContext ctx, ClassContext ctx2)

@Prepare and @Conclude

These methods execute outside argument processing and only support no parameters OR ClassContext:

@Verifyica.Prepare
public void prepare() {
    // No parameters
}

@Verifyica.Prepare
public void prepare(ClassContext classContext) {
    // ClassContext only (NOT EngineContext, NOT ArgumentContext)
    classContext.getMap().put("key", "value");
}

@Verifyica.Conclude
public void conclude() {
    // No parameters
}

@Verifyica.Conclude
public void conclude(ClassContext classContext) {
    // ClassContext only
}

Common Patterns

Conditional Test Execution Based on Argument

@Verifyica.BeforeAll
public void beforeAll(ArgumentContext argumentContext) {
    String argumentName = argumentContext.getArgument().getName();

    if (argumentName.startsWith("skip-")) {
        throw new TestSkippedException("Skipping: " + argumentName);
    }
}

Dynamic Test Configuration

Use a context class to encapsulate per-argument state:

// Define a context class to hold per-argument state
public static class TestContext {
    private final Connection connection;
    private final int port;

    public TestContext(Connection connection, int port) {
        this.connection = connection;
        this.port = port;
    }

    public Connection getConnection() {
        return connection;
    }

    public int getPort() {
        return port;
    }
}

@Verifyica.BeforeAll
public void beforeAll(ArgumentContext argumentContext) {
    int index = argumentContext.getArgumentIndex();

    // Use different ports based on argument index to avoid conflicts
    int port = 5432 + index;

    // Get config from argument payload
    DatabaseConfig config = argumentContext.getArgument().getPayloadAs(DatabaseConfig.class);
    config.setPort(port);

    // Create connection and store in context
    Connection conn = database.connect(config);
    TestContext context = new TestContext(conn, port);

    // Store context in ArgumentContext map
    argumentContext.getMap().put("testContext", context);
}

@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    TestContext context = (TestContext) argumentContext.getMap().get("testContext");
    Connection conn = context.getConnection();
    // Use connection...
}

@Verifyica.AfterAll
public void afterAll(ArgumentContext argumentContext) {
    TestContext context = (TestContext) argumentContext.getMap().get("testContext");
    if (context != null && context.getConnection() != null) {
        context.getConnection().close();
    }
}

Aggregate Reporting

Store results in ClassContext map for aggregation:

@Verifyica.AfterAll
public void afterAll(ArgumentContext argumentContext) {
    String argumentName = argumentContext.getArgument().getName();
    TestResult result = calculateResult();

    // Store result in ClassContext map (thread-safe)
    ClassContext classContext = argumentContext.getClassContext();
    @SuppressWarnings("unchecked")
    Map<String, TestResult> results = (Map<String, TestResult>)
        classContext.getMap().computeIfAbsent("results", k -> new ConcurrentHashMap<>());
    results.put(argumentName, result);
}

@Verifyica.Conclude
public void conclude(ClassContext classContext) {
    // Retrieve results from ClassContext map
    @SuppressWarnings("unchecked")
    Map<String, TestResult> results = (Map<String, TestResult>)
        classContext.getMap().get("results");

    if (results != null) {
        // Generate aggregate report from all argument results
        AggregateReport report = new AggregateReport(results);
        report.save(classContext.getTestClass().getSimpleName() + "-report.html");
    }
}

Accessing Test Method Information

While contexts don’t directly provide test method information, you can use interceptors for method-level access. See Interceptors.

Context Best Practices

Use the Right Context Level

// Good: Use ArgumentContext for argument-specific logic
@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    String name = argumentContext.getArgument().getName();
    logger.info("Testing: " + name);
}

// Bad: Don't pass context if you don't need it
@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    // Don't use argumentContext
    assert config.isValid();
}

Cache Expensive Context Lookups

private String argumentName;

@Verifyica.BeforeAll
public void beforeAll(ArgumentContext argumentContext) {
    // Cache the argument name
    argumentName = argumentContext.getArgument().getName();
}

@Verifyica.Test
public void test(Config config) {
    // Use cached value instead of passing ArgumentContext to every test
    logger.info("Testing: " + argumentName);
}

Don’t Modify Context State

Contexts are read-only - don’t try to modify them:

// Good: Read from context
@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    int index = argumentContext.getArgumentIndex();
    // Use index...
}

// Bad: Don't try to modify context
@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    // This would throw an exception if it were possible
    // argumentContext.setArgumentIndex(5);
}

Leverage Context Hierarchy

@Verifyica.Test
public void test(ArgumentContext argumentContext) {
    // Navigate the context hierarchy
    ClassContext classContext = argumentContext.getClassContext();
    EngineContext engineContext = classContext.getEngineContext();

    // Access configuration from engine level
    String property = engineContext.getConfiguration().getProperty("key");

    // Access test class from class level
    Class<?> testClass = classContext.getTestClass();

    // Access argument from argument level
    Argument<?> argument = argumentContext.getArgument();
}

Next Steps

Interceptors - Hook into lifecycle events with full context access
Execution Model - Understand how contexts flow through execution
API Reference → Context API - Complete context API reference

4 - Interceptors

Hook into test execution with ClassInterceptor and EngineInterceptor

Interceptors provide powerful extension points to hook into test execution for cross-cutting concerns like logging, metrics, resource management, or custom behavior.

Interceptor Types

Verifyica provides two interceptor interfaces:

ClassInterceptor - Intercepts class-level and argument-level execution
EngineInterceptor - Intercepts engine-level execution

ClassInterceptor

The ClassInterceptor interface provides hooks at multiple lifecycle points for each test class and argument.

Interface Overview

public interface ClassInterceptor {
    // Initialization
    void initialize(EngineContext engineContext) throws Throwable;

    // Filter which classes this interceptor applies to
    Predicate<ClassContext> predicate();

    // Class instantiation
    void preInstantiate(EngineContext engineContext, Class<?> testClass) throws Throwable;
    void postInstantiate(EngineContext engineContext, Class<?> testClass,
                         Object testInstance, Throwable throwable) throws Throwable;

    // Lifecycle hooks
    void prePrepare(ClassContext classContext, Method method) throws Throwable;
    void postPrepare(ClassContext classContext, Method method, Throwable throwable) throws Throwable;

    void preBeforeAll(ArgumentContext argumentContext, Method method) throws Throwable;
    void postBeforeAll(ArgumentContext argumentContext, Method method, Throwable throwable) throws Throwable;

    void preBeforeEach(ArgumentContext argumentContext, Method method) throws Throwable;
    void postBeforeEach(ArgumentContext argumentContext, Method method, Throwable throwable) throws Throwable;

    void preTest(ArgumentContext argumentContext, Method method) throws Throwable;
    void postTest(ArgumentContext argumentContext, Method method, Throwable throwable) throws Throwable;

    void preAfterEach(ArgumentContext argumentContext, Method method) throws Throwable;
    void postAfterEach(ArgumentContext argumentContext, Method method, Throwable throwable) throws Throwable;

    void preAfterAll(ArgumentContext argumentContext, Method method) throws Throwable;
    void postAfterAll(ArgumentContext argumentContext, Method method, Throwable throwable) throws Throwable;

    void preConclude(ClassContext classContext, Method method) throws Throwable;
    void postConclude(ClassContext classContext, Method method, Throwable throwable) throws Throwable;

    // Cleanup
    void destroy(EngineContext engineContext) throws Throwable;
}

All methods have default implementations, so you only need to override the hooks you need.

Creating a ClassInterceptor

Implement the ClassInterceptor interface and register it via ServiceLoader:

Step 1: Implement ClassInterceptor

package com.example.interceptors;

import org.verifyica.api.ArgumentContext;
import org.verifyica.api.ClassContext;
import org.verifyica.api.ClassInterceptor;
import org.verifyica.api.EngineContext;
import java.lang.reflect.Method;

public class LoggingInterceptor implements ClassInterceptor {

    @Override
    public void initialize(EngineContext engineContext) {
        System.out.println("Logging interceptor initialized");
    }

    @Override
    public void preBeforeAll(ArgumentContext argumentContext, Method method) {
        String argumentName = argumentContext.getArgument().getName();
        System.out.println("Starting tests for argument: " + argumentName);
    }

    @Override
    public void preTest(ArgumentContext argumentContext, Method method) {
        String argumentName = argumentContext.getArgument().getName();
        String methodName = method.getName();
        System.out.println("Executing test: " + methodName + " with " + argumentName);
    }

    @Override
    public void postTest(ArgumentContext argumentContext, Method method, Throwable throwable) {
        String methodName = method.getName();
        if (throwable != null) {
            System.out.println("Test failed: " + methodName + " - " + throwable.getMessage());
        } else {
            System.out.println("Test passed: " + methodName);
        }
    }

    @Override
    public void postAfterAll(ArgumentContext argumentContext, Method method, Throwable throwable) {
        String argumentName = argumentContext.getArgument().getName();
        System.out.println("Completed tests for argument: " + argumentName);
    }

    @Override
    public void destroy(EngineContext engineContext) {
        System.out.println("Logging interceptor destroyed");
    }
}

Step 2: Register via ServiceLoader

Create META-INF/services/org.verifyica.api.ClassInterceptor:

com.example.interceptors.LoggingInterceptor

Filtering Classes with Predicate

Use predicate() to control which test classes the interceptor applies to:

public class DatabaseInterceptor implements ClassInterceptor {

    @Override
    public Predicate<ClassContext> predicate() {
        // Only apply to classes annotated with @DatabaseTest
        return classContext -> {
            Class<?> testClass = classContext.getTestClass();
            return testClass.isAnnotationPresent(DatabaseTest.class);
        };
    }

    @Override
    public void preBeforeAll(ArgumentContext argumentContext, Method method) {
        // Only executed for classes with @DatabaseTest
        startDatabaseConnection();
    }
}

Common Interceptor Patterns

Timing and Performance Monitoring

public class TimingInterceptor implements ClassInterceptor {

    private final Map<String, Long> startTimes = new ConcurrentHashMap<>();

    @Override
    public void preTest(ArgumentContext argumentContext, Method method) {
        String key = getKey(argumentContext, method);
        startTimes.put(key, System.nanoTime());
    }

    @Override
    public void postTest(ArgumentContext argumentContext, Method method, Throwable throwable) {
        String key = getKey(argumentContext, method);
        Long startTime = startTimes.remove(key);
        if (startTime != null) {
            long duration = System.nanoTime() - startTime;
            System.out.printf("Test %s took %d ms%n", key, duration / 1_000_000);
        }
    }

    private String getKey(ArgumentContext argumentContext, Method method) {
        return argumentContext.getArgument().getName() + ":" + method.getName();
    }
}

Resource Management

public class ResourceInterceptor implements ClassInterceptor {

    @Override
    public void preBeforeAll(ArgumentContext argumentContext, Method method) {
        String argumentName = argumentContext.getArgument().getName();
        Resource resource = Resource.create(argumentName);

        // Store resource in ArgumentContext map
        argumentContext.getMap().put("resource", resource);
    }

    @Override
    public void postAfterAll(ArgumentContext argumentContext, Method method, Throwable throwable) {
        // Retrieve and cleanup resource
        Resource resource = (Resource) argumentContext.getMap().get("resource");
        if (resource != null) {
            resource.close();
        }
    }
}

Retry Logic

public class RetryInterceptor implements ClassInterceptor {

    private static final int MAX_RETRIES = 3;

    @Override
    public void postTest(ArgumentContext argumentContext, Method method, Throwable throwable)
            throws Throwable {
        if (throwable != null && shouldRetry(throwable)) {
            for (int i = 0; i < MAX_RETRIES; i++) {
                try {
                    // Retry the test
                    Object testInstance = argumentContext.getClassContext().getTestInstance();
                    method.invoke(testInstance, getMethodArguments(argumentContext, method));
                    return; // Success, no need to rethrow
                } catch (Throwable retryThrowable) {
                    if (i == MAX_RETRIES - 1) {
                        throw retryThrowable; // Final retry failed
                    }
                    Thread.sleep(1000 * (i + 1)); // Backoff
                }
            }
        }
        rethrow(throwable); // Rethrow original exception if not retried
    }

    private boolean shouldRetry(Throwable throwable) {
        return throwable instanceof TransientException;
    }
}

Test Result Collection

public class ResultCollectorInterceptor implements ClassInterceptor {

    private final List<TestResult> results = new ArrayList<>();

    @Override
    public void postTest(ArgumentContext argumentContext, Method method, Throwable throwable) {
        TestResult result = new TestResult(
            argumentContext.getArgument().getName(),
            method.getName(),
            throwable == null,
            throwable != null ? throwable.getMessage() : null
        );
        synchronized (results) {
            results.add(result);
        }
    }

    @Override
    public void destroy(EngineContext engineContext) {
        // Generate report with all results
        ReportGenerator.generate(results);
    }
}

EngineInterceptor

The EngineInterceptor interface provides hooks at the engine level, before any test classes are processed.

Interface Overview

public interface EngineInterceptor {
    void initialize(EngineContext engineContext) throws Throwable;
    void preDiscovery(EngineContext engineContext) throws Throwable;
    void postDiscovery(EngineContext engineContext) throws Throwable;
    void destroy(EngineContext engineContext) throws Throwable;
}

Example: Global Setup

package com.example.interceptors;

import org.verifyica.api.EngineContext;
import org.verifyica.api.EngineInterceptor;

public class GlobalSetupInterceptor implements EngineInterceptor {

    @Override
    public void initialize(EngineContext engineContext) {
        System.out.println("Engine starting");
    }

    @Override
    public void preDiscovery(EngineContext engineContext) {
        System.out.println("Starting test discovery");
        // Start global services
        GlobalServices.start();
    }

    @Override
    public void postDiscovery(EngineContext engineContext) {
        System.out.println("Test discovery complete");
    }

    @Override
    public void destroy(EngineContext engineContext) {
        System.out.println("Engine shutting down");
        // Stop global services
        GlobalServices.stop();
    }
}

com.example.interceptors.GlobalSetupInterceptor

Interceptor Best Practices

Keep Interceptors Fast

Interceptors run for every test, so keep them efficient:

// Good: Fast logging
@Override
public void preTest(ArgumentContext argumentContext, Method method) {
    logger.debug("Test: {}", method.getName());
}

// Bad: Slow synchronous operations
@Override
public void preTest(ArgumentContext argumentContext, Method method) {
    sendHttpRequest("http://slow-service.com/log"); // Blocks test execution!
}

Handle Exceptions Carefully

Exceptions in interceptors can affect test execution:

@Override
public void postTest(ArgumentContext argumentContext, Method method, Throwable throwable) {
    try {
        // Interceptor logic that might fail
        saveTestResult(method, throwable);
    } catch (Exception e) {
        // Log but don't throw - don't cascade failures
        logger.error("Failed to save test result", e);
    }
}

Use Thread-Safe Collections

When collecting data across tests:

// Good: Thread-safe collection
private final ConcurrentHashMap<String, TestResult> results = new ConcurrentHashMap<>();

// Bad: Not thread-safe (will cause issues with parallel execution)
private final HashMap<String, TestResult> results = new HashMap<>();

Rethrow Original Exceptions

When intercepting failures, rethrow the original exception if you don’t handle it:

@Override
public void postTest(ArgumentContext argumentContext, Method method, Throwable throwable)
        throws Throwable {
    if (throwable != null) {
        logger.error("Test failed: {}", method.getName(), throwable);
    }
    rethrow(throwable); // Rethrow so test is marked as failed
}

Use Predicates for Targeted Interception

Don’t process every test class if you only care about specific ones:

@Override
public Predicate<ClassContext> predicate() {
    return classContext -> {
        Class<?> testClass = classContext.getTestClass();
        // Only intercept integration tests
        return testClass.getPackage().getName().contains(".integration.");
    };
}

Example: Complete Monitoring Interceptor

Here’s a complete example that demonstrates multiple interceptor features:

package com.example.interceptors;

import org.verifyica.api.*;
import java.lang.reflect.Method;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Predicate;

public class MonitoringInterceptor implements ClassInterceptor {

    private final Map<String, Metrics> metrics = new ConcurrentHashMap<>();

    @Override
    public void initialize(EngineContext engineContext) {
        System.out.println("Monitoring initialized");
    }

    @Override
    public Predicate<ClassContext> predicate() {
        // Only monitor classes annotated with @Monitored
        return classContext -> classContext.getTestClass()
            .isAnnotationPresent(Monitored.class);
    }

    @Override
    public void preBeforeAll(ArgumentContext argumentContext, Method method) {
        String key = getKey(argumentContext);
        metrics.put(key, new Metrics());
        metrics.get(key).startTime = System.currentTimeMillis();
    }

    @Override
    public void preTest(ArgumentContext argumentContext, Method method) {
        String key = getKey(argumentContext);
        metrics.get(key).testCount++;
        metrics.get(key).currentTestStart = System.nanoTime();
    }

    @Override
    public void postTest(ArgumentContext argumentContext, Method method, Throwable throwable) {
        String key = getKey(argumentContext);
        Metrics m = metrics.get(key);
        long duration = System.nanoTime() - m.currentTestStart;
        m.totalTestTime += duration;

        if (throwable != null) {
            m.failureCount++;
        } else {
            m.successCount++;
        }
    }

    @Override
    public void postAfterAll(ArgumentContext argumentContext, Method method, Throwable throwable) {
        String key = getKey(argumentContext);
        Metrics m = metrics.get(key);
        m.endTime = System.currentTimeMillis();

        System.out.printf("Metrics for %s:%n", key);
        System.out.printf("  Total time: %d ms%n", m.endTime - m.startTime);
        System.out.printf("  Test count: %d%n", m.testCount);
        System.out.printf("  Successes: %d%n", m.successCount);
        System.out.printf("  Failures: %d%n", m.failureCount);
        System.out.printf("  Avg test time: %.2f ms%n",
            m.totalTestTime / (double) m.testCount / 1_000_000);
    }

    @Override
    public void destroy(EngineContext engineContext) {
        System.out.println("Monitoring destroyed");
        // Could save aggregate metrics here
    }

    private String getKey(ArgumentContext argumentContext) {
        return argumentContext.getClassContext().getTestClass().getSimpleName() +
               ":" + argumentContext.getArgument().getName();
    }

    private static class Metrics {
        long startTime;
        long endTime;
        int testCount;
        int successCount;
        int failureCount;
        long totalTestTime;
        long currentTestStart;
    }
}

Next Steps

Execution Model - Understand when interceptors fire during execution
API Reference → Interceptor API - Complete interceptor API reference
Examples → Interceptor Examples - See more interceptor examples

5 - Execution Model

Understanding how Verifyica executes tests, manages parallelism, and processes arguments

Understanding Verifyica’s execution model is crucial for writing efficient, parallel-friendly tests and reasoning about test behavior.

Execution Overview

Verifyica executes tests in a structured lifecycle that provides isolation, parallelism, and predictable ordering.

High-Level Flow

1. Test Discovery
   ↓
2. Engine Initialization
   ↓
3. For each Test Class:
   a. Instantiate test class
   b. Call @Prepare
   c. Call @ArgumentSupplier
   d. For each Argument (potentially parallel):
      - Call @BeforeAll
      - For each @Test method:
         - Call @BeforeEach
         - Call @Test
         - Call @AfterEach
      - Call @AfterAll
   e. Call @Conclude
   ↓
4. Engine Shutdown

Test Discovery

Verifyica uses the JUnit Platform test discovery mechanism to find test classes.

Discovery Rules

A class is discovered as a Verifyica test if it:

Contains at least one @Verifyica.ArgumentSupplier method
Contains at least one @Verifyica.Test method
Is not abstract
Has a no-arg constructor (public or package-private)

Discovery Example

// Discovered: Has ArgumentSupplier and Test
public class ValidTest {

    @Verifyica.ArgumentSupplier
    public static Object arguments() { return List.of("arg"); }

    @Verifyica.Test
    public void test(String arg) { }
}

// NOT discovered: Missing ArgumentSupplier
public class NotATest {

    @Verifyica.Test
    public void test() { }
}

// NOT discovered: Abstract class
public abstract class AbstractTest {
    @Verifyica.ArgumentSupplier
    public static Object arguments() { return List.of("arg"); }

    @Verifyica.Test
    public void test(String arg) { }
}

Class Instantiation

A single test class instance is created and shared across all arguments.

Single Shared Instance

public class InstanceTest {

    private int counter = 0;

    @Verifyica.ArgumentSupplier
    public static Collection<String> arguments() {
        return Arrays.asList("arg1", "arg2", "arg3");
    }

    @Verifyica.BeforeAll
    public void beforeAll(String argument) {
        counter++; // Increments with each argument
    }

    @Verifyica.Test
    public void test(String argument) {
        System.out.println("Counter: " + counter);
        // Sequential: prints 1, 2, 3
        // Parallel: prints unpredictable values due to race condition!
    }
}

With 3 arguments, there is one instance of InstanceTest, and the counter field is shared across all arguments.

Implication: Instance Variables Require Careful Management

Since all arguments share the same instance, instance variables must be managed carefully:

✅ Safe Pattern: Argument-Specific State with Context Classes

public class SafeStateTest {
    // Define a context class to hold per-argument state
    public static class TestContext {
        private final Connection connection;

        public TestContext(Connection connection) {
            this.connection = connection;
        }

        public Connection getConnection() {
            return connection;
        }
    }

    @Verifyica.BeforeAll
    public void beforeAll(ArgumentContext argumentContext) {
        Config config = argumentContext.getArgument().getPayloadAs(Config.class);
        Connection conn = database.connect(config);

        // Store context in ArgumentContext map
        TestContext context = new TestContext(conn);
        argumentContext.getMap().put("testContext", context);
    }

    @Verifyica.Test
    public void test1(ArgumentContext argumentContext) {
        TestContext context = (TestContext) argumentContext.getMap().get("testContext");
        context.getConnection().execute("SELECT 1");
    }

    @Verifyica.AfterAll
    public void afterAll(ArgumentContext argumentContext) {
        TestContext context = (TestContext) argumentContext.getMap().get("testContext");
        if (context != null && context.getConnection() != null) {
            context.getConnection().close();
        }
    }
}

❌ Unsafe Pattern: Shared Mutable State

public class UnsafeStateTest {

    private Connection connection; // UNSAFE: shared across arguments!

    @Verifyica.BeforeAll
    public void beforeAll(Config config) {
        connection = database.connect(config);
        // If arguments run in parallel, this creates race conditions!
    }

    @Verifyica.Test
    public void test(Config config) {
        connection.execute("SELECT 1");
        // May use wrong connection or encounter race conditions!
    }
}

Argument Processing

Arguments are processed sequentially by default, but can be parallelized.

Sequential Processing (Default)

@Verifyica.ArgumentSupplier
public static Collection<String> arguments() {
    return Arrays.asList("arg1", "arg2", "arg3");
}

Execution order:

1. Process arg1 completely (BeforeAll → Tests → AfterAll)
2. Process arg2 completely (BeforeAll → Tests → AfterAll)
3. Process arg3 completely (BeforeAll → Tests → AfterAll)

Parallel Processing

@Verifyica.ArgumentSupplier(parallelism = 2)
public static Collection<String> arguments() {
    return Arrays.asList("arg1", "arg2", "arg3");
}

Execution with parallelism = 2:

1. Process arg1 and arg2 in parallel
2. When one completes, start arg3

Test Method Execution

Within an argument, test methods execute sequentially by default.

Sequential Test Execution

@Verifyica.Test
public void test1(String argument) {
    System.out.println("Test 1: " + argument);
}

@Verifyica.Test
public void test2(String argument) {
    System.out.println("Test 2: " + argument);
}

@Verifyica.Test
public void test3(String argument) {
    System.out.println("Test 3: " + argument);
}

For each argument, tests execute in order:

BeforeAll
  BeforeEach → Test1 → AfterEach
  BeforeEach → Test2 → AfterEach
  BeforeEach → Test3 → AfterEach
AfterAll

Test Method Ordering

Control test method order with @Order:

@Verifyica.Test
@Order(3)
public void test1() { }

@Verifyica.Test
@Order(1)
public void test2() { } // Executes first

@Verifyica.Test
@Order(2)
public void test3() { }

See Advanced → Ordering for details.

Parallelism Levels

Verifyica supports three levels of parallelism:

1. Class-Level Parallelism

Multiple test classes execute in parallel (controlled by test execution framework):

TestClass1 (parallel with)
TestClass2 (parallel with)
TestClass3

2. Argument-Level Parallelism

Multiple arguments within a class execute in parallel:

@Verifyica.ArgumentSupplier(parallelism = 4)
public static Collection<String> arguments() {
    return generateArguments();
}

With parallelism = 4:

arg1 (parallel)
arg2 (parallel)
arg3 (parallel)
arg4 (parallel)
arg5 (waits for slot)

3. Test Method Parallelism

Test methods within an argument can execute in parallel (configured globally):

# verifyica.properties
verifyica.test.parallelism=4

For each argument:
  BeforeAll
    BeforeEach → Test1 → AfterEach (parallel)
    BeforeEach → Test2 → AfterEach (parallel)
    BeforeEach → Test3 → AfterEach (parallel)
    BeforeEach → Test4 → AfterEach (parallel)
  AfterAll

Warning: Test method parallelism requires careful state management to avoid race conditions.

See Configuration → Parallelism for complete details.

Thread Safety Considerations

Safe Patterns

Instance Variables with Sequential Arguments (parallelism = 1)

public class SequentialArgumentsTest {

    private Connection connection; // Safe ONLY with parallelism = 1

    @Verifyica.ArgumentSupplier(parallelism = 1) // Sequential execution
    public static Collection<Config> arguments() {
        return getConfigs();
    }

    @Verifyica.BeforeAll
    public void beforeAll(Config config) {
        connection = database.connect(config);
    }

    @Verifyica.Test
    public void test(Config config) {
        connection.query("SELECT 1"); // Safe: one argument at a time
    }
}

Instance Variables with Parallel Arguments

For parallel argument execution, use context classes:

public class ParallelArgumentsTest {
    // Define a context class to encapsulate per-argument state
    public static class TestContext {
        private final Connection connection;

        public TestContext(Connection connection) {
            this.connection = connection;
        }

        public Connection getConnection() {
            return connection;
        }
    }

    @Verifyica.ArgumentSupplier(parallelism = 4) // Parallel execution
    public static Collection<Argument<Config>> arguments() {
        return getArguments();
    }

    @Verifyica.BeforeAll
    public void beforeAll(ArgumentContext argumentContext) {
        Config config = argumentContext.getArgument().getPayloadAs(Config.class);
        Connection conn = database.connect(config);

        // Store context in ArgumentContext map (thread-safe)
        TestContext context = new TestContext(conn);
        argumentContext.getMap().put("testContext", context);
    }

    @Verifyica.Test
    public void test(ArgumentContext argumentContext) {
        TestContext context = (TestContext) argumentContext.getMap().get("testContext");
        context.getConnection().query("SELECT 1"); // Safe: isolated per argument
    }

    @Verifyica.AfterAll
    public void afterAll(ArgumentContext argumentContext) {
        TestContext context = (TestContext) argumentContext.getMap().get("testContext");
        if (context != null && context.getConnection() != null) {
            context.getConnection().close();
        }
    }
}

Static Variables with Proper Synchronization

public class SynchronizedTest {

    private static final AtomicInteger counter = new AtomicInteger(0);

    @Verifyica.Test
    public void test(String argument) {
        counter.incrementAndGet(); // Safe: atomic operation
    }
}

Unsafe Patterns

Unsynchronized Static State

public class UnsafeTest {

    private static int counter = 0; // UNSAFE with parallelism!

    @Verifyica.Test
    public void test(String argument) {
        counter++; // Race condition!
    }
}

Shared Mutable State in Instance Variables with Test Parallelism

public class UnsafeWithTestParallelism {

    private int value = 0; // UNSAFE if test methods run in parallel

    @Verifyica.Test
    public void test1(String argument) {
        value = 1; // Race condition if test2 runs concurrently
    }

    @Verifyica.Test
    public void test2(String argument) {
        value = 2; // Race condition if test1 runs concurrently
    }
}

Error Handling and Propagation

Exception in @Prepare

@Verifyica.Prepare
public void prepare() {
    throw new RuntimeException("Setup failed");
}

Effect: All test execution stops. No arguments are processed.

Exception in @BeforeAll

@Verifyica.BeforeAll
public void beforeAll(String argument) {
    throw new RuntimeException("Argument setup failed");
}

Effect: All tests for this argument are skipped. Other arguments continue.

Exception in @Test

@Verifyica.Test
public void test(String argument) {
    throw new AssertionError("Test failed");
}

Effect: Test marked as failed. @AfterEach still executes. Other tests continue.

@AfterEach and @AfterAll Always Execute

Cleanup methods execute even if tests fail:

@Verifyica.AfterEach
public void afterEach(String argument) {
    // Executes even if test failed
    cleanup();
}

@Verifyica.AfterAll
public void afterAll(String argument) {
    // Executes even if all tests failed
    connection.close();
}

Execution State Machine

Each argument progresses through states:

PENDING → RUNNING_BEFORE_ALL → RUNNING_TESTS → RUNNING_AFTER_ALL → COMPLETED
                                     ↓
                              (If test fails: continues to AFTER_ALL)

Failed states:

PENDING → FAILED_BEFORE_ALL → SKIPPED (tests skipped)
PENDING → RUNNING_BEFORE_ALL → RUNNING_TESTS → FAILED_TEST → RUNNING_AFTER_ALL → COMPLETED

Performance Considerations

Optimize Argument-Level Parallelism

// Good: Parallel argument processing for independent arguments
@Verifyica.ArgumentSupplier(parallelism = 4)
public static Collection<Config> arguments() {
    return getIndependentConfigs(); // Each config is independent
}

Keep @Prepare and @Conclude Fast

// Good: Fast global setup
@Verifyica.Prepare
public void prepare() {
    System.setProperty("test.mode", "true");
}

// Bad: Slow operations block everything
@Verifyica.Prepare
public void prepare() throws InterruptedException {
    Thread.sleep(10000); // Blocks all test execution!
}

Use @BeforeAll for Expensive Per-Argument Setup

// Good: Expensive setup once per argument
@Verifyica.BeforeAll
public void beforeAll(Config config) {
    connection = database.connect(config); // Expensive, do once per argument
}

// Bad: Expensive setup repeated for every test
@Verifyica.BeforeEach
public void beforeEach(Config config) {
    connection = database.connect(config); // Wasteful, reconnects for every test!
}

Note: This example demonstrates efficiency patterns. For thread-safe state management with parallel arguments, use context classes stored in the ArgumentContext map. See the Safe Patterns section above.

Execution Timeline Example

For this test:

@Verifyica.ArgumentSupplier(parallelism = 2)
public static Collection<String> arguments() {
    return Arrays.asList("arg1", "arg2", "arg3");
}

With 2 test methods, the timeline is:

Time 0: Prepare
Time 1: ArgumentSupplier returns ["arg1", "arg2", "arg3"]
Time 2: Start arg1 and arg2 in parallel
  Thread 1:
    - BeforeAll(arg1)
    - BeforeEach(arg1) → Test1(arg1) → AfterEach(arg1)
    - BeforeEach(arg1) → Test2(arg1) → AfterEach(arg1)
    - AfterAll(arg1)
  Thread 2 (parallel):
    - BeforeAll(arg2)
    - BeforeEach(arg2) → Test1(arg2) → AfterEach(arg2)
    - BeforeEach(arg2) → Test2(arg2) → AfterEach(arg2)
    - AfterAll(arg2)
Time 3: First thread to finish picks up arg3
  Thread 1 or 2:
    - BeforeAll(arg3)
    - BeforeEach(arg3) → Test1(arg3) → AfterEach(arg3)
    - BeforeEach(arg3) → Test2(arg3) → AfterEach(arg3)
    - AfterAll(arg3)
Time 4: Conclude

Next Steps

Configuration → Parallelism - Configure parallelism in detail
Advanced → Parallelism - Advanced parallel execution patterns
Advanced → Ordering - Control test method execution order