or run

npx @tessl/cli init

Version

Tile

Overview

Evals

Files

docs

core-specifications.md dsl-components.md index.md integration-features.md matcher-system.md mutable-specifications.md reporting.md test-execution.md

tile.json

Test Execution and Configuration

Specs2 provides comprehensive test execution capabilities with flexible configuration options, multiple runners for different environments, and detailed control over test execution behavior.

Core Execution

Runner

Main execution utility for running specifications.

object Runner {
  def execute(actions: Action[Unit], arguments: Arguments, exit: Boolean): Unit
  def execute(specs: Seq[SpecificationStructure], arguments: Arguments, exit: Boolean): Unit
  def run(specs: SpecificationStructure*): Unit
}

Executor

Core execution engine that processes specification fragments using stream-based processing.

trait Executor {
  def execute(env: Env): Process[Task, Fragment] => Process[Task, Fragment]
  def execute1(env: Env): Process1[Fragment, Task[Fragment]]
}

The executor is responsible for:

Converting fragments into executable tasks
Managing parallel vs sequential execution
Handling fragment dependencies and ordering
Coordinating step and action execution

DefaultExecutor

Default implementation of the Executor trait using scalaz-stream processing.

trait DefaultExecutor extends Executor {
  def execute1(env: Env): Process1[Fragment, Task[Fragment]]
  def sequencedExecution(env: Env): Process1[Fragment, Task[Fragment]]
  def isolatedExecution(env: Env): Process1[Fragment, Task[Fragment]]
}

Execution Strategies:

Sequenced: Examples run one after another
Isolated: Each example runs in complete isolation
Parallel: Examples run concurrently (default)

Env (Environment)

Execution environment that holds configuration and resources for test execution.

case class Env(
  arguments: Arguments,
  systemLogger: Logger,
  printers: List[Printer] = Nil,
  statisticsRepository: StatisticsRepository = StatisticsRepository.memory,
  random: scala.util.Random = new scala.util.Random,
  fileSystem: FileSystem = FileSystem,
  executionContext: ExecutionContext = ExecutionContext.global
) {
  def setArguments(args: Arguments): Env
  def setPrinters(ps: List[Printer]): Env
  def addPrinter(p: Printer): Env
}

The environment provides:

Execution arguments and configuration
Logging infrastructure
Output printers for reporting
Statistics collection
File system access
Execution context for concurrent operations

Fragment Execution Lifecycle

Process-Based Execution Model

Specs2 uses scalaz-stream Process[Task, Fragment] for fragment processing, enabling:

Lazy evaluation: Fragments are processed on-demand
Stream composition: Fragment streams can be combined and transformed
Resource management: Automatic cleanup of resources
Backpressure handling: Controlled memory usage for large test suites

// Core execution pipeline
trait SpecificationExecution {
  def executeFragments(env: Env): Process[Task, Fragment] => Process[Task, Fragment]
  def executeBodies(env: Env): Process1[Fragment, Fragment]
  def executeActions(env: Env): Process1[Fragment, Fragment]  
}

Fragment Processing Pipeline

Fragment Creation: Specification is method creates fragment stream
Body Execution: Example bodies are executed to produce results
Action Processing: Steps and actions are executed at appropriate times
Result Collection: Results are collected for reporting
Statistics Gathering: Execution statistics are accumulated

Synchronization and Ordering

trait SpecificationStructure {
  def fragments: Process[Task, Fragment]
  def linked: Process[Task, Fragment]
  def contents: Process[Task, Fragment]
}

Fragment Ordering:

Steps: Executed immediately during fragment stream processing
Examples: Executed according to execution strategy (parallel/sequential)
Actions: Executed after example completion
Text/Formatting: Processed for output formatting

Usage Example:

// Run a single specification
Runner.run(new MySpec)

// Run multiple specifications  
Runner.run(new UserSpec, new OrderSpec, new PaymentSpec)

// Run with custom arguments
Runner.execute(
  List(new IntegrationSpec), 
  Arguments(sequential = true, stopOnFail = true),
  exit = false
)

Test Runners

ClassRunner

Runs specifications from fully qualified class names.

class ClassRunner {
  def run(className: String): Unit
  def run(className: String, arguments: Arguments): Unit
  def run(classNames: List[String], arguments: Arguments): Unit
}

Usage:

# Command line execution
scala -cp "classpath" org.specs2.runner.ClassRunner com.example.UserSpec

# With arguments
scala -cp "classpath" org.specs2.runner.ClassRunner com.example.UserSpec -- sequential

FilesRunner

Discovers and runs specification files from filesystem.

class FilesRunner {
  def run(path: String): Unit
  def run(path: String, pattern: String): Unit
  def run(paths: List[String], arguments: Arguments): Unit
}

Usage:

# Run all specs in directory
scala -cp "classpath" org.specs2.runner.FilesRunner src/test/scala

# Run specs matching pattern
scala -cp "classpath" org.specs2.runner.FilesRunner src/test/scala ".*UserSpec.*"

SbtRunner

Integration with SBT test framework.

class SbtRunner extends TestFramework {
  def name: String = "specs2"
  def fingerprints: Array[Fingerprint]
  def runner(args: Array[String], remoteArgs: Array[String], testClassLoader: ClassLoader): Runner2
}

SBT Configuration:

// build.sbt
libraryDependencies += "org.specs2" %% "specs2-core" % "3.3.1" % "test"
testFrameworks += TestFrameworks.Specs2

// Run all tests
sbt test

// Run specific test
sbt "testOnly com.example.UserSpec"

// Run with arguments
sbt "testOnly * -- sequential stopOnFail"

NotifierRunner

Integration with test framework notifiers for IDE support.

class NotifierRunner {
  def run(specs: List[SpecificationStructure], notifier: EventHandler): Unit
  def run(className: String, notifier: EventHandler): Unit
}

Configuration System

Arguments

Main configuration object controlling test execution behavior.

case class Arguments(
  // Execution control
  plan: Boolean = false,
  skipAll: Boolean = false,
  stopOnFail: Boolean = false,
  sequential: Boolean = false,
  isolated: Boolean = false,
  threadsNb: Int = Runtime.getRuntime.availableProcessors,
  
  // Filtering
  include: String = "",
  exclude: String = "",
  
  // Output control
  colors: Boolean = true,
  noindent: Boolean = false,
  showTimes: Boolean = false,
  offset: Int = 0,
  
  // Reporting
  console: Boolean = true,
  html: Boolean = false,
  markdown: Boolean = false,
  junitxml: Boolean = false,
  
  // File paths
  outdir: String = "target/specs2-reports",
  srcdir: String = "src/test/scala"
)

ArgumentsArgs

Command-line argument parsing for Arguments object.

trait ArgumentsArgs {
  def parse(args: Array[String]): Arguments
  def parseArguments(args: String*): Arguments
}

Command Line Usage:

# Basic arguments
--sequential          # Run examples sequentially
--stopOnFail         # Stop on first failure  
--plan              # Show execution plan only
--skipAll           # Skip all examples

# Filtering  
--include "unit"     # Include examples tagged with "unit"
--exclude "slow"     # Exclude examples tagged with "slow"

# Output control
--colors            # Enable colors (default)
--noColors          # Disable colors
--showTimes         # Show execution times
--offset 2          # Indent output by 2 spaces

# Reporting formats
--console           # Console output (default)
--html              # Generate HTML reports
--markdown          # Generate Markdown reports  
--junitxml          # Generate JUnit XML reports

# File paths
--outdir target/reports    # Output directory
--srcdir src/test/scala   # Source directory

ArgumentsShortcuts

Convenient methods for creating common argument combinations.

trait ArgumentsShortcuts {
  def sequential: Arguments
  def isolated: Arguments
  def stopOnFail: Arguments
  def plan: Arguments
  def skipAll: Arguments
  def noColors: Arguments
  def showTimes: Arguments
  def html: Arguments
  def markdown: Arguments
  def junitxml: Arguments
}

Usage in Specifications:

class ConfiguredSpec extends Specification { def is = 
  args(sequential, stopOnFail, showTimes) ^ s2"""
  Configured specification
    test 1    $test1  
    test 2    $test2
    test 3    $test3
  """
}

Execution Control

Execute

Execution lifecycle control and coordination.

trait Execute {
  def execute[T](action: Action[T]): T
  def execute[T](action: Action[T], timeout: FiniteDuration): T
  def executeAll[T](actions: List[Action[T]]): List[T]
  def executeSequentially[T](actions: List[Action[T]]): List[T]
  def executeConcurrently[T](actions: List[Action[T]]): List[T]
}

Action[T]

Monadic action type for composable operations with error handling.

case class Action[T](run: () => T) {
  def map[S](f: T => S): Action[S]
  def flatMap[S](f: T => Action[S]): Action[S]
  def filter(p: T => Boolean): Action[T]
  def recover(f: Throwable => T): Action[T]
  def timeout(duration: FiniteDuration): Action[T]
}

Usage Example:

val setupAction = Action { () =>
  Database.createTables()
  Database.seedTestData()
}

val testAction = Action { () =>
  Database.query("SELECT * FROM users") must not(beEmpty)
}

val cleanupAction = Action { () =>
  Database.cleanup()
}

val composedAction = for {
  _ <- setupAction
  result <- testAction
  _ <- cleanupAction
} yield result

Specification Discovery

SpecificationsFinder

Discovers specification classes dynamically.

trait SpecificationsFinder {
  def find(pattern: String): List[String] 
  def find(classLoader: ClassLoader, pattern: String): List[String]
  def findSpecifications(packages: List[String]): List[SpecificationStructure]
}

Usage:

val finder = new SpecificationsFinder {}
val specClasses = finder.find("com.example.*Spec")
val specifications = finder.findSpecifications(List("com.example"))

Specification Path Discovery

Automatic discovery patterns:

// File-based discovery
val fileSpecs = FilesRunner.findSpecifications("src/test/scala")

// Package-based discovery  
val packageSpecs = ClassRunner.findSpecifications("com.example")

// Pattern-based discovery
val patternSpecs = SpecificationsFinder.find(".*IntegrationSpec.*")

Execution Modes

Sequential Execution

Run examples one after another:

class SequentialSpec extends Specification { def is = sequential ^ s2"""
  Sequential execution (examples run in order)
    setup step         $setup
    test operation     $testOp  
    verify result      $verify
    cleanup step       $cleanup
  """
}

Parallel Execution

Run examples concurrently (default):

class ParallelSpec extends Specification { def is = s2"""
  Parallel execution (examples run concurrently)  
    independent test 1    $test1
    independent test 2    $test2
    independent test 3    $test3
  """
}

Isolated Execution

Run each example in isolation:

class IsolatedSpec extends Specification { def is = isolated ^ s2"""
  Isolated execution (examples in separate classloaders)
    test with global state 1    $testGlobal1
    test with global state 2    $testGlobal2  
  """
}

Filtering and Selection

Tag-Based Filtering

Filter examples by tags:

# Include specific tags
sbt "testOnly * -- include unit,fast"

# Exclude specific tags  
sbt "testOnly * -- exclude slow,integration"

# Complex filtering
sbt "testOnly * -- include unit exclude broken"

Pattern-Based Filtering

Filter by specification or example names:

# Run specifications matching pattern
sbt "testOnly *UserSpec*"

# Run examples matching description
sbt "testOnly * -- ex create.*user"

Conditional Execution

Skip examples based on conditions:

class ConditionalSpec extends Specification { def is = s2"""
  Conditional execution
    ${if (System.getProperty("integration") == "true") 
        "integration test" ! integrationTest
      else 
        "integration test (skipped)" ! skipped
    }
    unit test    $unitTest
  """
}

Error Handling and Recovery

StopOnFail

Stop execution on first failure:

class StopOnFailSpec extends Specification { def is = 
  args(stopOnFail = true) ^ s2"""
  Stop on fail behavior
    test 1 (will pass)     $test1
    test 2 (will fail)     $test2  
    test 3 (won't run)     $test3
  """

Exception Handling

Handle exceptions during execution:

class ExceptionHandlingSpec extends Specification { def is = s2"""
  Exception handling  
    recoverable error    $recoverableError
    fatal error         $fatalError
  """
  
  def recoverableError = {
    try {
      riskyOperation()
      success
    } catch {
      case _: RecoverableException => 
        fallbackOperation() must beEqualTo(expected)
    }
  }
  
  def fatalError = {
    fatalOperation() must throwA[FatalException]
  }
}

Performance and Monitoring

Execution Timing

Track execution performance:

# Show execution times
sbt "testOnly * -- showTimes"

# Show times with colors
sbt "testOnly * -- showTimes colors"

Thread Configuration

Control concurrent execution:

// Limit concurrent threads
Arguments(threadsNb = 4)

// Use all available processors (default)
Arguments(threadsNb = Runtime.getRuntime.availableProcessors)

Memory Management

Control memory usage during execution:

// Run with isolated classloaders (higher memory)
Arguments(isolated = true)

// Run sequentially (lower memory)  
Arguments(sequential = true)

Integration Patterns

Continuous Integration

CI-friendly configuration:

class CISpec extends Specification { def is = 
  args(
    stopOnFail = true,    // Fail fast
    noColors = true,      // No ANSI colors
    junitxml = true,      // Generate XML reports
    html = false          // Skip HTML for CI
  ) ^ s2"""
  CI-optimized specification
    critical test 1    $test1
    critical test 2    $test2
  """

IDE Integration

IDE-friendly execution:

// IntelliJ IDEA runner support
@RunWith(classOf[JUnitRunner])
class IDESpec extends Specification { def is = s2"""
  IDE-compatible specification
    test 1    $test1
    test 2    $test2
  """
}

Build Tool Integration

Maven integration:

<plugin>
  <groupId>org.scalatest</groupId>
  <artifactId>scalatest-maven-plugin</artifactId>
  <configuration>
    <reportsDirectory>${project.build.directory}/surefire-reports</reportsDirectory>
    <junitxml>.</junitxml>
    <filereports>WDF TestSuite.txt</filereports>
  </configuration>
</plugin>

Best Practices

Use appropriate execution mode: Sequential for dependent tests, parallel for independent tests
Configure for environment: Different arguments for development vs CI
Filter strategically: Use tags and patterns for selective test execution
Handle failures gracefully: Use stopOnFail judiciously and handle exceptions properly
Monitor performance: Use timing information to identify slow tests
Optimize resource usage: Balance parallelism with available resources
Integrate with tools: Configure for CI/CD and IDE support
Document execution requirements: Clearly specify any special execution needs

Version

Tile

Files

test-execution.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Test Execution and Configuration

Core Execution

Runner

Executor

DefaultExecutor

Env (Environment)

Fragment Execution Lifecycle

Process-Based Execution Model

Fragment Processing Pipeline

Synchronization and Ordering

Test Runners

ClassRunner

FilesRunner

SbtRunner

NotifierRunner

Configuration System

Arguments

ArgumentsArgs

ArgumentsShortcuts

Execution Control

Execute

Action[T]

Specification Discovery

SpecificationsFinder

Specification Path Discovery

Execution Modes

Sequential Execution

Parallel Execution

Isolated Execution

Filtering and Selection

Tag-Based Filtering

Pattern-Based Filtering

Conditional Execution

Error Handling and Recovery

StopOnFail

Exception Handling

Performance and Monitoring

Execution Timing

Thread Configuration

Memory Management

Integration Patterns

Continuous Integration

IDE Integration

Build Tool Integration

Best Practices

test-execution.mddocs/