or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

configuration.mdexecution.mdindex.mdproperty-creation.mdtype-integration.md
tile.json

execution.mddocs/

Execution

Execute property-based tests with comprehensive result reporting, failure analysis, seed capture for reproducibility, and statistical data collection. The execution system handles property evaluation, result interpretation, and detailed error reporting with shrinking of counterexamples.

Capabilities

Property Execution

Execute individual properties and property collections with configurable parameters and comprehensive result reporting.

trait ScalaCheckPropertyCheck extends ExpectationsCreation {
  def check(
    prop: Prop, 
    parameters: Parameters, 
    prettyFreqMap: FreqMap[Set[Any]] => Pretty
  ): Result
  
  def checkProperties(
    properties: Properties, 
    parameters: Parameters, 
    prettyFreqMap: FreqMap[Set[Any]] => Pretty
  ): Result
}

Usage:

import org.specs2._
import org.specs2.scalacheck._
import org.scalacheck._

class ExecutionSpec extends Specification with ScalaCheckPropertyCheck { def is = s2"""
  Manual property execution ${
    val prop = Prop.forAll((n: Int) => n + 0 == n)
    check(prop, Parameters().verbose, defaultFreqMapPretty) must beASuccess
  }
  
  Properties collection ${
    val props = new Properties("arithmetic") {
      property("addition") = Prop.forAll((a: Int, b: Int) => a + b == b + a)
      property("multiplication") = Prop.forAll((x: Int) => x * 1 == x)
    }
    checkProperties(props, Parameters(), defaultFreqMapPretty) must beASuccess
  }
"""
}

Result Interpretation

Process property execution results with detailed failure analysis, exception handling, and statistical reporting.

trait ScalaCheckPropertyCheck {
  def checkResultFailure(checkResult: Result): Result
  def showCause(t: Throwable): String
  def frequencies(fq: FreqMap[Set[Any]], parameters: Parameters, prettyFreqMap: FreqMap[Set[Any]] => Pretty): String
  def prettyResult(
    res: Test.Result, 
    parameters: Parameters, 
    initialSeed: =>Seed,
    freqMapPretty: FreqMap[Set[Any]] => Pretty
  ): Pretty
}

Result types include:

  • Success: Property passed all tests
  • Failure: Property failed with counterexample and shrinking
  • Error: Exception occurred during property evaluation
  • Exhausted: Too many discarded tests

Seed Capture and Reproducibility

Automatically capture random seeds used during property execution for reproducible test runs and debugging.

// Seed is automatically captured during property execution
// and included in failure reports for reproducibility

Usage:

class ReproducibilitySpec extends Specification with ScalaCheck { def is = s2"""
  Failing property debugging ${ 
    // When a property fails, the seed will be reported:
    // "The seed is AbCdEf1234567890"
    // Use this seed to reproduce the exact failure:
    prop((data: ComplexType) => processData(data).isValid)
      .setSeed("AbCdEf1234567890")  // Reproduce exact failure
  }
"""
}

Statistical Data Collection

Collect and display statistical information about generated test data using frequency maps and custom collectors.

// Frequency map handling for statistical reporting
def frequencies(fq: FreqMap[Set[Any]], parameters: Parameters, prettyFreqMap: FreqMap[Set[Any]] => Pretty): String

// Pretty printing of frequency maps
implicit def defaultFreqMapPretty: FreqMap[Set[Any]] => Pretty

Usage:

class StatisticsSpec extends Specification with ScalaCheck { def is = s2"""
  Data distribution analysis ${
    prop((x: Int) => x * x >= 0)
      .collectArg(x => if (x >= 0) "positive" else "negative")
      .display()  // Shows frequency distribution
  }
  
  Complex data collection ${
    prop((data: List[String]) => data.length >= 0)
      .collectArg(_.length)
      .collectArg(data => s"size-bucket-${data.length / 10}")
      .verbose
  }
"""
}

Exception Handling

Handle various types of exceptions that can occur during property evaluation with appropriate error categorization.

// Exception types handled:
// - FailureException: Converted to normal failures
// - DecoratedResultException: Enhanced failure reporting  
// - AssertionError: Test assertion failures
// - SkipException: Skipped tests
// - PendingException: Pending tests
// - NonFatal exceptions: Converted to errors

Usage:

class ExceptionHandlingSpec extends Specification with ScalaCheck { def is = s2"""
  Exception in property ${
    prop((n: Int) => {
      if (n == 42) throw new IllegalArgumentException("Special case")
      n >= 0 || n < 0  // Always true except for exception case
    }).set(minTestsOk = 1000)  // Increase chances of hitting exception
  }
  
  Assertion failure ${
    prop((s: String) => {
      assert(s.length >= 0, "String length must be non-negative")
      true
    })
  }
"""
}

Pretty Printing and Reporting

Format test results with detailed information including test statistics, failure details, and execution time.

def prettyResult(
  res: Test.Result, 
  parameters: Parameters, 
  initialSeed: =>Seed,
  freqMapPretty: FreqMap[Set[Any]] => Pretty
): Pretty

Output includes:

  • Passed tests: "OK, passed N tests"
  • Failed tests: "Falsified after N passed tests" with counterexample
  • Proved properties: "OK, proved property" with arguments
  • Exhausted tests: "Gave up after only N passed tests"
  • Seed information: Random seed for reproducibility
  • Execution time: Performance metrics
  • Frequency maps: Statistical data collection

Types

// Property checking trait
trait ScalaCheckPropertyCheck extends ExpectationsCreation {
  def check(prop: Prop, parameters: Parameters, prettyFreqMap: FreqMap[Set[Any]] => Pretty): Result
  def checkProperties(properties: Properties, parameters: Parameters, prettyFreqMap: FreqMap[Set[Any]] => Pretty): Result
  def checkResultFailure(checkResult: Result): Result
  def showCause(t: Throwable): String
  def frequencies(fq: FreqMap[Set[Any]], parameters: Parameters, prettyFreqMap: FreqMap[Set[Any]] => Pretty): String
  def prettyResult(res: Test.Result, parameters: Parameters, initialSeed: =>Seed, freqMapPretty: FreqMap[Set[Any]] => Pretty): Pretty
}

// Result types (from specs2)
sealed trait Result
case class Success(message: String, expected: String, successCount: Int) extends Result
case class Failure(message: String, expected: String = "", details: Details = NoDetails, trace: List[StackTraceElement] = Nil) extends Result
case class Error(message: String, exception: Throwable) extends Result