0
# Test Discovery
1

2
Test discovery system using fingerprints to identify test classes through annotations or inheritance patterns, enabling automatic detection of test suites during SBT's discovery phase.
3

4
## Capabilities
5

6
### Fingerprint Base Interface
7

8
Base interface for test class identification patterns used during discovery.
9

10
```scala { .api }
11
/**
12
 * A way to identify test classes and/or modules that should be discovered.
13
 * Implementations may not rely on identity of Fingerprints since they are serialized between JS/JVM.
14
 */
15
trait Fingerprint
16
```
17

18
### AnnotatedFingerprint
19

20
Identifies test classes or modules based on the presence of specific annotations.
21

22
```scala { .api }
23
/**
24
 * Indicates classes or modules with a specific annotation should be discovered as test classes.
25
 * The annotation can be on top-level methods or on the class/module itself.
26
 */
27
trait AnnotatedFingerprint extends Fingerprint {
28
  /**
29
   * Indicates whether modules with the annotation should be considered during discovery.
30
   * If a framework allows both classes and modules, return two different fingerprints:
31
   * one with isModule() = false and another with isModule() = true.
32
   * @return true for modules (singleton objects), false for classes
33
   */
34
  def isModule(): Boolean
35
  
36
  /**
37
   * The fully qualified name of the annotation that identifies test classes/modules.
38
   * @return annotation name for test identification
39
   */
40
  def annotationName(): String
41
}
42
```
43

44
**Usage Examples:**
45

46
```scala
47
// Discover classes with @Test annotation
48
val classAnnotationFingerprint = new AnnotatedFingerprint {
49
  def isModule() = false
50
  def annotationName() = "org.junit.Test"
51
}
52

53
// Discover objects with @TestSuite annotation  
54
val objectAnnotationFingerprint = new AnnotatedFingerprint {
55
  def isModule() = true
56
  def annotationName() = "com.example.TestSuite"
57
}
58

59
// Framework returning both fingerprints
60
class MyFramework extends Framework {
61
  def fingerprints(): Array[Fingerprint] = Array(
62
    classAnnotationFingerprint,
63
    objectAnnotationFingerprint
64
  )
65
}
66
```
67

68
### SubclassFingerprint
69

70
Identifies test classes or modules based on inheritance from specific superclasses or traits.
71

72
```scala { .api }
73
/**
74
 * Indicates classes (and possibly modules) that extend a particular superclass
75
 * or mix in a particular supertrait should be discovered as test classes.
76
 */
77
trait SubclassFingerprint extends Fingerprint {
78
  /**
79
   * Indicates whether modules (singleton objects) that extend the superclass/supertrait
80
   * should be considered during discovery.
81
   * Returning false speeds up discovery by quickly bypassing module classes.
82
   * @return true to discover modules, false for classes only
83
   */
84
  def isModule(): Boolean
85
  
86
  /**
87
   * The name of the superclass or supertrait that identifies test classes.
88
   * @return superclass/supertrait name for test identification
89
   */
90
  def superclassName(): String
91
  
92
  /**
93
   * Indicates whether discovered classes must have a no-arg constructor.
94
   * If true, client should not discover subclasses without no-arg constructors.
95
   * @return true if no-arg constructor required
96
   */
97
  def requireNoArgConstructor(): Boolean
98
}
99
```
100

101
**Usage Examples:**
102

103
```scala
104
// Discover classes extending TestSuite (with no-arg constructor)
105
val suiteFingerprint = new SubclassFingerprint {
106
  def isModule() = false
107
  def superclassName() = "com.example.TestSuite"
108
  def requireNoArgConstructor() = true
109
}
110

111
// Discover objects extending SpecBase (no constructor requirement for objects)
112
val specFingerprint = new SubclassFingerprint {
113
  def isModule() = true
114
  def superclassName() = "com.example.SpecBase" 
115
  def requireNoArgConstructor() = false
116
}
117

118
// Discover both classes and objects extending BaseTest
119
val flexibleFingerprint = new SubclassFingerprint {
120
  def isModule() = false // or true for separate fingerprint
121
  def superclassName() = "com.example.BaseTest"
122
  def requireNoArgConstructor() = false
123
}
124
```
125

126
## Discovery Process
127

128
### Framework Fingerprint Registration
129

130
Test frameworks register their discovery patterns by returning fingerprints from the `fingerprints()` method:
131

132
```scala
133
class MyTestFramework extends Framework {
134
  def fingerprints(): Array[Fingerprint] = Array(
135
    // JUnit-style annotation discovery
136
    new AnnotatedFingerprint {
137
      def isModule() = false
138
      def annotationName() = "org.junit.Test"
139
    },
140
    
141
    // ScalaTest-style inheritance discovery
142
    new SubclassFingerprint {
143
      def isModule() = false
144
      def superclassName() = "org.scalatest.Suite"
145
      def requireNoArgConstructor() = true
146
    },
147
    
148
    // Specs2-style object discovery
149
    new SubclassFingerprint {
150
      def isModule() = true
151
      def superclassName() = "org.specs2.Specification"
152
      def requireNoArgConstructor() = false
153
    }
154
  )
155
}
156
```
157

158
### Discovery Algorithm
159

160
SBT uses fingerprints to scan the classpath and identify test classes:
161

162
1. **Classpath Scanning**: SBT scans compiled classes in test directories
163
2. **Fingerprint Matching**: Each class/object checked against all registered fingerprints
164
3. **Annotation Checking**: For `AnnotatedFingerprint`, check for specified annotations
165
4. **Inheritance Checking**: For `SubclassFingerprint`, check superclass hierarchy
166
5. **Constructor Validation**: If `requireNoArgConstructor()` is true, verify no-arg constructor exists
167
6. **TaskDef Creation**: Create `TaskDef` instances for discovered test classes
168
7. **Task Creation**: Pass `TaskDef` array to `Runner.tasks()` for task creation
169

170
### Multiple Fingerprints
171

172
Frameworks can register multiple fingerprints to support different discovery patterns:
173

174
```scala
175
def fingerprints(): Array[Fingerprint] = Array(
176
  // Method-level annotations
177
  new AnnotatedFingerprint {
178
    def isModule() = false
179
    def annotationName() = "com.example.Test"
180
  },
181
  
182
  // Class-level annotations
183
  new AnnotatedFingerprint {
184
    def isModule() = false  
185
    def annotationName() = "com.example.TestClass"
186
  },
187
  
188
  // Inheritance-based discovery
189
  new SubclassFingerprint {
190
    def isModule() = false
191
    def superclassName() = "com.example.BaseTest"
192
    def requireNoArgConstructor() = true
193
  },
194
  
195
  // Object-based specs
196
  new SubclassFingerprint {
197
    def isModule() = true
198
    def superclassName() = "com.example.Spec"
199
    def requireNoArgConstructor() = false // Objects don't have constructors
200
  }
201
)
202
```
203

204
### Discovery Best Practices
205

206
**Performance Optimization:**
207
- Set `isModule() = false` for class-only frameworks to skip object scanning
208
- Use `requireNoArgConstructor() = true` to filter classes early
209
- Keep `superclassName()` and `annotationName()` as specific as possible
210

211
**Flexibility:**
212
- Support both classes and objects by providing separate fingerprints
213
- Consider annotation-based and inheritance-based patterns
214
- Allow for different constructor requirements
215

216
**Framework Compatibility:**
217
- Use standard annotation names when possible (e.g., JUnit annotations)
218
- Follow common inheritance patterns (e.g., ScalaTest Suite)
219
- Support both new and legacy test patterns
220

221
```scala
222
// Comprehensive discovery for flexible framework
223
class FlexibleFramework extends Framework {
224
  def fingerprints(): Array[Fingerprint] = Array(
225
    // JUnit compatibility
226
    new AnnotatedFingerprint {
227
      def isModule() = false
228
      def annotationName() = "org.junit.Test"
229
    },
230
    
231
    // Custom annotation support
232
    new AnnotatedFingerprint {
233
      def isModule() = false
234
      def annotationName() = "com.myframework.Test"
235
    },
236
    
237
    // Suite-based classes
238
    new SubclassFingerprint {
239
      def isModule() = false
240
      def superclassName() = "com.myframework.TestSuite"
241
      def requireNoArgConstructor() = true
242
    },
243
    
244
    // Spec-based objects
245
    new SubclassFingerprint {
246
      def isModule() = true
247
      def superclassName() = "com.myframework.Spec"
248
      def requireNoArgConstructor() = false
249
    }
250
  )
251
}
252
```
253

254
## Error Handling
255

256
### Discovery Failures
257

258
Discovery typically handles errors gracefully by skipping problematic classes:
259

260
```scala
261
// Framework should handle discovery errors internally
262
def fingerprints(): Array[Fingerprint] = {
263
  try {
264
    Array(
265
      createAnnotationFingerprint(),
266
      createSubclassFingerprint()
267
    )
268
  } catch {
269
    case _: ClassNotFoundException =>
270
      // Fallback to basic discovery
271
      Array(createBasicFingerprint())
272
  }
273
}
274
```
275

276
### Validation
277

278
Fingerprints should validate their configuration:
279

280
```scala
281
class MyAnnotatedFingerprint(annotation: String) extends AnnotatedFingerprint {
282
  require(annotation != null && annotation.nonEmpty, "Annotation name cannot be empty")
283
  
284
  def annotationName() = annotation
285
  def isModule() = false
286
}
287
```