or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

Files

docs

custom-config.mddata-formats.mdindex.mdrunner-annotations.mdutilities.md

runner-annotations.mddocs/

0

# Runner and Basic Annotations

1

2

This document covers the core components needed to use JUnit DataProvider: the custom runner and the two primary annotations.

3

4

## Required Imports

5

6

```java

7

import com.tngtech.java.junit.dataprovider.DataProvider;

8

import com.tngtech.java.junit.dataprovider.DataProviderRunner;

9

import com.tngtech.java.junit.dataprovider.DataProviderFilter;

10

import com.tngtech.java.junit.dataprovider.DataProviderFrameworkMethod;

11

import com.tngtech.java.junit.dataprovider.UseDataProvider;

12

import org.junit.runner.manipulation.Filter;

13

import org.junit.runner.manipulation.NoTestsRemainException;

14

import org.junit.runners.BlockJUnit4ClassRunner;

15

import org.junit.runners.model.FrameworkMethod;

16

import org.junit.runners.model.InitializationError;

17

```

18

19

## DataProviderRunner

20

21

The DataProviderRunner is a custom JUnit runner that extends BlockJUnit4ClassRunner to enable TestNG-style data provider functionality.

22

23

```java { .api }

24

public class DataProviderRunner extends BlockJUnit4ClassRunner {

25

26

/**

27

* Creates a DataProviderRunner to run supplied test class.

28

* @param clazz the test Class to run

29

* @throws InitializationError if the test Class is malformed

30

*/

31

public DataProviderRunner(Class<?> clazz) throws InitializationError

32

33

/**

34

* Apply custom filter that supports data provider row filtering.

35

* @param filter the Filter to be wrapped or applied

36

* @throws NoTestsRemainException if no tests remain after filtering

37

*/

38

public void filter(Filter filter) throws NoTestsRemainException

39

40

/**

41

* Initialize helper classes for data conversion, test generation, and validation.

42

* Override this method to customize internal behavior.

43

*/

44

protected void initializeHelpers()

45

46

/**

47

* Generate exploded list of test methods including parameterized variants.

48

* @return list of all test methods including data provider expansions

49

*/

50

protected List<FrameworkMethod> computeTestMethods()

51

52

// Protected fields for customization

53

protected DataConverter dataConverter;

54

protected TestGenerator testGenerator;

55

protected TestValidator testValidator;

56

}

57

```

58

59

### Usage

60

61

```java

62

@RunWith(DataProviderRunner.class)

63

public class MyTest {

64

// Test methods with data providers

65

}

66

```

67

68

The runner automatically:

69

- Discovers methods annotated with @DataProvider

70

- Expands test methods annotated with @UseDataProvider into multiple test cases

71

- Validates data provider methods and test method signatures

72

- Generates unique test names for each data provider row

73

74

## @DataProvider Annotation

75

76

The @DataProvider annotation marks methods as data providers or provides data directly inline.

77

78

```java { .api }

79

@Documented

80

@Retention(RetentionPolicy.RUNTIME)

81

@Target(ElementType.METHOD)

82

public @interface DataProvider {

83

84

/**

85

* Define list of parameters as regex-separated Strings for the test method.

86

* @return list of regex-separated String parameters

87

*/

88

String[] value() default {};

89

90

/**

91

* Delimiting regex for splitting String data. Defaults to comma.

92

* @return the regex to split String data

93

*/

94

String splitBy() default COMMA;

95

96

/**

97

* Convert "null" strings to null values. Default is true.

98

* @return true if "null" strings should be converted to null

99

*/

100

boolean convertNulls() default true;

101

102

/**

103

* Trim leading/trailing whitespace from split strings. Default is true.

104

* @return true if string data should be trimmed

105

*/

106

boolean trimValues() default true;

107

108

/**

109

* Format pattern for generating test method descriptions.

110

* Available placeholders: %c, %cc, %m, %cm, %i, %p[x]

111

* @return the format pattern for test method names

112

*/

113

String format() default DEFAULT_FORMAT;

114

115

/**

116

* Ignore case when converting enum values. Default is false.

117

* @return true if enum conversion should be case-insensitive

118

*/

119

boolean ignoreEnumCase() default false;

120

121

// Constants

122

String COMMA = ",";

123

String NULL = "null";

124

String DEFAULT_FORMAT = "%m[%i: %p[0..-1]]";

125

}

126

```

127

128

### Data Provider Method Usage

129

130

```java

131

// Method-based data provider

132

@DataProvider

133

public static Object[][] testData() {

134

return new Object[][] {

135

{ "input1", "expected1" },

136

{ "input2", "expected2" }

137

};

138

}

139

140

// Inline string data provider

141

@Test

142

@DataProvider({"test,4", "hello,5", "world,5"})

143

public void testStringLength(String input, int expectedLength) {

144

assertEquals(expectedLength, input.length());

145

}

146

147

// Custom configuration

148

@DataProvider(

149

splitBy = "\\|", // Use pipe as delimiter

150

convertNulls = false, // Keep "null" as string

151

trimValues = false, // Preserve whitespace

152

format = "%m[%i]" // Simple index format

153

)

154

public static String[] customData() {

155

return new String[] { "a|1", "b|2", "null|0" };

156

}

157

```

158

159

### Supported Return Types for Data Provider Methods

160

161

- `Object[][]` - Standard two-dimensional array

162

- `Iterable<Iterable<?>>` - Nested iterables

163

- `Iterable<?>` - Single iterable (each element becomes one test case)

164

- `String[]` - Array of regex-separated parameter strings

165

166

## @UseDataProvider Annotation

167

168

The @UseDataProvider annotation connects test methods to their data providers.

169

170

```java { .api }

171

@Documented

172

@Retention(RetentionPolicy.RUNTIME)

173

@Target(ElementType.METHOD)

174

public @interface UseDataProvider {

175

176

enum ResolveStrategy {

177

/** Use first matching resolver */

178

UNTIL_FIRST_MATCH,

179

/** Aggregate all matching resolvers */

180

AGGREGATE_ALL_MATCHES

181

}

182

183

/**

184

* Name or pattern to derive the data provider method.

185

* Default uses convention-based naming.

186

* @return value from which data provider method can be derived

187

*/

188

String value() default DEFAULT_VALUE;

189

190

/**

191

* Classes where data provider methods should be searched.

192

* Default searches in the test class itself.

193

* @return array of Classes to search for data provider methods

194

*/

195

Class<?>[] location() default {};

196

197

/**

198

* Resolvers used to find data provider methods.

199

* @return resolver classes to use for data provider method resolution

200

*/

201

Class<? extends DataProviderMethodResolver>[] resolver()

202

default { DefaultDataProviderMethodResolver.class };

203

204

/**

205

* Strategy for resolving multiple data provider methods.

206

* @return strategy for handling multiple resolver matches

207

*/

208

ResolveStrategy resolveStrategy() default ResolveStrategy.UNTIL_FIRST_MATCH;

209

210

// Constants

211

String DEFAULT_VALUE = "<use_convention>";

212

}

213

```

214

215

### Usage Examples

216

217

```java

218

// Convention-based (data provider method has same name as test method)

219

@Test

220

@UseDataProvider

221

public void testStringLength(String input, int expected) { /* ... */ }

222

223

@DataProvider

224

public static Object[][] testStringLength() { /* ... */ }

225

226

// Explicit data provider name

227

@Test

228

@UseDataProvider("stringData")

229

public void testStringLength(String input, int expected) { /* ... */ }

230

231

// Data provider in external class

232

@Test

233

@UseDataProvider(value = "commonTestData", location = TestDataClass.class)

234

public void testWithExternalData(String input, int expected) { /* ... */ }

235

236

// Multiple resolvers with aggregation

237

@Test

238

@UseDataProvider(

239

resolver = { CustomResolver1.class, CustomResolver2.class },

240

resolveStrategy = ResolveStrategy.AGGREGATE_ALL_MATCHES

241

)

242

public void testWithMultipleResolvers(Object data) { /* ... */ }

243

```

244

245

### Convention-Based Naming

246

247

When using `@UseDataProvider` without specifying a value, the DefaultDataProviderMethodResolver tries these naming patterns in order:

248

249

1. Exact method name match: `testMethod()``testMethod()`

250

2. Replace "test" prefix: `testMethod()``dataProviderMethod()` or `dataMethod()`

251

3. Add prefix: `testMethod()``dataProviderTestMethod()` or `dataTestMethod()`

252

253

## DataProviderFrameworkMethod

254

255

Internal class that represents parameterized test methods generated by the data provider.

256

257

```java { .api }

258

public class DataProviderFrameworkMethod extends FrameworkMethod {

259

260

/**

261

* Create a parameterized test method.

262

* @param method the original test method

263

* @param idx index of the data provider row

264

* @param parameters parameters for this test invocation

265

* @param nameFormat format pattern for generating test name

266

*/

267

public DataProviderFrameworkMethod(Method method, int idx,

268

Object[] parameters, String nameFormat)

269

270

/**

271

* Get formatted name for this parameterized test.

272

* @return formatted test method name

273

*/

274

public String getName()

275

276

/**

277

* Invoke test method with data provider parameters.

278

* @param target test instance

279

* @param params ignored - uses data provider parameters instead

280

* @return test method result

281

*/

282

public Object invokeExplosively(Object target, Object... params) throws Throwable

283

284

// Package-private fields for testing

285

final int idx;

286

final Object[] parameters;

287

final String nameFormat;

288

}

289

```

290

291

This class is primarily used internally by the DataProviderRunner but may be relevant for custom test runners or advanced integrations.

292

293

## DataProviderFilter

294

295

A specialized filter for parameterized tests that allows filtering specific data provider rows.

296

297

```java { .api }

298

public class DataProviderFilter extends Filter {

299

300

/**

301

* Create a filter that can handle data provider test filtering.

302

* @param filter the original filter to wrap

303

*/

304

public DataProviderFilter(Filter filter)

305

306

/**

307

* Determine if a test should run based on data provider row matching.

308

* @param description test method description

309

* @return true if the test should run

310

*/

311

public boolean shouldRun(Description description)

312

313

/**

314

* Get description of the wrapped filter.

315

* @return filter description

316

*/

317

public String describe()

318

319

// Pattern for parsing data provider test descriptions

320

static final Pattern DESCRIPTION_PATTERN;

321

static final Pattern GENEROUS_DESCRIPTION_PATTERN;

322

}

323

```

324

325

### Usage

326

327

The DataProviderFilter is automatically used by DataProviderRunner to enable filtering of individual parameterized test cases:

328

329

```java

330

// When running specific parameterized test cases, the filter can target:

331

// - Specific test method: MyTest.testMethod

332

// - Specific data provider row: MyTest.testMethod[2: param1, param2]

333

// - All rows of a method: MyTest.testMethod[*]

334

335

@RunWith(DataProviderRunner.class)

336

public class FilterableTest {

337

338

@Test

339

@UseDataProvider("testData")

340

public void testMethod(String input, int expected) {

341

assertEquals(expected, input.length());

342

}

343

344

@DataProvider

345

public static Object[][] testData() {

346

return new Object[][] {

347

{"hello", 5}, // Row 0

348

{"world", 5}, // Row 1

349

{"test", 4} // Row 2

350

};

351

}

352

}

353

```