0
# Build Tools
1

2
Comprehensive build system and development infrastructure for cross-platform development, testing, and deployment of the Apache Avro ecosystem across multiple programming languages.
3

4
## Capabilities
5

6
### Cross-Platform Build System
7

8
Central build orchestration system that coordinates compilation, testing, and packaging across all supported programming languages.
9

10
```bash { .api }
11
# Main build script interface
12
./build.sh [command]
13

14
# Available commands:
15
# lint             - Run code quality checks across all languages
16
# test             - Run comprehensive test suite including interop tests
17
# dist             - Build distribution packages for all languages
18
# sign             - Sign release artifacts with GPG
19
# clean            - Clean build artifacts
20
# veryclean        - Clean all build artifacts including dependencies
21
# docker           - Build and run development environment in Docker
22
# rat              - Run Apache RAT license checking
23
# githooks         - Install git hooks for development
24
# docker-test      - Run tests in Docker container
25

26
# Note: This is a meta-project coordinator script that operates on
27
# all language implementations simultaneously rather than individual languages
28
```
29

30
**Usage Examples:**
31

32
```bash
33
# Run code quality checks across all languages
34
./build.sh lint
35

36
# Run comprehensive test suite (all languages + interop)
37
./build.sh test
38

39
# Clean all build artifacts
40
./build.sh clean
41

42
# Very thorough clean including dependencies
43
./build.sh veryclean
44

45
# Build distribution packages for all languages
46
./build.sh dist
47

48
# Sign release artifacts
49
./build.sh sign
50

51
# Setup development environment in Docker
52
./build.sh docker
53

54
# Run Apache RAT license checking
55
./build.sh rat
56

57
# Install git hooks for development
58
./build.sh githooks
59

60
# Run tests in Docker container
61
./build.sh docker-test
62
```
63

64
### Maven Build Integration
65

66
Java-based build system integration using Apache Maven for dependency management, compilation, and lifecycle management.
67

68
```xml { .api }
69
<!-- Maven build configuration -->
70
<build>
71
  <plugins>
72
    <!-- Avro schema compilation -->
73
    <plugin>
74
      <groupId>org.apache.avro</groupId>
75
      <artifactId>avro-maven-plugin</artifactId>
76
      <version>1.12.0</version>
77
      <executions>
78
        <execution>
79
          <phase>generate-sources</phase>
80
          <goals>
81
            <goal>schema</goal>
82
          </goals>
83
          <configuration>
84
            <sourceDirectory>${project.basedir}/src/main/avro/</sourceDirectory>
85
            <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
86
          </configuration>
87
        </execution>
88
      </executions>
89
    </plugin>
90
    
91
    <!-- Compiler configuration -->
92
    <plugin>
93
      <groupId>org.apache.maven.plugins</groupId>
94
      <artifactId>maven-compiler-plugin</artifactId>
95
      <version>3.13.0</version>
96
      <configuration>
97
        <source>11</source>
98
        <target>11</target>
99
        <release>11</release>
100
      </configuration>
101
    </plugin>
102
  </plugins>
103
</build>
104

105
# Maven build profiles
106
<profiles>
107
  <profile>
108
    <id>full-build</id>
109
    <modules>
110
      <module>avro</module>
111
      <module>compiler</module>  
112
      <module>ipc</module>
113
      <module>tools</module>
114
      <module>mapred</module>
115
    </modules>
116
  </profile>
117
</profiles>
118
```
119

120
**Usage Examples:**
121

122
```bash
123
# Maven build commands
124
mvn clean compile                    # Compile Java sources
125
mvn clean test                      # Run unit tests
126
mvn clean package                   # Create JAR packages
127
mvn clean install                   # Install to local repository
128
mvn clean deploy                    # Deploy to remote repository
129

130
# Generate Avro classes from schemas
131
mvn avro:schema
132

133
# Build with specific profile
134
mvn clean install -P full-build
135

136
# Skip tests during build
137
mvn clean install -DskipTests
138

139
# Generate site documentation
140
mvn site
141
```
142

143
### Docker Development Environment
144

145
Containerized development and testing environment providing consistent build environments across different platforms.
146

147
```yaml { .api }
148
# Docker Compose configuration
149
version: '3.8'
150

151
services:
152
  avro-dev:
153
    build:
154
      context: .
155
      dockerfile: Dockerfile
156
    volumes:
157
      - .:/workspace
158
      - maven-cache:/root/.m2
159
    environment:
160
      - JAVA_HOME=/usr/local/openjdk-11
161
      - MAVEN_OPTS=-Xmx2g
162
    ports:
163
      - "8080:8080"   # Documentation server
164
      - "9090:9090"   # Test server
165
    
166
  avro-test:
167
    extends: avro-dev
168
    command: ["./build.sh", "test", "all"]
169
    
170
volumes:
171
  maven-cache:
172
```
173

174
**Docker Commands:**
175

176
```bash { .api }
177
# Docker build and development commands
178
docker-compose build              # Build development image
179
docker-compose up -d             # Start development environment
180
docker-compose exec avro-dev bash   # Interactive shell
181

182
# Run builds in container
183
docker-compose run avro-dev ./build.sh build all
184
docker-compose run avro-test    # Run test suite
185

186
# Clean up containers
187
docker-compose down             # Stop services
188
docker-compose down -v          # Stop and remove volumes
189
```
190

191
**Usage Examples:**
192

193
```bash
194
# Start development environment
195
docker-compose up -d avro-dev
196

197
# Build inside container
198
docker-compose exec avro-dev ./build.sh build java
199

200
# Run tests in container
201
docker-compose run avro-test
202

203
# Access documentation server
204
open http://localhost:8080/docs
205
```
206

207
### Testing Infrastructure
208

209
Comprehensive testing framework supporting unit tests, integration tests, and cross-language compatibility validation.
210

211
```yaml { .api }
212
# Test configuration structure
213
testing:
214
  unit_tests:
215
    java:
216
      framework: "junit5"
217
      runner: "maven-surefire-plugin"
218
      coverage: "jacoco"
219
    python:
220
      framework: "pytest"
221
      runner: "python -m pytest"
222
      coverage: "coverage.py"
223
    javascript:
224
      framework: "mocha"
225
      runner: "npm test"
226
      coverage: "nyc"
227
      
228
  integration_tests:
229
    location: "integration-test/"
230
    scenarios:
231
      - "schema_evolution"
232
      - "rpc_communication"
233
      - "file_format_compatibility"
234
      - "performance_benchmarks"
235
      
236
  interop_tests:
237
    data_location: "share/test/data/"
238
    schemas: "share/test/schemas/"
239
    languages: ["java", "python", "javascript", "c++", "csharp"]
240
    test_cases:
241
      - "round_trip_serialization"
242
      - "schema_resolution"
243
      - "rpc_protocol_compliance"
244
```
245

246
**Usage Examples:**
247

248
```bash
249
# Run all unit tests
250
./build.sh test all
251

252
# Run tests for specific language
253
./build.sh test java
254
./build.sh test python --coverage
255

256
# Run integration tests
257
./build.sh test integration
258

259
# Run interoperability tests
260
./build.sh interop-test
261

262
# Run performance benchmarks
263
./build.sh test performance --language java
264

265
# Generate test reports
266
./build.sh test all --report-html
267
```
268

269
### Code Quality and Validation Tools
270

271
Automated code quality checking, formatting, and validation tools to maintain consistency across the multi-language codebase.
272

273
```yaml { .api }
274
# Code quality configuration
275
quality_tools:
276
  java:
277
    checkstyle:
278
      config: "checkstyle.xml"
279
      rules: "google_checks"
280
    spotbugs:
281
      effort: "max"
282
      threshold: "medium"
283
    pmd:
284
      ruleset: "java-basic,java-design"
285
      
286
  formatting:
287
    java:
288
      tool: "spotless"
289
      config: "eclipse-java-formatter.xml"
290
    python:
291
      tool: "black"
292
      line_length: 88
293
    javascript:
294
      tool: "prettier"
295
      config: ".prettierrc"
296
      
297
  validation:
298
    schemas:
299
      tool: "avro-tools"
300
      command: "validate"
301
    protocols:
302
      tool: "avro-tools" 
303
      command: "idl2schemata"
304
```
305

306
**Usage Examples:**
307

308
```bash
309
# Code formatting
310
./build.sh format java          # Format Java code
311
./build.sh format python        # Format Python code  
312
./build.sh format all          # Format all languages
313

314
# Code quality checks
315
./build.sh check java          # Run checkstyle, spotbugs, PMD
316
./build.sh check all           # Check all languages
317

318
# Schema validation
319
avro-tools validate schema.avsc
320
avro-tools validate protocol.avpr
321

322
# Fix formatting issues
323
./build.sh format-fix all
324

325
# Generate quality reports
326
./build.sh quality-report java
327
```
328

329
### Documentation Generation
330

331
Automated documentation generation system creating API documentation, guides, and examples for all language implementations.
332

333
```yaml { .api }
334
# Documentation generation configuration
335
documentation:
336
  java:
337
    tool: "javadoc"
338
    output: "build/docs/java/"
339
    options: ["-quiet", "-Xdoclint:none"]
340
    
341
  python:
342
    tool: "sphinx"
343
    output: "build/docs/python/"
344
    config: "doc/python/conf.py"
345
    
346
  site:
347
    generator: "hugo"
348
    source: "doc/"
349
    output: "build/site/"
350
    theme: "docsy"
351
    
352
  api_docs:
353
    java: "https://avro.apache.org/docs/1.12.0/api/java/"
354
    python: "https://avro.apache.org/docs/1.12.0/api/python/"
355
    csharp: "https://avro.apache.org/docs/1.12.0/api/csharp/"
356
```
357

358
**Usage Examples:**
359

360
```bash
361
# Generate all documentation
362
./build.sh docs all
363

364
# Generate language-specific docs
365
./build.sh docs java
366
./build.sh docs python
367

368
# Generate website
369
./build.sh docs site
370

371
# Serve documentation locally
372
./build.sh serve-docs          # Start local documentation server
373

374
# Deploy documentation
375
./build.sh deploy-docs         # Deploy to documentation site
376
```
377

378
### Release and Distribution Tools
379

380
Tools and processes for building, packaging, and distributing Avro releases across multiple package managers and platforms.
381

382
```yaml { .api }
383
# Release configuration
384
release:
385
  version_file: "share/VERSION.txt"
386
  
387
  artifacts:
388
    java:
389
      repository: "maven_central"
390
      artifacts: ["avro", "avro-compiler", "avro-ipc", "avro-tools"]
391
      
392
    python:
393
      repository: "pypi"
394
      package: "avro-python3"
395
      
396
    javascript:
397
      repository: "npm"
398
      package: "avro-js"
399
      
400
    csharp:
401
      repository: "nuget"
402
      package: "Apache.Avro"
403
      
404
  distribution:
405
    source: "apache-avro-${version}-src.tar.gz"
406
    binary: "apache-avro-${version}-bin.tar.gz"
407
    checksums: ["sha256", "sha512"]
408
    signatures: "gpg"
409
```
410

411
**Usage Examples:**
412

413
```bash
414
# Prepare release
415
./build.sh prepare-release 1.12.1
416

417
# Build distribution packages
418
./build.sh dist all
419

420
# Sign release artifacts
421
./build.sh sign-release --key-id ABCDEF
422

423
# Validate release
424
./build.sh validate-release 1.12.1
425

426
# Deploy to staging
427
./build.sh deploy-staging
428

429
# Deploy to production
430
./build.sh deploy-release 1.12.1
431
```