CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-org-apache-avro--avro-toplevel

Apache Avro meta-framework that coordinates data serialization implementations across multiple programming languages

Pending
Overview
Eval results
Files

build-tools.mddocs/

Build Tools

Comprehensive build system and development infrastructure for cross-platform development, testing, and deployment of the Apache Avro ecosystem across multiple programming languages.

Capabilities

Cross-Platform Build System

Central build orchestration system that coordinates compilation, testing, and packaging across all supported programming languages.

# Main build script interface
./build.sh [command]

# Available commands:
# lint             - Run code quality checks across all languages
# test             - Run comprehensive test suite including interop tests
# dist             - Build distribution packages for all languages
# sign             - Sign release artifacts with GPG
# clean            - Clean build artifacts
# veryclean        - Clean all build artifacts including dependencies
# docker           - Build and run development environment in Docker
# rat              - Run Apache RAT license checking
# githooks         - Install git hooks for development
# docker-test      - Run tests in Docker container

# Note: This is a meta-project coordinator script that operates on
# all language implementations simultaneously rather than individual languages

Usage Examples:

# Run code quality checks across all languages
./build.sh lint

# Run comprehensive test suite (all languages + interop)
./build.sh test

# Clean all build artifacts
./build.sh clean

# Very thorough clean including dependencies
./build.sh veryclean

# Build distribution packages for all languages
./build.sh dist

# Sign release artifacts
./build.sh sign

# Setup development environment in Docker
./build.sh docker

# Run Apache RAT license checking
./build.sh rat

# Install git hooks for development
./build.sh githooks

# Run tests in Docker container
./build.sh docker-test

Maven Build Integration

Java-based build system integration using Apache Maven for dependency management, compilation, and lifecycle management.

<!-- Maven build configuration -->
<build>
  <plugins>
    <!-- Avro schema compilation -->
    <plugin>
      <groupId>org.apache.avro</groupId>
      <artifactId>avro-maven-plugin</artifactId>
      <version>1.12.0</version>
      <executions>
        <execution>
          <phase>generate-sources</phase>
          <goals>
            <goal>schema</goal>
          </goals>
          <configuration>
            <sourceDirectory>${project.basedir}/src/main/avro/</sourceDirectory>
            <outputDirectory>${project.basedir}/src/main/java/</outputDirectory>
          </configuration>
        </execution>
      </executions>
    </plugin>
    
    <!-- Compiler configuration -->
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <version>3.13.0</version>
      <configuration>
        <source>11</source>
        <target>11</target>
        <release>11</release>
      </configuration>
    </plugin>
  </plugins>
</build>

# Maven build profiles
<profiles>
  <profile>
    <id>full-build</id>
    <modules>
      <module>avro</module>
      <module>compiler</module>  
      <module>ipc</module>
      <module>tools</module>
      <module>mapred</module>
    </modules>
  </profile>
</profiles>

Usage Examples:

# Maven build commands
mvn clean compile                    # Compile Java sources
mvn clean test                      # Run unit tests
mvn clean package                   # Create JAR packages
mvn clean install                   # Install to local repository
mvn clean deploy                    # Deploy to remote repository

# Generate Avro classes from schemas
mvn avro:schema

# Build with specific profile
mvn clean install -P full-build

# Skip tests during build
mvn clean install -DskipTests

# Generate site documentation
mvn site

Docker Development Environment

Containerized development and testing environment providing consistent build environments across different platforms.

# Docker Compose configuration
version: '3.8'

services:
  avro-dev:
    build:
      context: .
      dockerfile: Dockerfile
    volumes:
      - .:/workspace
      - maven-cache:/root/.m2
    environment:
      - JAVA_HOME=/usr/local/openjdk-11
      - MAVEN_OPTS=-Xmx2g
    ports:
      - "8080:8080"   # Documentation server
      - "9090:9090"   # Test server
    
  avro-test:
    extends: avro-dev
    command: ["./build.sh", "test", "all"]
    
volumes:
  maven-cache:

Docker Commands:

# Docker build and development commands
docker-compose build              # Build development image
docker-compose up -d             # Start development environment
docker-compose exec avro-dev bash   # Interactive shell

# Run builds in container
docker-compose run avro-dev ./build.sh build all
docker-compose run avro-test    # Run test suite

# Clean up containers
docker-compose down             # Stop services
docker-compose down -v          # Stop and remove volumes

Usage Examples:

# Start development environment
docker-compose up -d avro-dev

# Build inside container
docker-compose exec avro-dev ./build.sh build java

# Run tests in container
docker-compose run avro-test

# Access documentation server
open http://localhost:8080/docs

Testing Infrastructure

Comprehensive testing framework supporting unit tests, integration tests, and cross-language compatibility validation.

# Test configuration structure
testing:
  unit_tests:
    java:
      framework: "junit5"
      runner: "maven-surefire-plugin"
      coverage: "jacoco"
    python:
      framework: "pytest"
      runner: "python -m pytest"
      coverage: "coverage.py"
    javascript:
      framework: "mocha"
      runner: "npm test"
      coverage: "nyc"
      
  integration_tests:
    location: "integration-test/"
    scenarios:
      - "schema_evolution"
      - "rpc_communication"
      - "file_format_compatibility"
      - "performance_benchmarks"
      
  interop_tests:
    data_location: "share/test/data/"
    schemas: "share/test/schemas/"
    languages: ["java", "python", "javascript", "c++", "csharp"]
    test_cases:
      - "round_trip_serialization"
      - "schema_resolution"
      - "rpc_protocol_compliance"

Usage Examples:

# Run all unit tests
./build.sh test all

# Run tests for specific language
./build.sh test java
./build.sh test python --coverage

# Run integration tests
./build.sh test integration

# Run interoperability tests
./build.sh interop-test

# Run performance benchmarks
./build.sh test performance --language java

# Generate test reports
./build.sh test all --report-html

Code Quality and Validation Tools

Automated code quality checking, formatting, and validation tools to maintain consistency across the multi-language codebase.

# Code quality configuration
quality_tools:
  java:
    checkstyle:
      config: "checkstyle.xml"
      rules: "google_checks"
    spotbugs:
      effort: "max"
      threshold: "medium"
    pmd:
      ruleset: "java-basic,java-design"
      
  formatting:
    java:
      tool: "spotless"
      config: "eclipse-java-formatter.xml"
    python:
      tool: "black"
      line_length: 88
    javascript:
      tool: "prettier"
      config: ".prettierrc"
      
  validation:
    schemas:
      tool: "avro-tools"
      command: "validate"
    protocols:
      tool: "avro-tools" 
      command: "idl2schemata"

Usage Examples:

# Code formatting
./build.sh format java          # Format Java code
./build.sh format python        # Format Python code  
./build.sh format all          # Format all languages

# Code quality checks
./build.sh check java          # Run checkstyle, spotbugs, PMD
./build.sh check all           # Check all languages

# Schema validation
avro-tools validate schema.avsc
avro-tools validate protocol.avpr

# Fix formatting issues
./build.sh format-fix all

# Generate quality reports
./build.sh quality-report java

Documentation Generation

Automated documentation generation system creating API documentation, guides, and examples for all language implementations.

# Documentation generation configuration
documentation:
  java:
    tool: "javadoc"
    output: "build/docs/java/"
    options: ["-quiet", "-Xdoclint:none"]
    
  python:
    tool: "sphinx"
    output: "build/docs/python/"
    config: "doc/python/conf.py"
    
  site:
    generator: "hugo"
    source: "doc/"
    output: "build/site/"
    theme: "docsy"
    
  api_docs:
    java: "https://avro.apache.org/docs/1.12.0/api/java/"
    python: "https://avro.apache.org/docs/1.12.0/api/python/"
    csharp: "https://avro.apache.org/docs/1.12.0/api/csharp/"

Usage Examples:

# Generate all documentation
./build.sh docs all

# Generate language-specific docs
./build.sh docs java
./build.sh docs python

# Generate website
./build.sh docs site

# Serve documentation locally
./build.sh serve-docs          # Start local documentation server

# Deploy documentation
./build.sh deploy-docs         # Deploy to documentation site

Release and Distribution Tools

Tools and processes for building, packaging, and distributing Avro releases across multiple package managers and platforms.

# Release configuration
release:
  version_file: "share/VERSION.txt"
  
  artifacts:
    java:
      repository: "maven_central"
      artifacts: ["avro", "avro-compiler", "avro-ipc", "avro-tools"]
      
    python:
      repository: "pypi"
      package: "avro-python3"
      
    javascript:
      repository: "npm"
      package: "avro-js"
      
    csharp:
      repository: "nuget"
      package: "Apache.Avro"
      
  distribution:
    source: "apache-avro-${version}-src.tar.gz"
    binary: "apache-avro-${version}-bin.tar.gz"
    checksums: ["sha256", "sha512"]
    signatures: "gpg"

Usage Examples:

# Prepare release
./build.sh prepare-release 1.12.1

# Build distribution packages
./build.sh dist all

# Sign release artifacts
./build.sh sign-release --key-id ABCDEF

# Validate release
./build.sh validate-release 1.12.1

# Deploy to staging
./build.sh deploy-staging

# Deploy to production
./build.sh deploy-release 1.12.1

Install with Tessl CLI

npx tessl i tessl/maven-org-apache-avro--avro-toplevel

docs

build-tools.md

framework-coordination.md

index.md

language-implementations.md

schema-registry.md

tile.json