txtar - Text Archive Tool

The txtar command provides a command-line interface for working with text-based file archives in the txtar format. It enables you to create, extract, and list files stored in text archives, making it useful for bundling files, test data, and documentation examples.

Package Information

• Package Name: golang.org/x/exp/cmd/txtar
• Package Type: Go command-line tool
• Language: Go
• Installation: Part of golang.org/x/exp; use go get golang.org/x/exp/cmd/txtar or go install golang.org/x/exp/cmd/txtar@latest
• Repository: https://golang.org/x/exp
• Format Support: Text-based file archives via golang.org/x/tools/txtar package

Overview

The txtar command manipulates text-based file archives in a human-readable format. These archives combine file metadata and content in a single text file, making them ideal for:

• Including test data and examples in text files
• Creating reproducible test cases
• Embedding file hierarchies in documentation
• Version-controlling complex file structures as text
• Sharing code examples with multiple files in a single readable format

The txtar format stores files as plain text with clear delimiters, allowing archives to be edited in any text editor and easily understood by humans and programs alike.

Core Usage Concepts

The txtar command operates in three modes:

1. Create/Write Mode (default): Reads files and directories, writes an archive to stdout
2. Extract Mode (-x/--extract flag): Reads an archive from stdin, extracts files to disk
3. List Mode (--list flag): Reads an archive from stdin, lists all contained files

Archives include a comment section at the beginning (separated from files by a blank line), which you can use to document the archive contents or provide metadata.

Basic Usage

Create an archive from files

txtar *.go <README >testdata/example.txt

This reads a comment from stdin (terminated by EOF or a blank line), then packages all .go files and the README file into an archive, writing it to testdata/example.txt.

Extract an archive

txtar --extract <playground_example.txt >main.go

This reads the archive from playground_example.txt, extracts all files to their archived locations relative to the current directory, and writes the archive's comment to stdout.

List archive contents

txtar --list <archive.txt

This displays all files contained in the archive without extracting them or expanding shell variables.

Capabilities

Archive Creation

Creates text-based archives from files and directories on disk.

Command:

txtar [files/directories...] <stdin >archive.txt

Behavior:

• Default mode without flags
• Reads archive comment from stdin until EOF
• Recursively includes contents of specified directories
• Includes hidden files (dot files)
• Preserves shell variables in paths without expansion
• Empty argument list defaults to current directory (.)
• Writes the complete archive to stdout

Environment Variables:

• Shell variables referenced in paths are preserved as-is during archive creation
• Variables are stored unexpanded in the archive

Archive Extraction

Extracts files from a text-based archive to disk locations.

Command:

txtar --extract [--unsafe] <archive.txt

Or using short form:

txtar -x [--unsafe] <archive.txt

Behavior:

• Reads the archive from stdin
• Creates files and directories as specified in the archive
• Writes the archive's comment to stdout
• Extracts only within the current directory and its subdirectories by default
• Expands shell variables in file paths using os.Expand if the corresponding environment variable is set
• Raises an error for paths attempting to escape the current directory unless --unsafe is specified

Flags:

• --unsafe: Allows extraction to locations outside the current directory and its subdirectories (use with caution)

Archive Listing

Lists all files contained in an archive without extracting or modifying them.

Command:

txtar --list <archive.txt

Behavior:

• Reads the archive from stdin
• Outputs one file path per line to stdout
• Does not expand shell variables in paths
• Does not extract files to disk

Advanced Usage

Working with Shell Variables

The txtar tool supports shell variable expansion during extraction, allowing you to create dynamic archives:

Archiving with variables:

# Variables in paths are preserved as-is
txtar $HOME/project/file.go >archive.txt
# Archive will contain: $HOME/project/file.go

Extracting with variables:

# Set environment variables before extraction
export OUTPUT_DIR=/tmp/output
txtar --extract <archive.txt
# If archive contains $OUTPUT_DIR/file.txt, it will be extracted to /tmp/output/file.txt

Security Considerations

By default, txtar prevents extraction of files outside the current directory tree:

# This fails by default (path tries to escape current dir)
# Error: file path attempts to escape current directory

# Use --unsafe to allow extraction anywhere (use carefully)
txtar --unsafe --extract <archive.txt

Recursive Directory Inclusion

The command automatically includes directory contents recursively:

# Include all files in src/ and docs/ directories plus specific files
txtar src/ docs/ main.go config.yaml <description.txt >archive.txt

This archives:

• All files in the src/ directory and subdirectories
• All files in the docs/ directory and subdirectories
• The main.go file
• The config.yaml file

Plus the comment from the provided description.

Piping and Stdin/Stdout

Create complex workflows using pipes:

# Create archive with inline comment
echo "Test data for example" | txtar *.go >tests.txt

# Extract and process files
txtar --extract <tests.txt | head -20

# Chain operations
cat original.txt | txtar --extract | tar czf output.tar.gz

# List files and filter
txtar --list <archive.txt | grep -E "\.go$"

txtar Archive Format

The txtar format is a human-readable text-based archive format consisting of:

1. Comment Section: Initial text content, typically used for documentation or metadata
2. Separator: A blank line (two consecutive newlines) separating the comment from files
3. File Entries: Each file is represented with:
   • A -- file_path line indicating the filename
   • The file's complete contents on following lines
   • Implicit termination at the next -- line or end of file

Example archive structure:

This is the archive comment.
It can span multiple lines.
It describes what's in the archive.

-- main.go
package main

func main() {
    fmt.Println("Hello")
}
-- config.yaml
server: localhost
port: 8080
-- nested/readme.txt
This file is in a nested directory.

Exit Behavior

• Success: Command exits with code 0, archive is written/extracted successfully
• Failure: Command exits with non-zero code on errors such as:
  • Invalid command-line flags
  • File access errors during read/write
  • Malformed archive during extraction
  • Path escape attempts (without --unsafe flag)

Common Patterns

Creating a reproducible test fixture

# Create an archive with test input
cat <<'EOF' | txtar *.go >testdata/fixture.txt
Test suite data for TestProcessing
EOF

Embedding examples in documentation

# Package example code with its expected output
echo "Example: Processing configuration files" | \
  txtar example.go example.yaml >docs/example.txt

Backing up a project structure

# Archive an entire project for later restoration
echo "Project backup - $(date)" | \
  txtar src/ tests/ config/ build.sh >backups/project.txt

Automated testing with multiple files

# Create test input from multiple source files
echo "Integration test input" | \
  txtar input1.json input2.json input3.json >test_data.txt

Integration with Other Tools

With go test

The txtar format is widely used in Go's standard testing patterns, particularly for:

• Integration tests
• Benchmark input files
• Example documentation with multiple files
• Testdata directories

With version control

Since txtar archives are plain text, they integrate naturally with Git and other version control systems:

• View diffs of archive changes
• Merge conflicts in archives
• Track historical versions
• Include in code reviews

Related Tools and Packages

• golang.org/x/tools/txtar: The Go package that implements the txtar format and archive manipulation
• txtar command: This tool, providing CLI access to txtar functionality
• go tool: The Go toolchain that provides access to this command when installed

Type Definitions

The txtar command is a standalone executable and does not export type definitions. It operates through command-line flags and standard I/O streams.

Standard I/O:

• stdin: Archive comment (during creation) or archive contents (during extraction/listing)
• stdout: Archive contents (during creation) or extracted comment/file listing
• stderr: Error messages

Command-line Arguments:

• Positional arguments: File and directory paths to archive (create mode)
• Flags: --extract / -x, --list, --unsafe