CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-atomicwrites

Atomic file writes for Python with cross-platform support and data integrity guarantees.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

Atomicwrites

Atomicwrites provides atomic file write operations that ensure data integrity during file operations by using temporary files and atomic rename operations. It offers cross-platform support for Windows and POSIX systems, handling platform-specific atomic file operations through appropriate system calls.

Package Information

  • Package Name: atomicwrites
  • Package Type: pypi
  • Language: Python
  • Installation: pip install atomicwrites
  • Version: 1.4.1

Core Imports

from atomicwrites import atomic_write

For class-based API:

from atomicwrites import AtomicWriter

For low-level functions:

from atomicwrites import replace_atomic, move_atomic

Basic Usage

from atomicwrites import atomic_write

# Simple atomic write with context manager
with atomic_write('example.txt', overwrite=True) as f:
    f.write('Hello, world!')
    # File doesn't exist on disk yet

# Now the file exists atomically

# Prevent overwriting existing files
try:
    with atomic_write('example.txt', overwrite=False) as f:
        f.write('This will fail')
except OSError as e:
    print(f"File exists: {e}")

# Advanced usage with class-based API
from atomicwrites import AtomicWriter

writer = AtomicWriter('config.json', mode='w', overwrite=True)
with writer.open() as f:
    f.write('{"setting": "value"}')

Architecture

Atomicwrites uses a temporary file approach to ensure atomicity:

  1. Temporary File Creation: Creates a temporary file in the same directory as the target
  2. Write Operations: All writes go to the temporary file
  3. Sync Operations: Data is flushed and synced to ensure disk persistence
  4. Atomic Commit: Uses platform-specific atomic operations to move temp file to target location
  5. Cleanup: Automatic cleanup of temporary files on exceptions

Capabilities

High-Level Context Manager API

Simple context manager interface for atomic file writes with automatic error handling and cleanup.

def atomic_write(path, writer_cls=AtomicWriter, **cls_kwargs):
    """
    Simple atomic writes using context manager.

    Parameters:
    - path: str, target path to write to
    - writer_cls: class, writer class to use (default: AtomicWriter)
    - **cls_kwargs: additional keyword arguments passed to writer class

    Returns:
    Context manager that yields file-like object

    Raises:
    - ValueError: for invalid file modes
    - OSError: when file exists and overwrite=False
    - Various OS-specific errors during file operations
    """

Class-Based Writer API

Flexible class-based API providing fine-grained control over atomic write operations and allowing for customization through subclassing.

class AtomicWriter:
    """
    A helper class for performing atomic writes with fine-grained control.

    Parameters:
    - path: str, destination filepath (may or may not exist)
    - mode: str, file mode (default: "wb" on Python 2, "w" on Python 3)
    - overwrite: bool, whether to overwrite existing files (default: False)
    - **open_kwargs: additional arguments passed to open()

    Raises:
    - ValueError: for unsupported modes ('a', 'x') or non-write modes
    """

    def __init__(self, path, mode=DEFAULT_MODE, overwrite=False, **open_kwargs):
        """Initialize AtomicWriter with target path and options."""

    def open(self):
        """
        Open the temporary file and return context manager.

        Returns:
        Context manager that yields file-like object
        """

    def get_fileobject(self, suffix="", prefix=tempfile.gettempprefix(), dir=None, **kwargs):
        """
        Return the temporary file to use.

        Parameters:
        - suffix: str, suffix for temporary filename
        - prefix: str, prefix for temporary filename  
        - dir: str, directory for temporary file (defaults to target file's directory)
        - **kwargs: additional arguments

        Returns:
        File-like object for writing
        """

    def sync(self, f):
        """
        Clear file caches before commit (flush and fsync).

        Parameters:
        - f: file object to sync
        """

    def commit(self, f):
        """
        Move temporary file to target location atomically.

        Parameters:
        - f: file object to commit
        """

    def rollback(self, f):
        """
        Clean up temporary resources (unlink temp file).

        Parameters:
        - f: file object to rollback

        Raises:
        - OSError: if temporary file cannot be removed
        """

Low-Level Atomic Operations

Direct atomic file operations for advanced use cases requiring precise control over file movement behavior.

def replace_atomic(src, dst):
    """
    Move src to dst atomically, overwriting dst if it exists.

    Parameters:
    - src: str, source file path
    - dst: str, destination file path

    Requirements:
    Both paths must reside on the same filesystem for atomicity.

    Platform Implementation:
    - POSIX: Uses rename() with directory fsync
    - Windows: Uses MoveFileEx() with MOVEFILE_REPLACE_EXISTING flag

    Raises:
    - OSError: for file system errors
    - WinError: on Windows for system-specific errors
    """

def move_atomic(src, dst):
    """
    Move src to dst atomically, raises FileExistsError if dst exists.

    Parameters:
    - src: str, source file path  
    - dst: str, destination file path

    Requirements:
    Both paths must reside on the same filesystem for atomicity.

    Platform Implementation:
    - POSIX: Uses link() + unlink() with directory fsync
    - Windows: Uses MoveFileEx() without MOVEFILE_REPLACE_EXISTING

    Raises:
    - FileExistsError: if destination file already exists
    - OSError: for other file system errors
    - WinError: on Windows for system-specific errors

    Note:
    There may be a time window where both filesystem entries exist.
    """

Types

import os
from typing import Union

# Module constants
__version__: str = "1.4.1"
DEFAULT_MODE: str  # "wb" on Python 2, "w" on Python 3

# Type aliases for clarity
FilePath = Union[str, bytes, os.PathLike]
FileMode = str  # Valid modes: 'w', 'wb', 'w+', 'wb+', etc. (no 'a' or 'x')

Error Handling

Common Exceptions

  • ValueError: Raised for invalid file modes (append 'a', exclusive 'x') or non-write modes
  • FileExistsError: When destination exists and overwrite=False
  • OSError: General file system errors (permissions, disk space, etc.)
  • WinError: Windows-specific system errors

Error Recovery

Atomicwrites automatically handles cleanup on exceptions:

  • Temporary files are automatically removed on write failures
  • Original files remain unchanged if atomic operation fails
  • No partial writes or corruption occur due to interruption

Example Error Handling

from atomicwrites import atomic_write
import os

# Handle overwrite protection
try:
    with atomic_write('existing_file.txt', overwrite=False) as f:
        f.write('This will fail if file exists')
except FileExistsError:
    print("File already exists and overwrite=False")

# Handle permission errors
try:
    with atomic_write('/root/protected.txt', overwrite=True) as f:
        f.write('This may fail due to permissions')
except PermissionError:
    print("Insufficient permissions to write file")

# Handle invalid modes
try:
    from atomicwrites import AtomicWriter
    writer = AtomicWriter('test.txt', mode='a')  # append mode
except ValueError as e:
    print(f"Invalid mode: {e}")

Platform Support

POSIX Systems (Linux, macOS, Unix)

  • Uses os.rename() for overwrite operations
  • Uses os.link() + os.unlink() for non-overwrite operations
  • Performs directory fsync() to ensure filename persistence
  • On macOS: Uses fcntl.F_FULLFSYNC for complete data flushing

Windows Systems

  • Uses MoveFileEx() via ctypes with appropriate flags
  • MOVEFILE_WRITE_THROUGH flag ensures data reaches disk
  • MOVEFILE_REPLACE_EXISTING flag for overwrite operations
  • Handles Unicode paths correctly

Cross-Platform Guarantees

  • Atomic file replacement on same filesystem
  • Proper data synchronization before atomic operation
  • Consistent error handling across platforms
  • Unicode filename support

Install with Tessl CLI

npx tessl i tessl/pypi-atomicwrites
Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/atomicwrites@1.4.x
Publish Source
CLI
Badge
tessl/pypi-atomicwrites badge