or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.md
tile.json

tessl/pypi-atomicwrites

Atomic file writes for Python with cross-platform support and data integrity guarantees.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/atomicwrites@1.4.x

To install, run

npx @tessl/cli install tessl/pypi-atomicwrites@1.4.0

index.mddocs/

Atomicwrites

Atomicwrites provides atomic file write operations that ensure data integrity during file operations by using temporary files and atomic rename operations. It offers cross-platform support for Windows and POSIX systems, handling platform-specific atomic file operations through appropriate system calls.

Package Information

  • Package Name: atomicwrites
  • Package Type: pypi
  • Language: Python
  • Installation: pip install atomicwrites
  • Version: 1.4.1

Core Imports

from atomicwrites import atomic_write

For class-based API:

from atomicwrites import AtomicWriter

For low-level functions:

from atomicwrites import replace_atomic, move_atomic

Basic Usage

from atomicwrites import atomic_write

# Simple atomic write with context manager
with atomic_write('example.txt', overwrite=True) as f:
    f.write('Hello, world!')
    # File doesn't exist on disk yet

# Now the file exists atomically

# Prevent overwriting existing files
try:
    with atomic_write('example.txt', overwrite=False) as f:
        f.write('This will fail')
except OSError as e:
    print(f"File exists: {e}")

# Advanced usage with class-based API
from atomicwrites import AtomicWriter

writer = AtomicWriter('config.json', mode='w', overwrite=True)
with writer.open() as f:
    f.write('{"setting": "value"}')

Architecture

Atomicwrites uses a temporary file approach to ensure atomicity:

  1. Temporary File Creation: Creates a temporary file in the same directory as the target
  2. Write Operations: All writes go to the temporary file
  3. Sync Operations: Data is flushed and synced to ensure disk persistence
  4. Atomic Commit: Uses platform-specific atomic operations to move temp file to target location
  5. Cleanup: Automatic cleanup of temporary files on exceptions

Capabilities

High-Level Context Manager API

Simple context manager interface for atomic file writes with automatic error handling and cleanup.

def atomic_write(path, writer_cls=AtomicWriter, **cls_kwargs):
    """
    Simple atomic writes using context manager.

    Parameters:
    - path: str, target path to write to
    - writer_cls: class, writer class to use (default: AtomicWriter)
    - **cls_kwargs: additional keyword arguments passed to writer class

    Returns:
    Context manager that yields file-like object

    Raises:
    - ValueError: for invalid file modes
    - OSError: when file exists and overwrite=False
    - Various OS-specific errors during file operations
    """

Class-Based Writer API

Flexible class-based API providing fine-grained control over atomic write operations and allowing for customization through subclassing.

class AtomicWriter:
    """
    A helper class for performing atomic writes with fine-grained control.

    Parameters:
    - path: str, destination filepath (may or may not exist)
    - mode: str, file mode (default: "wb" on Python 2, "w" on Python 3)
    - overwrite: bool, whether to overwrite existing files (default: False)
    - **open_kwargs: additional arguments passed to open()

    Raises:
    - ValueError: for unsupported modes ('a', 'x') or non-write modes
    """

    def __init__(self, path, mode=DEFAULT_MODE, overwrite=False, **open_kwargs):
        """Initialize AtomicWriter with target path and options."""

    def open(self):
        """
        Open the temporary file and return context manager.

        Returns:
        Context manager that yields file-like object
        """

    def get_fileobject(self, suffix="", prefix=tempfile.gettempprefix(), dir=None, **kwargs):
        """
        Return the temporary file to use.

        Parameters:
        - suffix: str, suffix for temporary filename
        - prefix: str, prefix for temporary filename  
        - dir: str, directory for temporary file (defaults to target file's directory)
        - **kwargs: additional arguments

        Returns:
        File-like object for writing
        """

    def sync(self, f):
        """
        Clear file caches before commit (flush and fsync).

        Parameters:
        - f: file object to sync
        """

    def commit(self, f):
        """
        Move temporary file to target location atomically.

        Parameters:
        - f: file object to commit
        """

    def rollback(self, f):
        """
        Clean up temporary resources (unlink temp file).

        Parameters:
        - f: file object to rollback

        Raises:
        - OSError: if temporary file cannot be removed
        """

Low-Level Atomic Operations

Direct atomic file operations for advanced use cases requiring precise control over file movement behavior.

def replace_atomic(src, dst):
    """
    Move src to dst atomically, overwriting dst if it exists.

    Parameters:
    - src: str, source file path
    - dst: str, destination file path

    Requirements:
    Both paths must reside on the same filesystem for atomicity.

    Platform Implementation:
    - POSIX: Uses rename() with directory fsync
    - Windows: Uses MoveFileEx() with MOVEFILE_REPLACE_EXISTING flag

    Raises:
    - OSError: for file system errors
    - WinError: on Windows for system-specific errors
    """

def move_atomic(src, dst):
    """
    Move src to dst atomically, raises FileExistsError if dst exists.

    Parameters:
    - src: str, source file path  
    - dst: str, destination file path

    Requirements:
    Both paths must reside on the same filesystem for atomicity.

    Platform Implementation:
    - POSIX: Uses link() + unlink() with directory fsync
    - Windows: Uses MoveFileEx() without MOVEFILE_REPLACE_EXISTING

    Raises:
    - FileExistsError: if destination file already exists
    - OSError: for other file system errors
    - WinError: on Windows for system-specific errors

    Note:
    There may be a time window where both filesystem entries exist.
    """

Types

import os
from typing import Union

# Module constants
__version__: str = "1.4.1"
DEFAULT_MODE: str  # "wb" on Python 2, "w" on Python 3

# Type aliases for clarity
FilePath = Union[str, bytes, os.PathLike]
FileMode = str  # Valid modes: 'w', 'wb', 'w+', 'wb+', etc. (no 'a' or 'x')

Error Handling

Common Exceptions

  • ValueError: Raised for invalid file modes (append 'a', exclusive 'x') or non-write modes
  • FileExistsError: When destination exists and overwrite=False
  • OSError: General file system errors (permissions, disk space, etc.)
  • WinError: Windows-specific system errors

Error Recovery

Atomicwrites automatically handles cleanup on exceptions:

  • Temporary files are automatically removed on write failures
  • Original files remain unchanged if atomic operation fails
  • No partial writes or corruption occur due to interruption

Example Error Handling

from atomicwrites import atomic_write
import os

# Handle overwrite protection
try:
    with atomic_write('existing_file.txt', overwrite=False) as f:
        f.write('This will fail if file exists')
except FileExistsError:
    print("File already exists and overwrite=False")

# Handle permission errors
try:
    with atomic_write('/root/protected.txt', overwrite=True) as f:
        f.write('This may fail due to permissions')
except PermissionError:
    print("Insufficient permissions to write file")

# Handle invalid modes
try:
    from atomicwrites import AtomicWriter
    writer = AtomicWriter('test.txt', mode='a')  # append mode
except ValueError as e:
    print(f"Invalid mode: {e}")

Platform Support

POSIX Systems (Linux, macOS, Unix)

  • Uses os.rename() for overwrite operations
  • Uses os.link() + os.unlink() for non-overwrite operations
  • Performs directory fsync() to ensure filename persistence
  • On macOS: Uses fcntl.F_FULLFSYNC for complete data flushing

Windows Systems

  • Uses MoveFileEx() via ctypes with appropriate flags
  • MOVEFILE_WRITE_THROUGH flag ensures data reaches disk
  • MOVEFILE_REPLACE_EXISTING flag for overwrite operations
  • Handles Unicode paths correctly

Cross-Platform Guarantees

  • Atomic file replacement on same filesystem
  • Proper data synchronization before atomic operation
  • Consistent error handling across platforms
  • Unicode filename support