or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

index.md
tile.json

tessl/pypi-ndjson

A simple and familiar interface for handling NDJSON (Newline Delimited JSON) data in Python

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/ndjson@0.3.x

To install, run

npx @tessl/cli install tessl/pypi-ndjson@0.3.0

index.mddocs/

ndjson

A simple and familiar interface for handling NDJSON (Newline Delimited JSON) data in Python. This package exposes the same API as Python's built-in json and pickle packages, making it easy to integrate into existing codebases. It includes JSONEncoder and JSONDecoder classes for seamless integration with other libraries like requests, as well as reader and writer classes similar to the standard csv module for streaming NDJSON data.

Package Information

  • Package Name: ndjson
  • Language: Python
  • Installation: pip install ndjson
  • Version: 0.3.1
  • License: GPL-3.0
  • Dependencies: None (zero dependencies)
  • Python Version Support: Python 2.7, 3.5-3.7

Core Imports

import ndjson

All public functions and classes are available directly from the main module:

# Functions
from ndjson import load, loads, dump, dumps

# Classes  
from ndjson import reader, writer, Decoder, Encoder

Basic Usage

import ndjson

# Load from file-like objects
with open('data.ndjson') as f:
    data = ndjson.load(f)

# Convert to and from objects
text = ndjson.dumps(data)
data = ndjson.loads(text)

# Dump to file-like objects
with open('backup.ndjson', 'w') as f:
    ndjson.dump(items, f)

# Streaming processing with reader/writer classes
with open('./posts.ndjson') as f:
    reader = ndjson.reader(f)
    for post in reader:
        print(post)

# Writing items to a ndjson file
with open('./posts.ndjson', 'w') as f:
    writer = ndjson.writer(f, ensure_ascii=False)
    for post in posts:
        writer.writerow(post)

Capabilities

Core Functions

Standard json-like API for loading and dumping NDJSON data.

def load(*args, **kwargs):
    """
    Load NDJSON from file-like object. Uses json.load() with ndjson.Decoder by default.
    
    Parameters:
    - *args: Arguments passed to json.load() (typically fp)
    - **kwargs: Keyword arguments passed to json.load() (cls defaults to ndjson.Decoder)
    
    Returns:
    List of parsed objects from each JSON line
    """

def loads(*args, **kwargs):
    """
    Load NDJSON from string. Uses json.loads() with ndjson.Decoder by default.
    
    Parameters:
    - *args: Arguments passed to json.loads() (typically s)
    - **kwargs: Keyword arguments passed to json.loads() (cls defaults to ndjson.Decoder)
    
    Returns:
    List of parsed objects from each JSON line
    """

def dump(obj, fp, cls=None, **kwargs):
    """
    Dump object to NDJSON file.
    
    Parameters:
    - obj: list/iterable of objects to serialize
    - fp: file-like object to write to
    - cls: JSONEncoder class to use (defaults to ndjson.Encoder if None)
    - **kwargs: additional arguments passed to encoder
    """

def dumps(*args, **kwargs):
    """
    Dump object to NDJSON string. Uses json.dumps() with ndjson.Encoder by default.
    
    Parameters:
    - *args: Arguments passed to json.dumps() (typically obj)
    - **kwargs: Keyword arguments passed to json.dumps() (cls defaults to ndjson.Encoder)
    
    Returns:
    String containing NDJSON text
    """

Streaming Classes

CSV-like reader and writer classes for efficient streaming processing of NDJSON files.

class reader:
    """
    CSV-like reader for streaming NDJSON input.
    
    Iterates over lines in a file-like object, parsing each JSON line.
    """
    
    def __init__(self, f, **kwargs):
        """
        Initialize reader with file-like object.
        
        Parameters:
        - f: file-like object containing NDJSON text
        - **kwargs: additional arguments passed to json.loads()
        """
    
    def __iter__(self):
        """Return iterator protocol support."""
    
    def __next__(self):
        """
        Get next JSON object from stream.
        
        Returns:
        Parsed object from next JSON line
        
        Raises:
        StopIteration: when end of file is reached
        """
    
    def next(self):
        """
        Python 2.7 compatibility wrapper for __next__().
        
        Returns:
        Parsed object from next JSON line
        
        Raises:
        StopIteration: when end of file is reached
        """

class writer:
    """
    CSV-like writer for streaming NDJSON output.
    
    Writes objects as JSON lines to a file-like object.
    """
    
    def __init__(self, f, **kwargs):
        """
        Initialize writer with file-like object.
        
        Parameters:
        - f: file-like object to write to
        - **kwargs: additional arguments passed to json.dumps()
        """
    
    def writerow(self, row):
        """
        Write single row as JSON line.
        
        Parameters:
        - row: object to serialize and write as JSON line
        """

Codec Classes

JSONEncoder and JSONDecoder subclasses for integration with other libraries.

class Decoder(json.JSONDecoder):
    """
    JSON decoder that handles NDJSON format.
    
    Converts NDJSON text (multiple JSON objects separated by newlines)
    into a list of Python objects.
    """
    
    def decode(self, s, *args, **kwargs):
        """
        Decode NDJSON string to list of objects.
        
        Parameters:
        - s: NDJSON string to decode
        - *args, **kwargs: additional arguments passed to parent decode()
        
        Returns:
        List of decoded objects from each JSON line
        """

class Encoder(json.JSONEncoder):
    """
    JSON encoder that outputs NDJSON format.
    
    Converts a list/iterable of Python objects into NDJSON text
    (multiple JSON objects separated by newlines).
    """
    
    def encode(self, obj, *args, **kwargs):
        """
        Encode list of objects to NDJSON string.
        
        Parameters:
        - obj: list/iterable of objects to encode
        - *args, **kwargs: additional arguments passed to parent encode()
        
        Returns:
        NDJSON string with objects separated by newlines
        """

Integration Examples

Using with requests

import ndjson
import requests

# Use custom decoder class with requests
response = requests.get('https://example.com/api/data')
items = response.json(cls=ndjson.Decoder)

Working with file-like objects

import ndjson
from io import StringIO

# Works with StringIO and other file-like objects
data = ['{"name": "Alice"}', '{"name": "Bob"}']
ndjson_text = '\\n'.join(data)

# Load from StringIO
f = StringIO(ndjson_text)
objects = ndjson.load(f)

# Dump to StringIO
output = StringIO()
ndjson.dump(objects, output)
result = output.getvalue()

Large file streaming

import ndjson

# Memory-efficient processing of large NDJSON files
def process_large_file(filename):
    with open(filename) as f:
        reader = ndjson.reader(f)
        for item in reader:
            # Process each item individually without loading entire file
            yield process_item(item)

# Writing large datasets
def write_large_dataset(filename, data_generator):
    with open(filename, 'w') as f:
        writer = ndjson.writer(f)
        for item in data_generator:
            writer.writerow(item)