tessl/pypi-awkward
Manipulate JSON-like data with NumPy-like idioms for scientific computing and high-energy physics.
—
Pending
Quality
Pending
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
array-manipulation.mddocs/
Array Manipulation and Transformation
Structural operations for reshaping, filtering, combining, and transforming arrays while preserving type information and handling variable-length data gracefully. These operations form the core of awkward array's data manipulation capabilities.
Capabilities
Array Concatenation and Joining
Functions for combining multiple arrays along specified axes while handling complex nested structures and maintaining type consistency.
def concatenate(arrays, axis=0, highlevel=True, behavior=None):
"""
Concatenate arrays along the specified axis.
Parameters:
- arrays: sequence of Arrays to concatenate
- axis: int, axis along which to concatenate
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array containing concatenated data
"""
def zip(arrays, depth_limit=None, parameters=None, with_name=None,
highlevel=True, behavior=None):
"""
Combine arrays into a record structure with field names from dict keys.
Parameters:
- arrays: dict mapping field names to Arrays, or sequence of Arrays
- depth_limit: int, maximum depth to zip (None for unlimited)
- parameters: dict, parameters for the resulting record type
- with_name: str, name for the record type
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with record structure containing input arrays as fields
"""
def unzip(array, highlevel=True, behavior=None):
"""
Split a record array into separate arrays for each field.
Parameters:
- array: Array with record structure to unzip
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
dict mapping field names to Arrays
"""Structure Manipulation
Operations that change the shape and organization of nested data while preserving content and maintaining data integrity.
def flatten(array, axis=1, highlevel=True, behavior=None):
"""
Flatten nested lists by removing one level of list structure.
Parameters:
- array: Array to flatten
- axis: int, axis to flatten (default 1 removes innermost list level)
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with one fewer level of nesting
"""
def unflatten(array, counts, axis=0, highlevel=True, behavior=None):
"""
Add a level of list structure by partitioning elements according to counts.
Parameters:
- array: Array to unflatten
- counts: Array of integers specifying partition sizes
- axis: int, axis along which to unflatten
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with additional level of list nesting
"""
def ravel(array, highlevel=True, behavior=None):
"""
Flatten array to one dimension by removing all list structure.
Parameters:
- array: Array to ravel
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
One-dimensional Array containing all leaf elements
"""
def num(array, axis=1, highlevel=True, behavior=None):
"""
Count elements in each nested structure.
Parameters:
- array: Array to count elements in
- axis: int, axis along which to count
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array of integers representing element counts
"""Array Filtering and Selection
Functions for selecting subsets of data based on conditions, indices, or structural patterns while maintaining array structure.
def mask(array, selection, highlevel=True, behavior=None):
"""
Apply boolean mask to select elements.
Parameters:
- array: Array to mask
- selection: Array of booleans indicating which elements to select
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array containing only elements where selection is True
"""
def where(condition, x, y, highlevel=True, behavior=None):
"""
Select elements from x or y depending on condition.
Parameters:
- condition: Array of booleans
- x: Array of values to select when condition is True
- y: Array of values to select when condition is False
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with elements selected based on condition
"""
def drop_none(array, axis=None, highlevel=True, behavior=None):
"""
Remove None/missing values from array.
Parameters:
- array: Array to process
- axis: int, axis along which to drop None values (None for all axes)
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with None values removed
"""
def fill_none(array, value, axis=None, highlevel=True, behavior=None):
"""
Replace None/missing values with specified value.
Parameters:
- array: Array to process
- value: Value to use as replacement for None
- axis: int, axis along which to fill (None for all axes)
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with None values replaced by value
"""
def pad_none(array, target, axis=1, clip=False, highlevel=True, behavior=None):
"""
Pad variable-length lists to target length using None values.
Parameters:
- array: Array to pad
- target: int, target length for lists
- axis: int, axis along which to pad
- clip: bool, if True clip lists longer than target
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with lists padded to target length
"""Field and Attribute Manipulation
Operations for working with record structures, adding/removing fields, and managing metadata attributes.
def fields(array):
"""
Get field names from record array.
Parameters:
- array: Array with record structure
Returns:
list of str containing field names
"""
def with_field(array, what, where=None, highlevel=True, behavior=None):
"""
Add or replace a field in record array.
Parameters:
- array: Array with record structure
- what: Array containing values for the field
- where: str, field name (if None, what must be dict mapping names to values)
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with field added or modified
"""
def without_field(array, where, highlevel=True, behavior=None):
"""
Remove a field from record array.
Parameters:
- array: Array with record structure
- where: str or sequence of str, field name(s) to remove
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with specified field(s) removed
"""
def with_name(array, name, highlevel=True, behavior=None):
"""
Add a name to the array's type.
Parameters:
- array: Array to name
- name: str, name for the type
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with named type
"""
def with_parameter(array, key, value, highlevel=True, behavior=None):
"""
Add a parameter to the array's type.
Parameters:
- array: Array to modify
- key: str, parameter name
- value: parameter value
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with parameter added
"""
def without_parameters(array, highlevel=True, behavior=None):
"""
Remove all parameters from array's type.
Parameters:
- array: Array to modify
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array without type parameters
"""
def parameters(array):
"""
Get parameters from array's type.
Parameters:
- array: Array to examine
Returns:
dict of parameters from the array's type
"""Advanced Structural Operations
Complex operations for generating combinations, cartesian products, and other advanced structural transformations.
def combinations(array, n, axis=1, fields=None, parameters=None,
with_name=None, highlevel=True, behavior=None):
"""
Generate n-element combinations from each list in the array.
Parameters:
- array: Array containing lists to generate combinations from
- n: int, number of elements per combination
- axis: int, axis along which to generate combinations
- fields: list of str, field names for tuple elements
- parameters: dict, parameters for the resulting type
- with_name: str, name for the resulting type
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array of n-tuples containing all combinations
"""
def argcombinations(array, n, axis=1, fields=None, parameters=None,
with_name=None, highlevel=True, behavior=None):
"""
Generate indices of n-element combinations from each list.
Parameters:
- array: Array containing lists to generate combination indices from
- n: int, number of elements per combination
- axis: int, axis along which to generate combinations
- fields: list of str, field names for tuple elements
- parameters: dict, parameters for the resulting type
- with_name: str, name for the resulting type
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array of n-tuples containing indices of all combinations
"""
def cartesian(arrays, axis=1, nested=None, parameters=None, with_name=None,
highlevel=True, behavior=None):
"""
Generate cartesian product of arrays.
Parameters:
- arrays: dict mapping field names to Arrays, or sequence of Arrays
- axis: int, axis along which to form cartesian product
- nested: bool or sequence, control nesting behavior
- parameters: dict, parameters for the resulting type
- with_name: str, name for the resulting type
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array containing cartesian product as records or tuples
"""
def argcartesian(arrays, axis=1, nested=None, parameters=None, with_name=None,
highlevel=True, behavior=None):
"""
Generate indices of cartesian product elements.
Parameters:
- arrays: dict mapping field names to Arrays, or sequence of Arrays
- axis: int, axis along which to form cartesian product
- nested: bool or sequence, control nesting behavior
- parameters: dict, parameters for the resulting type
- with_name: str, name for the resulting type
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array containing indices of cartesian product elements
"""Array Transformation and Broadcasting
Functions for transforming array structure and ensuring compatible shapes for operations.
def broadcast_arrays(*arrays, depth_limit=None, highlevel=True, behavior=None):
"""
Broadcast arrays to a common structure.
Parameters:
- arrays: Arrays to broadcast
- depth_limit: int, maximum depth to broadcast (None for unlimited)
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
tuple of Arrays broadcasted to common structure
"""
def broadcast_fields(*arrays, highlevel=True, behavior=None):
"""
Broadcast record fields to common structure.
Parameters:
- arrays: Arrays with record structure to broadcast
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
tuple of Arrays with fields broadcasted to common structure
"""
def singletons(array, highlevel=True, behavior=None):
"""
Wrap each element in a list of length 1.
Parameters:
- array: Array to wrap in singletons
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array where each element is wrapped in a singleton list
"""
def firsts(array, axis=1, highlevel=True, behavior=None):
"""
Extract the first element from each list.
Parameters:
- array: Array containing lists
- axis: int, axis along which to extract firsts
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array containing first element from each list (None for empty lists)
"""Utility Functions
Helper functions for copying arrays, checking array properties, and performing basic transformations.
def copy(array, highlevel=True, behavior=None):
"""
Create a deep copy of the array.
Parameters:
- array: Array to copy
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Deep copy of the input array
"""
def transform(array, function, *args, highlevel=True, behavior=None, **kwargs):
"""
Apply a function to transform array structure.
Parameters:
- array: Array to transform
- function: callable that transforms Content layouts
- args: positional arguments for function
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
- kwargs: keyword arguments for function
Returns:
Array transformed by function
"""
def materialize(array, highlevel=True, behavior=None):
"""
Force materialization of lazy array operations.
Parameters:
- array: Array to materialize
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Materialized Array with all lazy operations evaluated
"""
def to_packed(array, highlevel=True, behavior=None):
"""
Pack array into contiguous memory layout.
Parameters:
- array: Array to pack
- highlevel: bool, if True return Array, if False return Content layout
- behavior: dict, custom behavior for the result
Returns:
Array with packed memory layout
"""Usage Examples
Basic Manipulation
import awkward as ak
# Create nested array
data = [[1, 2, 3], [4], [5, 6, 7, 8]]
array = ak.Array(data)
# Flatten one level
flat = ak.flatten(array) # [1, 2, 3, 4, 5, 6, 7, 8]
# Count elements per list
counts = ak.num(array) # [3, 1, 4]
# Unflatten back
reconstructed = ak.unflatten(flat, counts)Record Manipulation
import awkward as ak
# Create record array
records = ak.Array([
{"x": 1, "y": [1, 2]},
{"x": 2, "y": [3, 4, 5]}
])
# Add field
with_z = ak.with_field(records, [10, 20], "z")
# Remove field
without_y = ak.without_field(records, "y")
# Get field names
field_names = ak.fields(records) # ["x", "y"]Filtering and Selection
import awkward as ak
data = ak.Array([[1, 2, 3], [4], [5, 6]])
# Create mask (lists with more than 1 element)
mask = ak.num(data) > 1
# Apply mask
filtered = data[mask] # [[1, 2, 3], [5, 6]]
# Fill None values
with_nones = ak.Array([[1, None, 3], None, [5, 6]])
filled = ak.fill_none(with_nones, 0)Combinations and Products
import awkward as ak
# Generate combinations
data = ak.Array([[1, 2, 3, 4], [5, 6]])
pairs = ak.combinations(data, 2) # [[(1,2), (1,3), (1,4), (2,3), (2,4), (3,4)], [(5,6)]]
# Cartesian product of two arrays
a = ak.Array([[1, 2], [3]])
b = ak.Array([["x", "y"], ["z"]])
product = ak.cartesian({"a": a, "b": b})Install with Tessl CLI
npx tessl i tessl/pypi-awkward