CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/pypi-xorbits

Scalable Python data science, in an API compatible & lightning fast way.

Overview
Eval results
Files

numpy-integration.mddocs/

NumPy Integration

Distributed array computing with NumPy-compatible API. Xorbits numpy enables computation on large distributed arrays that exceed single-machine memory while maintaining full compatibility with the NumPy API.

Capabilities

Core Array Class

The fundamental distributed array class that mirrors numpy.ndarray functionality.

class ndarray:
    """
    Distributed array with numpy-compatible API.
    
    N-dimensional array object that supports all NumPy operations
    with automatic distribution across multiple workers for
    scalable array computing.
    """

Data Types

All NumPy data types are available for creating and working with distributed arrays.

# Boolean types
bool_: type          # Boolean type

# Integer types
int8: type           # 8-bit signed integer
int16: type          # 16-bit signed integer  
int32: type          # 32-bit signed integer
int64: type          # 64-bit signed integer
int_: type           # Default integer type
intc: type           # C integer type
intp: type           # Pointer-sized integer

uint8: type          # 8-bit unsigned integer
uint16: type         # 16-bit unsigned integer
uint32: type         # 32-bit unsigned integer
uint64: type         # 64-bit unsigned integer
uint: type           # Unsigned integer

# Floating point types
float16: type        # 16-bit floating point
float32: type        # 32-bit floating point
float64: type        # 64-bit floating point
double: type         # Double precision floating point
floating: type       # Floating point type

# Complex types
complex64: type      # 64-bit complex type
complex128: type     # 128-bit complex type
complexfloating: type # Complex floating point type

# String and character types
bytes_: type         # Bytes type
str_: type           # String type
character: type      # Character type

# Date and time types
datetime64: type     # 64-bit datetime type
timedelta64: type    # Time delta type

# Other types
object_: type        # Object type
void: type           # Void type
flexible: type       # Flexible type
generic: type        # Generic numpy type
inexact: type        # Inexact numeric type
number: type         # Number type
signedinteger: type  # Signed integer type
unsignedinteger: type # Unsigned integer type

# Type information classes
dtype: type          # Data type class
finfo: type          # Floating point type information

Constants

Mathematical and special constants from NumPy.

pi: float            # Pi constant (3.14159...)
e: float             # Euler's number (2.71828...)
inf: float           # Positive infinity
nan: float           # Not a Number
newaxis: object      # New axis constant for array indexing

Utility Classes

Classes for advanced array operations and error handling.

class errstate:
    """Error state context manager for controlling floating-point error handling."""

class ndindex:
    """Multi-dimensional index iterator for arrays."""

# Exceptions
class AxisError(Exception):
    """Exception raised for axis-related errors."""

Submodules

Specialized NumPy functionality through submodules.

# Submodules providing specialized functionality
fft      # Fast Fourier Transform functions
linalg   # Linear algebra functions  
random   # Random number generation
special  # Special mathematical functions

Dynamic Function Access

All NumPy functions are available through dynamic import, including but not limited to:

# Array creation functions
def array(object, dtype=None, **kwargs): ...
def zeros(shape, dtype=float, **kwargs): ...
def ones(shape, dtype=None, **kwargs): ...
def empty(shape, dtype=float, **kwargs): ...
def full(shape, fill_value, dtype=None, **kwargs): ...
def arange(start, stop=None, step=1, dtype=None, **kwargs): ...
def linspace(start, stop, num=50, **kwargs): ...
def logspace(start, stop, num=50, **kwargs): ...
def eye(N, M=None, k=0, dtype=float, **kwargs): ...
def identity(n, dtype=None, **kwargs): ...

# Array manipulation functions
def reshape(a, newshape, **kwargs): ...
def flatten(a, **kwargs): ...
def ravel(a, **kwargs): ...
def transpose(a, axes=None): ...
def concatenate(arrays, axis=0, **kwargs): ...
def stack(arrays, axis=0, **kwargs): ...
def split(ary, indices_or_sections, axis=0): ...
def squeeze(a, axis=None): ...
def expand_dims(a, axis): ...

# Mathematical functions
def add(x1, x2, **kwargs): ...
def subtract(x1, x2, **kwargs): ...
def multiply(x1, x2, **kwargs): ...
def divide(x1, x2, **kwargs): ...
def power(x1, x2, **kwargs): ...
def sqrt(x, **kwargs): ...
def exp(x, **kwargs): ...
def log(x, **kwargs): ...
def sin(x, **kwargs): ...
def cos(x, **kwargs): ...
def tan(x, **kwargs): ...

# Statistical functions
def mean(a, axis=None, **kwargs): ...
def median(a, axis=None, **kwargs): ...
def std(a, axis=None, **kwargs): ...
def var(a, axis=None, **kwargs): ...
def sum(a, axis=None, **kwargs): ...
def min(a, axis=None, **kwargs): ...
def max(a, axis=None, **kwargs): ...
def argmin(a, axis=None, **kwargs): ...
def argmax(a, axis=None, **kwargs): ...

# Logical functions
def all(a, axis=None, **kwargs): ...
def any(a, axis=None, **kwargs): ...
def logical_and(x1, x2, **kwargs): ...
def logical_or(x1, x2, **kwargs): ...
def logical_not(x, **kwargs): ...
def where(condition, x=None, y=None): ...

# Sorting and searching
def sort(a, axis=-1, **kwargs): ...
def argsort(a, axis=-1, **kwargs): ...
def searchsorted(a, v, **kwargs): ...
def unique(ar, **kwargs): ...

# Input/output functions
def save(file, arr, **kwargs): ...
def load(file, **kwargs): ...
def savetxt(fname, X, **kwargs): ...
def loadtxt(fname, **kwargs): ...

Usage Examples:

import xorbits
import xorbits.numpy as np

xorbits.init()

# Array creation (same as NumPy)
arr = np.array([1, 2, 3, 4, 5])
zeros_arr = np.zeros((1000, 1000))
ones_arr = np.ones((500, 500))
random_arr = np.random.random((1000, 1000))

# Mathematical operations (same as NumPy)
result = np.sqrt(arr + 10)
matrix_mult = np.dot(zeros_arr, ones_arr)

# Statistical operations
mean_val = np.mean(random_arr)
std_val = np.std(random_arr, axis=0)

# Array manipulation
reshaped = np.reshape(arr, (5, 1))
transposed = np.transpose(matrix_mult)

# All NumPy operations work the same way
complex_result = np.fft.fft(np.sin(np.linspace(0, 2*np.pi, 1000)))

# Execute computation
computed = xorbits.run(complex_result)

xorbits.shutdown()

Linear Algebra Examples

import xorbits.numpy as np
import xorbits

xorbits.init()

# Matrix operations
A = np.random.random((1000, 1000))
B = np.random.random((1000, 1000))

# Linear algebra operations
dot_product = np.dot(A, B)
eigenvals = np.linalg.eigvals(A)
determinant = np.linalg.det(A)
inverse = np.linalg.inv(A)

# Execute computations
results = xorbits.run(dot_product, eigenvals, determinant)

xorbits.shutdown()

FFT Examples

import xorbits.numpy as np
import xorbits

xorbits.init()

# Signal processing with FFT
signal = np.sin(2 * np.pi * np.linspace(0, 1, 10000))
fft_result = np.fft.fft(signal)
frequencies = np.fft.fftfreq(len(signal))

# Execute computation
computed_fft = xorbits.run(fft_result)

xorbits.shutdown()

Install with Tessl CLI

npx tessl i tessl/pypi-xorbits

docs

configuration.md

datasets.md

index.md

machine-learning.md

numpy-integration.md

pandas-integration.md

remote-computing.md

runtime-management.md

tile.json