0
# Arctic
1

2
A high-performance versioned TimeSeries and tick database designed for financial data storage and analysis. Arctic provides multiple storage engines including VersionStore for versioned key-value TimeSeries data with pandas support, TickStore for column-oriented tick data, and ChunkStore for customizable chunk-based storage.
3

4
## Package Information
5

6
- **Package Name**: arctic
7
- **Language**: Python
8
- **Installation**: `pip install arctic`
9

10
## Core Imports
11

12
```python
13
import arctic
14
```
15

16
Common usage pattern:
17

18
```python
19
from arctic import Arctic, VERSION_STORE, TICK_STORE, CHUNK_STORE
20
```
21

22
## Basic Usage
23

24
```python
25
from arctic import Arctic, VERSION_STORE
26
import pandas as pd
27
import numpy as np
28

29
# Connect to Arctic
30
arctic_conn = Arctic('mongodb://localhost:27017')
31

32
# Initialize a library
33
arctic_conn.initialize_library('my_data', VERSION_STORE)
34

35
# Get library
36
library = arctic_conn['my_data']
37

38
# Create sample data
39
data = pd.DataFrame({
40
    'price': np.random.randn(1000),
41
    'volume': np.random.randint(100, 1000, 1000)
42
}, index=pd.date_range('2020-01-01', periods=1000, freq='min'))
43

44
# Write data
45
library.write('AAPL', data)
46

47
# Read data back
48
retrieved_data = library.read('AAPL')
49
print(retrieved_data.data.head())
50
```
51

52
## Architecture
53

54
Arctic uses a multi-layered architecture optimized for financial time series data:
55

56
- **Arctic**: Top-level connection manager handling MongoDB connections, authentication, and library dispatch
57
- **ArcticLibraryBinding**: Library-specific interface providing quota management, metadata operations, and store access
58
- **Storage Engines**: Specialized backends for different data types and use cases
59
  - **VersionStore**: Versioned storage for pandas DataFrames/Series with temporal snapshots
60
  - **TickStore**: High-frequency tick data with efficient columnar storage and date partitioning
61
  - **ChunkStore**: Configurable chunking for large datasets with custom serialization
62
  - **BSONStore**: Raw MongoDB document storage with full PyMongo interface
63
  - **MetadataStore**: Time-series metadata storage with historical tracking
64

65
## Capabilities
66

67
### Arctic Connection Management
68

69
Core functionality for connecting to MongoDB, managing libraries, and handling authentication. Provides the main entry point for all Arctic operations including library lifecycle management and quota control.
70

71
```python { .api }
72
class Arctic:
73
    def __init__(self, mongo_host, app_name='arctic', allow_secondary=False, 
74
                 socketTimeoutMS=600000, connectTimeoutMS=2000, 
75
                 serverSelectionTimeoutMS=30000, **kwargs): ...
76
    def list_libraries(self, newer_than_secs=None): ...
77
    def initialize_library(self, library, lib_type=VERSION_STORE, **kwargs): ...
78
    def get_library(self, library): ...
79
    def delete_library(self, library): ...
80

81
def register_library_type(name, type_): ...
82
```
83

84
[Arctic Connection](./arctic-connection.md)
85

86
### Version Store Operations
87

88
Versioned storage for pandas DataFrames and Series with complete audit trails, point-in-time snapshots, and efficient data retrieval. Supports temporal data access, metadata management, and multi-version data handling.
89

90
```python { .api }
91
class VersionStore:
92
    def read(self, symbol, as_of=None, date_range=None, from_version=None, **kwargs): ...
93
    def write(self, symbol, data, metadata=None, prune_previous_version=True, **kwargs): ...
94
    def append(self, symbol, data, metadata=None, prune_previous_version=True, **kwargs): ...
95
    def list_versions(self, symbol=None, snapshot=None, latest_only=False): ...
96
    def snapshot(self, snap_name, metadata=None, skip_symbols=None, versions=None): ...
97
```
98

99
[Version Store](./version-store.md)
100

101
### Tick Store Operations
102

103
High-frequency tick data storage with efficient columnar compression and time-based partitioning. Optimized for financial tick data with support for initial images, metadata persistence, and date range queries.
104

105
```python { .api }
106
class TickStore:
107
    def __init__(self, arctic_lib, chunk_size=100000): ...
108
    def read(self, symbol, date_range=None, columns=None, include_images=False, **kwargs): ...
109
    def write(self, symbol, data, initial_image=None, metadata=None): ...
110
    def max_date(self, symbol): ...
111
    def min_date(self, symbol): ...
112
```
113

114
[Tick Store](./tick-store.md)
115

116
### Chunk Store Operations
117

118
Configurable chunked storage for large datasets with custom serialization strategies and date-based chunking. Supports append/update operations, audit trails, and flexible data organization patterns.
119

120
```python { .api }
121
class ChunkStore:
122
    def read(self, symbol, chunk_range=None, filter_data=True, **kwargs): ...
123
    def write(self, symbol, item, metadata=None, chunker=DateChunker(), **kwargs): ...
124
    def append(self, symbol, item, upsert=False, metadata=None, **kwargs): ...
125
    def update(self, symbol, item, chunk_range=None, upsert=False, **kwargs): ...
126
    def get_chunk_ranges(self, symbol, chunk_range=None, reverse=False): ...
127
```
128

129
[Chunk Store](./chunk-store.md)
130

131
### BSON Store Operations
132

133
Raw MongoDB document storage providing full PyMongo interface for direct database operations. Enables custom document structures, aggregation pipelines, and advanced MongoDB features within Arctic's framework.
134

135
```python { .api }
136
class BSONStore:
137
    def find(self, *args, **kwargs): ...
138
    def insert_one(self, document, **kwargs): ...
139
    def update_one(self, filter, update, **kwargs): ...
140
    def aggregate(self, pipeline, **kwargs): ...
141
    def create_index(self, keys, **kwargs): ...
142
```
143

144
[BSON Store](./bson-store.md)
145

146
### Date and Time Utilities
147

148
Comprehensive date/time handling for financial data including timezone management, date range operations, and time series utilities. Provides robust datetime conversion and time-based data filtering capabilities.
149

150
```python { .api }
151
class DateRange:
152
    def __init__(self, start, end, step=None): ...
153

154
def mktz(timezone): ...
155
def datetime_to_ms(dt): ...
156
def ms_to_datetime(ms): ...
157
def to_pandas_closed_closed(date_range): ...
158
```
159

160
[Date Utilities](./date-utilities.md)
161

162
### Asynchronous Operations
163

164
Asynchronous execution framework for Arctic operations enabling concurrent data processing and improved performance for batch operations. Provides thread pool management and request tracking.
165

166
```python { .api }
167
def async_arctic_submit(store, fun, is_modifier, *args, **kwargs): ...
168
def async_wait_request(request, timeout=None): ...
169
def async_wait_requests(requests, timeout=None): ...
170
def async_shutdown(timeout=None): ...
171
```
172

173
[Async Operations](./async-operations.md)
174

175
## Types
176

177
### Core Types
178

179
```python { .api }
180
class Arctic:
181
    """Main Arctic database connection and library manager."""
182
    
183
class ArcticLibraryBinding:
184
    """Library-specific interface for Arctic operations."""
185

186
class VersionedItem:
187
    """Container for versioned data with metadata."""
188
    def __init__(self, symbol, library, data, version, metadata, host=None): ...
189
    def metadata_dict(self): ...
190

191
class DateRange:
192
    """Date/time range specification for queries and operations."""
193
    def __init__(self, start, end, step=None): ...
194
```
195

196
### Store Type Constants
197

198
```python { .api }
199
VERSION_STORE = "VersionStore"
200
TICK_STORE = "TickStoreV3"  
201
CHUNK_STORE = "ChunkStoreV1"
202
METADATA_STORE = "MetadataStore"
203
BSON_STORE = "BSONStore"
204
```
205

206
### Exception Types
207

208
```python { .api }
209
class ArcticException(Exception):
210
    """Base exception for all Arctic errors."""
211

212
class NoDataFoundException(ArcticException):
213
    """Raised when requested data doesn't exist."""
214

215
class LibraryNotFoundException(ArcticException):
216
    """Raised when library doesn't exist."""
217

218
class QuotaExceededException(ArcticException):
219
    """Raised when storage quota exceeded."""
220

221
class UnhandledDtypeException(ArcticException):
222
    """Raised for unsupported data types."""
223

224
class OverlappingDataException(ArcticException):
225
    """Raised when data ranges overlap improperly."""
226

227
class UnorderedDataException(ArcticException):
228
    """Raised when data isn't properly time-ordered."""
229
```