0
# Error Handling
1

2
Deep Lake provides comprehensive exception handling for various failure scenarios including authentication, authorization, storage, dataset operations, and data validation. All exceptions are importable from the main deeplake module and provide detailed error information for debugging and recovery.
3

4
## Capabilities
5

6
### Authentication and Authorization Errors
7

8
Errors related to user authentication and access permissions for datasets and cloud services.
9

10
```python { .api }
11
class AuthenticationError:
12
    """Authentication failed or credentials invalid."""
13

14
class AuthorizationError:
15
    """User lacks permissions for requested operation."""
16

17
class AgreementError:
18
    """Dataset agreement acceptance required."""
19

20
class AgreementNotAcceptedError:
21
    """User has not accepted required agreements."""
22

23
class NotLoggedInAgreementError:
24
    """Agreement acceptance requires user login."""
25

26
class ExpiredTokenError:
27
    """Authentication token has expired."""
28
```
29

30
### Storage and Network Errors
31

32
Errors related to storage backend operations, network connectivity, and data transfer.
33

34
```python { .api }
35
class StorageAccessDenied:
36
    """Access denied to storage location."""
37

38
class StorageKeyNotFound:
39
    """Requested storage key does not exist."""
40

41
class StorageKeyAlreadyExists:
42
    """Storage key already exists, cannot overwrite."""
43

44
class StorageInternalError:
45
    """Internal storage system error."""
46

47
class StorageNetworkConnectionError:
48
    """Network connection to storage failed."""
49

50
class GcsStorageProviderFailed:
51
    """Google Cloud Storage operation failed."""
52
```
53

54
### Dataset Operation Errors
55

56
Errors related to dataset creation, access, modification, and lifecycle management.
57

58
```python { .api }
59
class DatasetUnavailableError:
60
    """Dataset temporarily unavailable."""
61

62
class NotFoundError:
63
    """Requested dataset or resource not found."""
64

65
class ReadOnlyDatasetModificationError:
66
    """Attempted modification of read-only dataset."""
67

68
class IncorrectDeeplakePathError:
69
    """Invalid Deep Lake dataset path format."""
70

71
class InvalidURIError:
72
    """Invalid URI format provided."""
73

74
class PermissionDeniedError:
75
    """Insufficient permissions for operation."""
76

77
class BadRequestError:
78
    """Invalid request parameters or format."""
79
```
80

81
### Version Control Errors
82

83
Errors related to branching, tagging, merging, and version management operations.
84

85
```python { .api }
86
class BranchExistsError:
87
    """Branch with given name already exists."""
88

89
class BranchNotFoundError:
90
    """Requested branch does not exist."""
91

92
class CannotDeleteMainBranchError:
93
    """Main branch cannot be deleted."""
94

95
class CannotRenameMainBranchError:
96
    """Main branch cannot be renamed."""
97

98
class TagExistsError:
99
    """Tag with given name already exists."""
100

101
class TagNotFoundError:
102
    """Requested tag does not exist."""
103

104
class VersionNotFoundError:
105
    """Requested version does not exist."""
106

107
class PushError:
108
    """Error occurred during push operation."""
109
```
110

111
### Column and Schema Errors
112

113
Errors related to column operations, schema validation, and data structure modifications.
114

115
```python { .api }
116
class ColumnAlreadyExistsError:
117
    """Column with given name already exists."""
118

119
class ColumnDoesNotExistError:
120
    """Requested column does not exist."""
121

122
class EmptyColumnNameError:
123
    """Column name cannot be empty."""
124

125
class ColumnMissingAppendValueError:
126
    """Value required for column during append operation."""
127

128
class InvalidColumnValueError:
129
    """Invalid value provided for column."""
130

131
class UnevenColumnsError:
132
    """Columns have mismatched lengths."""
133

134
class UnevenUpdateError:
135
    """Update data has inconsistent dimensions."""
136
```
137

138
### Data Type and Validation Errors
139

140
Errors related to data types, dimensions, format validation, and compression settings.
141

142
```python { .api }
143
class DimensionsMismatch:
144
    """Data dimensions do not match expected schema."""
145

146
class DimensionsMismatchError:
147
    """Dimension validation failed."""
148

149
class DtypeMismatch:
150
    """Data type does not match column type."""
151

152
class InvalidType:
153
    """Invalid or unsupported data type."""
154

155
class InvalidTypeDimensions:
156
    """Invalid dimensions for given type."""
157

158
class UnknownType:
159
    """Unknown or unrecognized data type."""
160

161
class UnspecifiedDtype:
162
    """Data type must be specified."""
163

164
class UnsupportedPythonType:
165
    """Python type not supported for conversion."""
166

167
class InvalidTypeAndFormatPair:
168
    """Type and format combination not supported."""
169
```
170

171
### Compression and Format Errors
172

173
Errors related to data compression, format validation, and encoding issues.
174

175
```python { .api }
176
class FormatNotSupportedError:
177
    """Data format not supported."""
178

179
class UnknownFormat:
180
    """Unrecognized data format."""
181

182
class InvalidImageCompression:
183
    """Invalid compression settings for image data."""
184

185
class InvalidBinaryMaskCompression:
186
    """Invalid compression for binary mask data."""
187

188
class InvalidSegmentMaskCompression:
189
    """Invalid compression for segmentation mask data."""
190

191
class InvalidMedicalCompression:
192
    """Invalid compression for medical imaging data."""
193

194
class UnsupportedChunkCompression:
195
    """Chunk compression type not supported."""
196

197
class UnsupportedSampleCompression:
198
    """Sample compression type not supported."""
199

200
class WrongChunkCompression:
201
    """Incorrect chunk compression applied."""
202

203
class WrongSampleCompression:
204
    """Incorrect sample compression applied."""
205
```
206

207
### Index and Query Errors
208

209
Errors related to indexing operations, query execution, and search functionality.
210

211
```python { .api }
212
class IndexAlreadyExistsError:
213
    """Index with given configuration already exists."""
214

215
class InvalidIndexCreationError:
216
    """Index creation parameters invalid."""
217

218
class JSONIndexNotFound:
219
    """JSON index not found for query."""
220

221
class JSONKeyNotFound:
222
    """Requested JSON key not found."""
223
```
224

225
### Data Processing and Ingestion Errors
226

227
Errors related to data processing, ingestion pipelines, and specialized data formats.
228

229
```python { .api }
230
class InvalidLinkDataError:
231
    """Linked data validation failed."""
232

233
class InvalidLinkType:
234
    """Invalid link type specified."""
235

236
class InvalidSequenceOfSequence:
237
    """Invalid nested sequence structure."""
238

239
class InvalidPolygonShapeError:
240
    """Polygon data has invalid shape."""
241

242
class UnexpectedInputDataForDicomColumn:
243
    """DICOM column received unexpected data type."""
244

245
class UnexpectedMedicalTypeInputData:
246
    """Medical imaging data validation failed."""
247

248
class UnknownBoundingBoxCoordinateFormat:
249
    """Bounding box coordinate format not recognized."""
250

251
class UnknownBoundingBoxPixelFormat:
252
    """Bounding box pixel format not recognized."""
253

254
class UnknownStringType:
255
    """String type specification not recognized."""
256

257
class InvalidTextType:
258
    """Text data type validation failed."""
259
```
260

261
### Tensor and Array Errors
262

263
Errors related to tensor operations, array indexing, and chunk management.
264

265
```python { .api }
266
class TensorAlreadyExists:
267
    """Tensor with given name already exists."""
268

269
class BytePositionIndexOutOfChunk:
270
    """Byte position index exceeds chunk boundaries."""
271

272
class ShapeIndexOutOfChunk:
273
    """Shape index exceeds chunk boundaries."""
274

275
class CanNotCreateTensorWithProvidedCompressions:
276
    """Cannot create tensor with specified compression settings."""
277
```
278

279
### HTTP and Network Errors
280

281
Errors related to HTTP requests, API communication, and network operations.
282

283
```python { .api }
284
class HTTPRequestFailedError:
285
    """HTTP request failed with error response."""
286

287
class HTTPBodyIsMissingError:
288
    """Required HTTP body is missing."""
289

290
class HTTPBodyIsNotJSONError:
291
    """HTTP body is not valid JSON format."""
292
```
293

294
### Configuration and Credentials Errors
295

296
Errors related to configuration management and credential handling.
297

298
```python { .api }
299
class CredsKeyAlreadyAssignedError:
300
    """Credentials key already assigned to dataset."""
301

302
class InvalidCredsKeyAssignmentError:
303
    """Invalid credentials key assignment."""
304
```
305

306
### Specialized Data Format Errors
307

308
Errors related to specialized data formats and processing operations.
309

310
```python { .api }
311
class EmbeddingSizeMismatch:
312
    """Embedding vector size does not match expected dimensions."""
313

314
class InvalidChunkStrategyType:
315
    """Invalid chunk strategy specified."""
316

317
class WriteFailedError:
318
    """Data write operation failed."""
319

320
class LogExistsError:
321
    """Log file already exists."""
322

323
class LogNotexistsError:
324
    """Required log file does not exist."""
325
```
326

327
## Usage Examples
328

329
### Basic Error Handling
330

331
```python
332
import deeplake
333

334
try:
335
    dataset = deeplake.open("./nonexistent_dataset")
336
except deeplake.NotFoundError:
337
    print("Dataset not found, creating new one...")
338
    dataset = deeplake.create("./new_dataset")
339

340
try:
341
    dataset.add_column("images", deeplake.types.Image())
342
except deeplake.ColumnAlreadyExistsError:
343
    print("Column 'images' already exists")
344
```
345

346
### Storage and Authentication Errors
347

348
```python
349
try:
350
    dataset = deeplake.create(
351
        "s3://my-bucket/dataset",
352
        creds={"aws_access_key_id": "invalid", "aws_secret_access_key": "invalid"}
353
    )
354
except deeplake.AuthenticationError:
355
    print("AWS credentials are invalid")
356
except deeplake.StorageAccessDenied:
357
    print("Access denied to S3 bucket")
358
except deeplake.StorageNetworkConnectionError:
359
    print("Network connection to S3 failed")
360
```
361

362
### Version Control Error Handling
363

364
```python
365
try:
366
    branch = dataset.branch("feature-branch")
367
except deeplake.BranchExistsError:
368
    print("Branch already exists, switching to it...")
369
    branch = dataset.branches["feature-branch"]
370

371
try:
372
    dataset.tag("v1.0", "Release version 1.0")
373
except deeplake.TagExistsError:
374
    print("Tag v1.0 already exists")
375
```
376

377
### Data Validation Errors
378

379
```python
380
try:
381
    dataset.append({
382
        "images": "invalid_image_path.txt",  # Wrong file type
383
        "embeddings": [1, 2, 3]  # Wrong dimensions
384
    })
385
except deeplake.FormatNotSupportedError:
386
    print("Image format not supported")
387
except deeplake.DimensionsMismatch:
388
    print("Embedding dimensions don't match schema")
389
except deeplake.InvalidColumnValueError as e:
390
    print(f"Invalid column value: {e}")
391
```
392

393
### Comprehensive Error Handling
394

395
```python
396
def safe_dataset_operation(url, operation_func):
397
    """Safely execute dataset operations with comprehensive error handling."""
398
    try:
399
        return operation_func(url)
400
    except deeplake.NotFoundError:
401
        print(f"Dataset not found: {url}")
402
    except deeplake.AuthenticationError:
403
        print("Authentication failed - check your token")
404
    except deeplake.AuthorizationError:
405
        print("Access denied - insufficient permissions")
406
    except deeplake.StorageNetworkConnectionError:
407
        print("Network connection failed - check connectivity")
408
    except deeplake.ReadOnlyDatasetModificationError:
409
        print("Cannot modify read-only dataset")
410
    except deeplake.DatasetUnavailableError:
411
        print("Dataset temporarily unavailable - try again later")
412
    except Exception as e:
413
        print(f"Unexpected error: {type(e).__name__}: {e}")
414
        raise
415

416
# Usage
417
safe_dataset_operation("./my_dataset", lambda url: deeplake.open(url))
418
```
419

420
### Query Error Handling
421

422
```python
423
try:
424
    results = deeplake.query("SELECT * FROM invalid_dataset WHERE column = 'value'")
425
except deeplake.NotFoundError:
426
    print("Referenced dataset not found in query")
427
except deeplake.JSONKeyNotFound:
428
    print("Query references non-existent column")
429
except deeplake.BadRequestError:
430
    print("Invalid query syntax")
431
```
432

433
## Error Recovery Strategies
434

435
### Automatic Retry Pattern
436

437
```python
438
import time
439
import random
440

441
def retry_with_backoff(func, max_retries=3, base_delay=1):
442
    """Retry function with exponential backoff for transient errors."""
443
    for attempt in range(max_retries):
444
        try:
445
            return func()
446
        except (deeplake.StorageNetworkConnectionError, 
447
                deeplake.DatasetUnavailableError,
448
                deeplake.StorageInternalError) as e:
449
            if attempt == max_retries - 1:
450
                raise
451
            delay = base_delay * (2 ** attempt) + random.uniform(0, 1)
452
            print(f"Transient error {type(e).__name__}, retrying in {delay:.2f}s...")
453
            time.sleep(delay)
454

455
# Usage
456
dataset = retry_with_backoff(lambda: deeplake.open("s3://my-bucket/dataset"))
457
```
458

459
### Fallback Strategy
460

461
```python
462
def open_dataset_with_fallback(primary_url, fallback_url):
463
    """Open dataset with fallback location."""
464
    try:
465
        return deeplake.open(primary_url)
466
    except (deeplake.NotFoundError, deeplake.StorageAccessDenied):
467
        print(f"Primary dataset unavailable, using fallback: {fallback_url}")
468
        return deeplake.open(fallback_url)
469

470
dataset = open_dataset_with_fallback(
471
    "s3://primary-bucket/dataset",
472
    "./local_backup/dataset"
473
)
474
```