0
# Utilities and Helpers
1

2
Utility functions for dataset processing, model hub integration, and common operations. Keras Hub provides helpers for working with standard datasets and uploading models to various hubs.
3

4
## Capabilities
5

6
### Model Hub Integration
7

8
Functions for uploading and managing model presets on various model hubs.
9

10
```python { .api }
11
def upload_preset(uri: str, preset: str) -> None:
12
    """
13
    Upload a preset directory to a model hub.
14
    
15
    Supports uploading to Kaggle Models and other model hosting platforms.
16
    The URI format determines the target platform:
17
    - Kaggle: kaggle://<username>/<model>/<framework>/<variation>
18
    - HuggingFace: hf://<username>/<model>
19
    - Google Cloud Storage: gs://<bucket>/<path>
20
    
21
    Args:
22
        uri: The URI identifying where to upload the model.
23
             Format depends on the target platform.
24
        preset: Path to the local preset directory containing model files,
25
                configuration, and metadata.
26
                
27
    Raises:
28
        ValueError: If URI format is invalid or unsupported
29
        ConnectionError: If upload fails due to network issues
30
        AuthenticationError: If credentials are invalid or missing
31
        
32
    Example:
33
        # Upload to Kaggle Models
34
        keras_hub.upload_preset(
35
            "kaggle://username/bert-sentiment/keras/v1",
36
            "./my_bert_preset"
37
        )
38
    """
39
    ...
40
```
41

42
### Dataset Utilities
43

44
Helper functions for working with popular computer vision datasets.
45

46
```python { .api }
47
def imagenet_id_to_name(class_id: int) -> str:
48
    """
49
    Convert ImageNet class ID to human-readable class name.
50
    
51
    Args:
52
        class_id: Integer class ID (0-999 for ImageNet-1K)
53
        
54
    Returns:
55
        Human-readable class name string
56
        
57
    Raises:
58
        ValueError: If class_id is outside valid range
59
        
60
    Example:
61
        name = keras_hub.utils.imagenet_id_to_name(285)
62
        # Returns: "Egyptian cat"
63
    """
64
    ...
65

66
def imagenet_name_to_id(class_name: str) -> int:
67
    """
68
    Convert ImageNet class name to class ID.
69
    
70
    Args:
71
        class_name: Human-readable class name
72
        
73
    Returns:
74
        Integer class ID
75
        
76
    Raises:
77
        KeyError: If class_name not found in ImageNet classes
78
        
79
    Example:
80
        class_id = keras_hub.utils.imagenet_name_to_id("Egyptian cat")
81
        # Returns: 285
82
    """
83
    ...
84

85
def decode_imagenet_predictions(predictions, top: int = 5):
86
    """
87
    Decode ImageNet model predictions to human-readable format.
88
    
89
    Args:
90
        predictions: Model prediction array of shape (batch_size, 1000)
91
                    or (1000,) for single prediction
92
        top: Number of top predictions to return
93
        
94
    Returns:
95
        List of tuples (class_id, class_name, probability) for top predictions
96
        
97
    Example:
98
        # predictions shape: (1, 1000)
99
        decoded = keras_hub.utils.decode_imagenet_predictions(predictions, top=3)
100
        # Returns: [(285, "Egyptian cat", 0.85), (281, "tabby cat", 0.12), ...]
101
    """
102
    ...
103
```
104

105
### COCO Dataset Utilities
106

107
Helper functions for working with the COCO (Common Objects in Context) dataset.
108

109
```python { .api }
110
def coco_id_to_name(class_id: int) -> str:
111
    """
112
    Convert COCO class ID to human-readable class name.
113
    
114
    Args:
115
        class_id: Integer class ID (1-80 for COCO)
116
        
117
    Returns:
118
        Human-readable class name string
119
        
120
    Raises:
121
        ValueError: If class_id is outside valid range
122
        
123
    Example:
124
        name = keras_hub.utils.coco_id_to_name(1)
125
        # Returns: "person"
126
    """
127
    ...
128

129
def coco_name_to_id(class_name: str) -> int:
130
    """
131
    Convert COCO class name to class ID.
132
    
133
    Args:
134
        class_name: Human-readable class name
135
        
136
    Returns:
137
        Integer class ID
138
        
139
    Raises:
140
        KeyError: If class_name not found in COCO classes
141
        
142
    Example:
143
        class_id = keras_hub.utils.coco_name_to_id("person")
144
        # Returns: 1
145
    """
146
    ...
147
```
148

149
### Version Information
150

151
Functions and constants for accessing version information.
152

153
```python { .api }
154
def version() -> str:
155
    """
156
    Return the current Keras Hub version string.
157
    
158
    Returns:
159
        Version string in semantic versioning format
160
        
161
    Example:
162
        version_str = keras_hub.version()
163
        # Returns: "0.22.1"
164
    """
165
    ...
166

167
__version__: str = "0.22.1"
168
"""Current Keras Hub version as a string constant."""
169
```
170

171
## Usage Examples
172

173
### Working with ImageNet Predictions
174

175
```python
176
import keras_hub
177
import numpy as np
178

179
# Load an ImageNet-trained model
180
model = keras_hub.models.ResNetImageClassifier.from_preset("resnet50_imagenet")
181

182
# Make predictions (example with random data)
183
images = np.random.random((2, 224, 224, 3))
184
predictions = model.predict(images)
185

186
# Decode predictions to human-readable format
187
for i, pred in enumerate(predictions):
188
    decoded = keras_hub.utils.decode_imagenet_predictions(pred, top=3)
189
    print(f"Image {i+1} predictions:")
190
    for class_id, class_name, probability in decoded:
191
        print(f"  {class_name} (ID: {class_id}): {probability:.3f}")
192
    print()
193
```
194

195
### Converting Class IDs and Names
196

197
```python
198
import keras_hub
199

200
# Convert ImageNet class ID to name
201
class_id = 285
202
class_name = keras_hub.utils.imagenet_id_to_name(class_id)
203
print(f"Class {class_id}: {class_name}")
204

205
# Convert back to ID
206
recovered_id = keras_hub.utils.imagenet_name_to_id(class_name)
207
print(f"Class '{class_name}': ID {recovered_id}")
208

209
# COCO dataset utilities
210
coco_id = 1
211
coco_name = keras_hub.utils.coco_id_to_name(coco_id)
212
print(f"COCO class {coco_id}: {coco_name}")
213

214
coco_recovered_id = keras_hub.utils.coco_name_to_id(coco_name)
215
print(f"COCO class '{coco_name}': ID {coco_recovered_id}")
216
```
217

218
### Uploading Model Presets
219

220
```python
221
import keras_hub
222

223
# Train and save a custom model
224
model = keras_hub.models.BertTextClassifier.from_preset("bert_base_en", num_classes=2)
225

226
# Fine-tune the model on your data
227
# model.fit(train_data, epochs=3)
228

229
# Save as preset
230
model.save_to_preset("./my_custom_bert_sentiment")
231

232
# Upload to Kaggle Models
233
try:
234
    keras_hub.upload_preset(
235
        uri="kaggle://myusername/bert-sentiment-classifier/keras/v1",
236
        preset="./my_custom_bert_sentiment"
237
    )
238
    print("Successfully uploaded to Kaggle Models!")
239
except Exception as e:
240
    print(f"Upload failed: {e}")
241
```
242

243
### Version Information
244

245
```python
246
import keras_hub
247

248
# Get version information
249
current_version = keras_hub.version()
250
print(f"Keras Hub version: {current_version}")
251

252
# Access version constant
253
version_constant = keras_hub.__version__
254
print(f"Version constant: {version_constant}")
255

256
# Check version compatibility
257
required_version = "0.22.0"
258
if keras_hub.__version__ >= required_version:
259
    print("Version requirements satisfied")
260
else:
261
    print(f"Please upgrade to version {required_version} or higher")
262
```
263

264
### Batch Processing ImageNet Results
265

266
```python
267
import keras_hub
268
import numpy as np
269

270
def process_imagenet_batch(model, images, top_k=5):
271
    """Process a batch of images and return decoded predictions."""
272
    predictions = model.predict(images)
273
    
274
    results = []
275
    for i, pred in enumerate(predictions):
276
        decoded = keras_hub.utils.decode_imagenet_predictions(pred, top=top_k)
277
        results.append({
278
            'image_index': i,
279
            'predictions': decoded
280
        })
281
    
282
    return results
283

284
# Example usage
285
model = keras_hub.models.EfficientNetImageClassifier.from_preset("efficientnet_b0_imagenet")
286
batch_images = np.random.random((4, 224, 224, 3))
287

288
results = process_imagenet_batch(model, batch_images, top_k=3)
289

290
for result in results:
291
    print(f"Image {result['image_index']}:")
292
    for class_id, class_name, prob in result['predictions']:
293
        print(f"  {class_name}: {prob:.3f}")
294
    print()
295
```
296

297
### Object Detection with COCO Utilities
298

299
```python
300
import keras_hub
301
import numpy as np
302

303
# Load object detection model
304
detector = keras_hub.models.RetinaNetObjectDetector.from_preset("retinanet_resnet50_pascalvoc")
305

306
# Make predictions (example with random data)
307
images = np.random.random((1, 512, 512, 3))
308
detections = detector.predict(images)
309

310
# Process detections (assuming COCO-format output)
311
def process_detections(detections, confidence_threshold=0.5):
312
    """Process detection results and convert class IDs to names."""
313
    processed = []
314
    
315
    # Extract detection components (format depends on model)
316
    boxes = detections['boxes'][0]  # Bounding boxes
317
    classes = detections['classes'][0]  # Class predictions
318
    scores = detections['scores'][0]  # Confidence scores
319
    
320
    # Filter by confidence
321
    valid_detections = scores > confidence_threshold
322
    
323
    for i in range(len(boxes)):
324
        if valid_detections[i]:
325
            class_id = int(classes[i])
326
            try:
327
                class_name = keras_hub.utils.coco_id_to_name(class_id)
328
            except ValueError:
329
                class_name = f"Class_{class_id}"
330
            
331
            processed.append({
332
                'box': boxes[i],
333
                'class_id': class_id,
334
                'class_name': class_name,
335
                'score': float(scores[i])
336
            })
337
    
338
    return processed
339

340
# Process results
341
results = process_detections(detections)
342
for detection in results:
343
    print(f"Detected {detection['class_name']} "
344
          f"(score: {detection['score']:.3f}) "
345
          f"at {detection['box']}")
346
```
347

348
### Model Hub Upload with Error Handling
349

350
```python
351
import keras_hub
352
import os
353

354
def upload_model_safely(model, preset_name, upload_uri):
355
    """Safely upload a model with proper error handling."""
356
    
357
    # Save model to temporary preset directory
358
    temp_preset_dir = f"./temp_preset_{preset_name}"
359
    
360
    try:
361
        # Save model as preset
362
        print("Saving model as preset...")
363
        model.save_to_preset(temp_preset_dir)
364
        
365
        # Upload to model hub
366
        print(f"Uploading to {upload_uri}...")
367
        keras_hub.upload_preset(upload_uri, temp_preset_dir)
368
        
369
        print("Upload completed successfully!")
370
        return True
371
        
372
    except FileNotFoundError:
373
        print("Error: Model files not found")
374
        return False
375
    except ConnectionError:
376
        print("Error: Network connection failed")
377
        return False
378
    except PermissionError:
379
        print("Error: Authentication failed or insufficient permissions")
380
        return False
381
    except Exception as e:
382
        print(f"Unexpected error: {e}")
383
        return False
384
    finally:
385
        # Clean up temporary files
386
        if os.path.exists(temp_preset_dir):
387
            import shutil
388
            shutil.rmtree(temp_preset_dir)
389
            print("Cleaned up temporary files")
390

391
# Example usage
392
model = keras_hub.models.GPT2CausalLM.from_preset("gpt2_base_en")
393
success = upload_model_safely(
394
    model, 
395
    "my_gpt2_model",
396
    "kaggle://username/my-gpt2/keras/v1"
397
)
398

399
if success:
400
    print("Model is now available on Kaggle Models!")
401
```
402

403
### Tokenizer Utilities
404

405
Functions for creating and managing tokenizer vocabularies.
406

407
```python { .api }
408
def compute_sentence_piece_proto(
409
    data: list,
410
    vocab_size: int,
411
    model_type: str = "unigram",
412
    pad_id: int = 0,
413
    eos_id: int = 1,
414
    unk_id: int = 2,
415
    bos_id: int = -1,
416
    **kwargs
417
) -> bytes:
418
    """
419
    Compute a SentencePiece model proto from training data.
420
    
421
    Creates a SentencePiece model protocol buffer that can be used
422
    to initialize SentencePiece tokenizers.
423
    
424
    Args:
425
        data: List of strings to train the tokenizer on
426
        vocab_size: Target vocabulary size
427
        model_type: SentencePiece model type ("unigram", "bpe", "word", "char")
428
        pad_id: ID for padding token
429
        eos_id: ID for end-of-sequence token  
430
        unk_id: ID for unknown token
431
        bos_id: ID for beginning-of-sequence token (-1 to disable)
432
        **kwargs: Additional SentencePiece training parameters
433
        
434
    Returns:
435
        bytes: Serialized SentencePiece model proto
436
        
437
    Example:
438
        training_data = ["Hello world", "How are you", "Fine thanks"]
439
        proto = keras_hub.tokenizers.compute_sentence_piece_proto(
440
            data=training_data,
441
            vocab_size=1000,
442
            model_type="unigram"
443
        )
444
    """
445
    ...
446

447
def compute_word_piece_vocabulary(
448
    data: list,
449
    vocab_size: int,
450
    reserved_tokens: list = None,
451
    **kwargs
452
) -> dict:
453
    """
454
    Compute a WordPiece vocabulary from training data.
455
    
456
    Creates a WordPiece vocabulary dictionary suitable for use with
457
    WordPieceTokenizer.
458
    
459
    Args:
460
        data: List of strings to train the vocabulary on
461
        vocab_size: Target vocabulary size
462
        reserved_tokens: List of tokens to reserve (e.g., ["[PAD]", "[UNK]"])
463
        **kwargs: Additional vocabulary computation parameters
464
        
465
    Returns:
466
        dict: Vocabulary mapping from token to ID
467
        
468
    Example:
469
        training_data = ["Hello world", "How are you", "Fine thanks"] 
470
        vocab = keras_hub.tokenizers.compute_word_piece_vocabulary(
471
            data=training_data,
472
            vocab_size=1000,
473
            reserved_tokens=["[PAD]", "[UNK]", "[CLS]", "[SEP]"]
474
        )
475
    """
476
    ...
477
```
478

479
### Utility Function Chaining
480

481
```python
482
import keras_hub
483
import numpy as np
484

485
def analyze_image_predictions(model, images, detailed=True):
486
    """Comprehensive analysis of image classification predictions."""
487
    
488
    # Get predictions
489
    predictions = model.predict(images)
490
    
491
    analysis = {
492
        'total_images': len(images),
493
        'model_version': keras_hub.version(),
494
        'predictions': []
495
    }
496
    
497
    for i, pred in enumerate(predictions):
498
        # Get top predictions
499
        top_preds = keras_hub.utils.decode_imagenet_predictions(pred, top=5)
500
        
501
        # Calculate prediction confidence
502
        top_prob = top_preds[0][2]
503
        entropy = -np.sum(pred * np.log(pred + 1e-10))
504
        
505
        image_analysis = {
506
            'image_index': i,
507
            'top_prediction': {
508
                'class_id': top_preds[0][0],
509
                'class_name': top_preds[0][1],
510
                'probability': float(top_prob)
511
            },
512
            'confidence_level': 'high' if top_prob > 0.8 else 'medium' if top_prob > 0.5 else 'low',
513
            'entropy': float(entropy)
514
        }
515
        
516
        if detailed:
517
            image_analysis['all_top_predictions'] = [
518
                {
519
                    'class_id': int(cid),
520
                    'class_name': cname,
521
                    'probability': float(prob)
522
                }
523
                for cid, cname, prob in top_preds
524
            ]
525
        
526
        analysis['predictions'].append(image_analysis)
527
    
528
    return analysis
529

530
# Example usage
531
model = keras_hub.models.ViTImageClassifier.from_preset("vit_base_patch16_224")
532
test_images = np.random.random((3, 224, 224, 3))
533

534
analysis = analyze_image_predictions(model, test_images, detailed=True)
535

536
print(f"Analysis using Keras Hub {analysis['model_version']}")
537
print(f"Processed {analysis['total_images']} images")
538

539
for pred in analysis['predictions']:
540
    top_pred = pred['top_prediction']
541
    print(f"\nImage {pred['image_index']}:")
542
    print(f"  Top prediction: {top_pred['class_name']} ({top_pred['probability']:.3f})")
543
    print(f"  Confidence: {pred['confidence_level']}")
544
    print(f"  Entropy: {pred['entropy']:.3f}")
545
```