0
# Prediction Operations
1

2
Inference operations for trained Azure Custom Vision models. The prediction client provides functionality for making predictions on images using published custom vision models, supporting both image classification and object detection scenarios.
3

4
## Capabilities
5

6
### Client Initialization
7

8
Create and configure the prediction client with authentication credentials.
9

10
```python { .api }
11
class CustomVisionPredictionClient:
12
    def __init__(self, endpoint: str, credentials):
13
        """
14
        Initialize the Custom Vision Prediction Client.
15

16
        Parameters:
17
        - endpoint: str, Custom Vision prediction endpoint URL
18
        - credentials: Authentication credentials object
19
        """
20

21
class CustomVisionPredictionClientConfiguration:
22
    """Prediction client configuration settings."""
23
```
24

25
### Image Classification
26

27
Classify images using trained classification models with options for storing prediction results.
28

29
```python { .api }
30
def classify_image(project_id: str, published_name: str, image_data: bytes, application: str = None) -> ImagePrediction:
31
    """
32
    Classify image and store prediction results.
33

34
    Parameters:
35
    - project_id: str, project identifier containing the model
36
    - published_name: str, name of published model iteration
37
    - image_data: bytes, image binary data to classify
38
    - application: str, optional application identifier for tracking
39

40
    Returns:
41
    ImagePrediction: Classification results with tag predictions and confidence scores
42
    """
43

44
def classify_image_with_no_store(project_id: str, published_name: str, image_data: bytes, 
45
                                application: str = None) -> ImagePrediction:
46
    """
47
    Classify image without storing prediction results.
48

49
    Parameters:
50
    - project_id: str, project identifier containing the model
51
    - published_name: str, name of published model iteration
52
    - image_data: bytes, image binary data to classify
53
    - application: str, optional application identifier for tracking
54

55
    Returns:
56
    ImagePrediction: Classification results without storage
57
    """
58

59
def classify_image_url(project_id: str, published_name: str, image_url: ImageUrl, 
60
                      application: str = None) -> ImagePrediction:
61
    """
62
    Classify image from URL and store prediction results.
63

64
    Parameters:
65
    - project_id: str, project identifier containing the model
66
    - published_name: str, name of published model iteration
67
    - image_url: ImageUrl, wrapper object containing image URL
68
    - application: str, optional application identifier for tracking
69

70
    Returns:
71
    ImagePrediction: Classification results with storage
72
    """
73

74
def classify_image_url_with_no_store(project_id: str, published_name: str, image_url: ImageUrl,
75
                                    application: str = None) -> ImagePrediction:
76
    """
77
    Classify image from URL without storing prediction results.
78

79
    Parameters:
80
    - project_id: str, project identifier containing the model
81
    - published_name: str, name of published model iteration
82
    - image_url: ImageUrl, wrapper object containing image URL
83
    - application: str, optional application identifier for tracking
84

85
    Returns:
86
    ImagePrediction: Classification results without storage
87
    """
88
```
89

90
### Object Detection
91

92
Detect objects in images using trained object detection models with bounding box predictions.
93

94
```python { .api }
95
def detect_image(project_id: str, published_name: str, image_data: bytes, application: str = None) -> ImagePrediction:
96
    """
97
    Detect objects in image and store prediction results.
98

99
    Parameters:
100
    - project_id: str, project identifier containing the model
101
    - published_name: str, name of published model iteration
102
    - image_data: bytes, image binary data for object detection
103
    - application: str, optional application identifier for tracking
104

105
    Returns:
106
    ImagePrediction: Object detection results with bounding boxes and confidence scores
107
    """
108

109
def detect_image_with_no_store(project_id: str, published_name: str, image_data: bytes,
110
                              application: str = None) -> ImagePrediction:
111
    """
112
    Detect objects in image without storing prediction results.
113

114
    Parameters:
115
    - project_id: str, project identifier containing the model
116
    - published_name: str, name of published model iteration
117
    - image_data: bytes, image binary data for object detection
118
    - application: str, optional application identifier for tracking
119

120
    Returns:
121
    ImagePrediction: Object detection results without storage
122
    """
123

124
def detect_image_url(project_id: str, published_name: str, image_url: ImageUrl,
125
                    application: str = None) -> ImagePrediction:
126
    """
127
    Detect objects in image from URL and store prediction results.
128

129
    Parameters:
130
    - project_id: str, project identifier containing the model
131
    - published_name: str, name of published model iteration
132
    - image_url: ImageUrl, wrapper object containing image URL
133
    - application: str, optional application identifier for tracking
134

135
    Returns:
136
    ImagePrediction: Object detection results with storage
137
    """
138

139
def detect_image_url_with_no_store(project_id: str, published_name: str, image_url: ImageUrl,
140
                                  application: str = None) -> ImagePrediction:
141
    """
142
    Detect objects in image from URL without storing prediction results.
143

144
    Parameters:
145
    - project_id: str, project identifier containing the model
146
    - published_name: str, name of published model iteration
147
    - image_url: ImageUrl, wrapper object containing image URL
148
    - application: str, optional application identifier for tracking
149

150
    Returns:
151
    ImagePrediction: Object detection results without storage
152
    """
153
```
154

155
## Usage Examples
156

157
### Classification Example
158

159
```python
160
from azure.cognitiveservices.vision.customvision.prediction import CustomVisionPredictionClient
161
from azure.cognitiveservices.vision.customvision.prediction.models import ImageUrl
162
from msrest.authentication import ApiKeyCredentials
163

164
# Initialize client
165
prediction_key = "your-prediction-key"
166
endpoint = "https://southcentralus.api.cognitive.microsoft.com"
167
credentials = ApiKeyCredentials(in_headers={"Prediction-key": prediction_key})
168
client = CustomVisionPredictionClient(endpoint, credentials)
169

170
# Classify image from file
171
with open("image.jpg", "rb") as image_data:
172
    results = client.classify_image(
173
        project_id="your-project-id",
174
        published_name="your-model-name", 
175
        image_data=image_data.read()
176
    )
177

178
# Process results
179
for prediction in results.predictions:
180
    if prediction.probability > 0.5:
181
        print(f"Tag: {prediction.tag_name}, Confidence: {prediction.probability:.2%}")
182

183
# Classify image from URL
184
image_url = ImageUrl(url="https://example.com/image.jpg")
185
results = client.classify_image_url(
186
    project_id="your-project-id",
187
    published_name="your-model-name",
188
    image_url=image_url
189
)
190
```
191

192
### Object Detection Example
193

194
```python
195
# Detect objects in image
196
with open("image.jpg", "rb") as image_data:
197
    results = client.detect_image(
198
        project_id="your-project-id",
199
        published_name="your-detector-name",
200
        image_data=image_data.read()
201
    )
202

203
# Process detection results
204
for prediction in results.predictions:
205
    if prediction.probability > 0.3:
206
        bbox = prediction.bounding_box
207
        print(f"Object: {prediction.tag_name}")
208
        print(f"Confidence: {prediction.probability:.2%}")
209
        print(f"Bounding Box: ({bbox.left:.3f}, {bbox.top:.3f}, {bbox.width:.3f}, {bbox.height:.3f})")
210
```
211

212
## Prediction Data Types
213

214
```python { .api }
215
class ImagePrediction:
216
    """Main prediction result containing predictions and metadata."""
217
    id: str
218
    project: str
219
    iteration: str
220
    created: datetime
221
    predictions: list  # list[Prediction]
222

223
class Prediction:
224
    """Individual prediction with tag, probability, and optional bounding box."""
225
    probability: float
226
    tag_id: str
227
    tag_name: str
228
    bounding_box: BoundingBox = None  # Only for object detection
229

230
class BoundingBox:
231
    """Image region coordinates for object detection (normalized 0-1 values)."""
232
    left: float    # Left edge as fraction of image width
233
    top: float     # Top edge as fraction of image height  
234
    width: float   # Width as fraction of image width
235
    height: float  # Height as fraction of image height
236

237
class ImageUrl:
238
    """URL wrapper for image predictions."""
239
    url: str
240

241
class CustomVisionErrorException(Exception):
242
    """Exception raised for prediction API operation failures."""
243
    error: CustomVisionError
244

245
class CustomVisionError:
246
    """Detailed error information for prediction failures."""
247
    code: str
248
    message: str
249
```
250

251
## Error Handling
252

253
```python
254
from azure.cognitiveservices.vision.customvision.prediction.models import CustomVisionErrorException
255

256
try:
257
    results = client.classify_image(project_id, published_name, image_data)
258
except CustomVisionErrorException as e:
259
    print(f"Prediction failed: {e.error.code} - {e.error.message}")
260
    # Common error codes:
261
    # - BadRequest: Invalid image format or size
262
    # - NotFound: Project or published model not found  
263
    # - Unauthorized: Invalid prediction key
264
    # - QuotaExceeded: Prediction quota exceeded
265
```
266

267
## Best Practices
268

269
### Performance Optimization
270
- Use `*_with_no_store` methods for high-volume inference to avoid storage costs
271
- Batch predictions when possible to reduce API calls
272
- Consider image size optimization (max 4MB, recommended under 1MB)
273

274
### Confidence Thresholds
275
- Classification: Typically filter results above 0.5-0.7 confidence
276
- Object Detection: Use lower thresholds (0.3-0.5) to avoid missing objects
277
- Adjust thresholds based on your specific use case and model performance
278

279
### URL vs Binary Data
280
- Use URL methods for images already hosted online
281
- Use binary data methods for local files or when you need full control over image data
282
- URLs must be publicly accessible (no authentication required)