0
# Image Tagging
1

2
Generate detailed tags for image content with confidence scores. The tagging service identifies objects, concepts, activities, and attributes present in images, providing a comprehensive set of keywords that describe the visual content.
3

4
## Capabilities
5

6
### Image Tag Generation
7

8
Extract comprehensive tags that describe various aspects of image content including objects, activities, settings, and visual attributes.
9

10
```python { .api }
11
def tag_image(url, language="en", model_version="latest", custom_headers=None, raw=False, **operation_config):
12
    """
13
    Generate tags for image content.
14
    
15
    Args:
16
        url (str): Publicly reachable URL of an image
17
        language (str, optional): Output language for tags.
18
            Supported: "en", "es", "ja", "pt", "zh". Default: "en"
19
        model_version (str, optional): AI model version. Default: "latest"
20
        custom_headers (dict, optional): Custom HTTP headers
21
        raw (bool, optional): Return raw response. Default: False
22
        
23
    Returns:
24
        TagResult: Generated tags with confidence scores
25
        
26
    Raises:
27
        ComputerVisionErrorResponseException: API error occurred
28
    """
29

30
def tag_image_in_stream(image, language="en", model_version="latest", custom_headers=None, raw=False, **operation_config):
31
    """
32
    Generate tags from binary image stream.
33
    
34
    Args:
35
        image (Generator): Binary image data stream
36
        language (str, optional): Output language for tags
37
        model_version (str, optional): AI model version
38
        
39
    Returns:
40
        TagResult: Generated tags with confidence scores and metadata
41
    """
42
```
43

44
## Usage Examples
45

46
### Basic Image Tagging
47

48
```python
49
from azure.cognitiveservices.vision.computervision import ComputerVisionClient
50
from msrest.authentication import CognitiveServicesCredentials
51

52
# Initialize client
53
credentials = CognitiveServicesCredentials("your-api-key")
54
client = ComputerVisionClient("https://your-endpoint.cognitiveservices.azure.com/", credentials)
55

56
# Generate tags for image
57
image_url = "https://example.com/beach-scene.jpg"
58
tag_result = client.tag_image(image_url)
59

60
print(f"Generated {len(tag_result.tags)} tags:")
61
for tag in tag_result.tags:
62
    print(f"  {tag.name}: {tag.confidence:.3f}")
63
```
64

65
### Filtering Tags by Confidence
66

67
```python
68
# Filter tags by confidence threshold
69
image_url = "https://example.com/complex-scene.jpg"
70
tag_result = client.tag_image(image_url)
71

72
# Define confidence thresholds
73
high_confidence = 0.8
74
medium_confidence = 0.5
75

76
high_conf_tags = [tag for tag in tag_result.tags if tag.confidence >= high_confidence]
77
medium_conf_tags = [tag for tag in tag_result.tags if medium_confidence <= tag.confidence < high_confidence]
78
low_conf_tags = [tag for tag in tag_result.tags if tag.confidence < medium_confidence]
79

80
print("High confidence tags:")
81
for tag in high_conf_tags:
82
    print(f"  {tag.name} ({tag.confidence:.3f})")
83

84
print(f"\nMedium confidence tags: {len(medium_conf_tags)}")
85
print(f"Low confidence tags: {len(low_conf_tags)}")
86
```
87

88
### Multilingual Tagging
89

90
```python
91
# Generate tags in different languages
92
image_url = "https://example.com/street-food.jpg"
93

94
languages = ["en", "es", "ja"]
95
all_tags = {}
96

97
for lang in languages:
98
    try:
99
        tag_result = client.tag_image(image_url, language=lang)
100
        all_tags[lang] = [(tag.name, tag.confidence) for tag in tag_result.tags[:5]]  # Top 5 tags
101
    except Exception as e:
102
        print(f"Failed to get tags in {lang}: {e}")
103

104
# Display results
105
for lang, tags in all_tags.items():
106
    print(f"\n{lang.upper()} tags:")
107
    for name, confidence in tags:
108
        print(f"  {name} ({confidence:.3f})")
109
```
110

111
### Tag Analysis and Categorization
112

113
```python
114
# Analyze tags by category/type
115
tag_result = client.tag_image(image_url)
116

117
# Categorize tags (example categorization)
118
categories = {
119
    'objects': [],
120
    'people': [],
121
    'activities': [],
122
    'settings': [],
123
    'colors': [],
124
    'other': []
125
}
126

127
# Simple categorization logic (you can expand this)
128
for tag in tag_result.tags:
129
    tag_name = tag.name.lower()
130
    if any(word in tag_name for word in ['person', 'man', 'woman', 'child', 'people']):
131
        categories['people'].append(tag)
132
    elif any(word in tag_name for word in ['red', 'blue', 'green', 'yellow', 'black', 'white']):
133
        categories['colors'].append(tag)
134
    elif any(word in tag_name for word in ['indoor', 'outdoor', 'room', 'kitchen', 'street']):
135
        categories['settings'].append(tag)
136
    elif any(word in tag_name for word in ['sitting', 'standing', 'walking', 'running', 'playing']):
137
        categories['activities'].append(tag)
138
    else:
139
        categories['objects'].append(tag)
140

141
# Display categorized results
142
for category, tags in categories.items():
143
    if tags:
144
        print(f"\n{category.upper()}:")
145
        for tag in tags:
146
            print(f"  {tag.name} ({tag.confidence:.3f})")
147
```
148

149
### Local File Tagging
150

151
```python
152
# Generate tags from local image file
153
with open("local_image.jpg", "rb") as image_stream:
154
    tag_result = client.tag_image_in_stream(image_stream)
155
    
156
    # Get top tags above threshold
157
    threshold = 0.7
158
    top_tags = [tag for tag in tag_result.tags if tag.confidence >= threshold]
159
    
160
    if top_tags:
161
        print(f"Top tags (confidence ≥ {threshold}):")
162
        for tag in top_tags:
163
            print(f"  {tag.name}: {tag.confidence:.3f}")
164
    else:
165
        print(f"No tags found above confidence threshold {threshold}")
166
        print("All tags:")
167
        for tag in tag_result.tags[:10]:  # Show top 10
168
            print(f"  {tag.name}: {tag.confidence:.3f}")
169
```
170

171
### Batch Tagging with Aggregation
172

173
```python
174
# Process multiple images and aggregate tags
175
image_urls = [
176
    "https://example.com/image1.jpg",
177
    "https://example.com/image2.jpg",
178
    "https://example.com/image3.jpg"
179
]
180

181
all_tags = {}  # Dictionary to aggregate tags across images
182

183
for i, url in enumerate(image_urls):
184
    try:
185
        tag_result = client.tag_image(url)
186
        print(f"Processed image {i+1}/{len(image_urls)}")
187
        
188
        # Aggregate tags
189
        for tag in tag_result.tags:
190
            if tag.name in all_tags:
191
                all_tags[tag.name].append(tag.confidence)
192
            else:
193
                all_tags[tag.name] = [tag.confidence]
194
                
195
    except Exception as e:
196
        print(f"Error processing {url}: {e}")
197

198
# Calculate average confidence for each tag
199
tag_averages = {
200
    tag_name: sum(confidences) / len(confidences)
201
    for tag_name, confidences in all_tags.items()
202
}
203

204
# Sort by average confidence and frequency
205
popular_tags = sorted(
206
    [(name, avg_conf, len(all_tags[name])) for name, avg_conf in tag_averages.items()],
207
    key=lambda x: (x[2], x[1]),  # Sort by frequency, then confidence
208
    reverse=True
209
)
210

211
print("\nMost common tags across all images:")
212
for tag_name, avg_confidence, frequency in popular_tags[:10]:
213
    print(f"  {tag_name}: appears in {frequency} images, avg confidence {avg_confidence:.3f}")
214
```
215

216
### Tag Hints Analysis
217

218
```python
219
# Analyze tag hints for additional context
220
tag_result = client.tag_image(image_url)
221

222
print("Tags with hints:")
223
for tag in tag_result.tags:
224
    if hasattr(tag, 'hint') and tag.hint:
225
        print(f"  {tag.name} ({tag.confidence:.3f}) - Hint: {tag.hint}")
226
    else:
227
        print(f"  {tag.name} ({tag.confidence:.3f})")
228
```
229

230
## Response Data Types
231

232
### TagResult
233

234
```python { .api }
235
class TagResult:
236
    """
237
    Image tagging operation result.
238
    
239
    Attributes:
240
        tags (list[ImageTag]): Generated tags with confidence scores
241
        request_id (str): Request identifier
242
        metadata (ImageMetadata): Image metadata (dimensions, format)
243
        model_version (str): AI model version used for tagging
244
    """
245
```
246

247
### ImageTag
248

249
```python { .api }
250
class ImageTag:
251
    """
252
    Individual image tag with confidence score.
253
    
254
    Attributes:
255
        name (str): Tag name/label describing image content
256
        confidence (float): Confidence score for the tag (0.0 to 1.0)
257
        hint (str, optional): Additional context or hint about the tag
258
    """
259
```
260

261
### ImageMetadata
262

263
```python { .api }
264
class ImageMetadata:
265
    """
266
    Image metadata information.
267
    
268
    Attributes:
269
        height (int): Image height in pixels
270
        width (int): Image width in pixels
271
        format (str): Image format (e.g., "Jpeg", "Png")
272
    """
273
```
274

275
## Tag Categories
276

277
The tagging service identifies various types of content:
278

279
### Objects and Items
280
- Physical objects (car, building, tree, computer, phone)
281
- Food items (pizza, apple, coffee, sandwich)
282
- Clothing (shirt, hat, shoes, jacket)
283
- Furniture (chair, table, bed, couch)
284

285
### People and Body Parts
286
- People descriptors (person, man, woman, child)
287
- Body parts (face, hand, eye, hair)
288
- Demographics (adult, young, elderly)
289

290
### Activities and Actions
291
- Actions (sitting, standing, walking, running, eating)
292
- Activities (playing, working, cooking, reading)
293
- Sports (tennis, soccer, swimming, cycling)
294

295
### Settings and Locations
296
- Indoor/outdoor classification
297
- Specific locations (kitchen, office, park, street)
298
- Environments (urban, rural, natural, architectural)
299

300
### Visual Attributes
301
- Colors (red, blue, colorful, black and white)
302
- Lighting (bright, dark, sunny, shadowy)
303
- Composition (close-up, wide shot, portrait)
304
- Style (modern, vintage, artistic)
305

306
### Abstract Concepts
307
- Emotions and mood (happy, peaceful, busy)
308
- Qualities (beautiful, interesting, professional)
309
- Concepts (transportation, technology, nature)
310

311
## Tag Quality and Confidence
312

313
### Confidence Score Interpretation
314

315
- **0.9-1.0**: Extremely confident - tag is almost certainly present
316
- **0.8-0.9**: High confidence - tag is very likely accurate
317
- **0.7-0.8**: Good confidence - tag is probably correct
318
- **0.5-0.7**: Moderate confidence - tag may be present but uncertain
319
- **Below 0.5**: Low confidence - tag presence is questionable
320

321
### Best Practices
322

323
- Use confidence thresholds appropriate for your application (typically 0.5-0.8)
324
- Consider the number of tags returned (usually 10-50 per image)
325
- Combine with other analysis features for comprehensive understanding
326
- Use multilingual support when working with international content
327
- Aggregate results across similar images for better accuracy