0
# Configuration Management
1

2
Configuration handling including validation, migration, and URL encoding for search prompts to ensure proper API integration. The JinaAiReaderConfigMigration class provides runtime configuration transformation capabilities.
3

4
## Capabilities
5

6
### Configuration Migration
7

8
Handles runtime migration of configuration values, specifically URL encoding of search prompts for proper API integration.
9

10
```python { .api }
11
class JinaAiReaderConfigMigration:
12
    """
13
    Handles runtime configuration migration for the Jina AI Reader connector.
14
    
15
    Primary purpose is to ensure search_prompt values are properly URL-encoded
16
    for API consumption while maintaining backward compatibility.
17
    """
18
    
19
    message_repository: MessageRepository = InMemoryMessageRepository()
20
```
21

22
### Migration Detection
23

24
Determines whether configuration migration is needed for a given config.
25

26
```python { .api }
27
@classmethod
28
def should_migrate(cls, config: Mapping[str, Any]) -> bool:
29
    """
30
    Determines if configuration migration is required.
31
    
32
    Args:
33
        config (Mapping[str, Any]): Configuration dictionary containing connector settings
34
        
35
    Returns:
36
        bool: True if search_prompt needs URL encoding, False otherwise
37
        
38
    The method checks if the search_prompt parameter is properly URL-encoded.
39
    If the search_prompt is not encoded, migration is required.
40
    """
41
```
42

43
### URL Encoding Validation
44

45
Validates whether a string is properly URL-encoded by comparing it with its unquoted version.
46

47
```python { .api }
48
@classmethod  
49
def is_url_encoded(cls, s: str) -> bool:
50
    """
51
    Check if a string is URL-encoded by comparing with its unquoted version.
52
    
53
    Args:
54
        s (str): String to validate for URL encoding
55
        
56
    Returns:
57
        bool: True if string is not URL-encoded (needs encoding), False if already encoded
58
        
59
    Implementation uses urllib.parse.unquote to determine if the string has been URL-encoded.
60
    Returns True when the string equals its unquoted version (meaning it needs encoding).
61
    This method is used internally by should_migrate() to determine migration necessity.
62
    """
63
```
64

65
### Configuration Transformation
66

67
Applies necessary transformations to configuration values.
68

69
```python { .api }
70
@classmethod
71
def modify(cls, config: Mapping[str, Any]) -> Mapping[str, Any]:
72
    """
73
    Apply configuration modifications, specifically URL-encoding the search_prompt.
74
    
75
    Args:
76
        config (Mapping[str, Any]): Original configuration dictionary
77
        
78
    Returns:
79
        Mapping[str, Any]: Modified configuration with URL-encoded search_prompt
80
        
81
    Raises:
82
        ValueError: If configuration is invalid or malformed
83
        
84
    If search_prompt exists in config and needs encoding, applies urllib.parse.quote.
85
    """
86
```
87

88
### Configuration Persistence
89

90
Modifies configuration and saves it to file.
91

92
```python { .api }
93
@classmethod
94
def modify_and_save(cls, config_path: str, source: Source, config: Mapping[str, Any]) -> Mapping[str, Any]:
95
    """
96
    Modify configuration and save to specified path.
97
    
98
    Args:
99
        config_path (str): Path to configuration file
100
        source (Source): Airbyte source instance for file operations
101
        config (Mapping[str, Any]): Configuration to modify and save
102
        
103
    Returns:
104
        Mapping[str, Any]: The modified configuration
105
        
106
    Applies modifications via modify() method then saves using source.write_config().
107
    """
108
```
109

110
### Control Message Emission
111

112
Emits Airbyte control messages for configuration changes.
113

114
```python { .api }
115
@classmethod
116
def emit_control_message(cls, migrated_config: Mapping[str, Any]) -> None:
117
    """
118
    Emit Airbyte control message for configuration changes.
119
    
120
    Args:
121
        migrated_config (Mapping[str, Any]): The migrated configuration
122
        
123
    Creates and emits an Airbyte connector config control message to notify
124
    the Airbyte platform about configuration changes. Messages are queued
125
    and printed to stdout for platform consumption.
126
    """
127
```
128

129
### Migration Orchestration
130

131
Main migration orchestration method that handles the complete migration workflow.
132

133
```python { .api }
134
@classmethod
135
def migrate(cls, args: List[str], source: Source) -> None:
136
    """
137
    Main migration orchestration method.
138
    
139
    Args:
140
        args (List[str]): Command-line arguments from sys.argv
141
        source (Source): Airbyte source instance
142
        
143
    Orchestrates the complete migration process:
144
    1. Extracts config path from command-line arguments
145
    2. Reads existing configuration if --config argument provided
146
    3. Checks if migration is needed via should_migrate()
147
    4. Applies modifications and saves config via modify_and_save()
148
    5. Emits control message via emit_control_message()
149
    
150
    Only performs migration when --config argument is provided and migration is needed.
151
    """
152
```
153

154
## Configuration Schema
155

156
The connector accepts the following configuration parameters:
157

158
```python { .api }
159
class ConfigSpec(TypedDict):
160
    """Configuration specification for Jina AI Reader connector."""
161
    api_key: str  # Optional API key for authentication (marked as secret)
162
    read_prompt: str  # URL to read content from (default: "https://www.google.com") 
163
    search_prompt: str  # URL-encoded search query (default: "Search%20airbyte")
164
    gather_links: bool  # Include links summary section (optional)
165
    gather_images: bool  # Include images summary section (optional)
166
```
167

168
## Usage Examples
169

170
### Basic Configuration
171

172
```python
173
config = {
174
    "api_key": "jina_your_api_key_here",
175
    "read_prompt": "https://example.com/article",
176
    "search_prompt": "machine%20learning%20news",
177
    "gather_links": True,
178
    "gather_images": False
179
}
180
```
181

182
### Migration Example
183

184
```python
185
from source_jina_ai_reader.config_migration import JinaAiReaderConfigMigration
186

187
# Check if migration needed
188
config = {"search_prompt": "AI news"}  # Not URL encoded
189
needs_migration = JinaAiReaderConfigMigration.should_migrate(config)  # True
190

191
# Apply migration
192
migrated_config = JinaAiReaderConfigMigration.modify(config)
193
# Result: {"search_prompt": "AI%20news"}  # URL encoded
194
```
195

196
## Error Handling
197

198
- **ValueError**: Raised when configuration is malformed, invalid, or when search_prompt is already URL-encoded but migration is attempted
199
- **Missing search_prompt**: If search_prompt key is missing from config, migration is skipped
200
- **Validation**: URL encoding validation prevents API request failures by ensuring proper encoding
201
- **Backward Compatibility**: Migration ensures existing configs continue working with newer versions
202
- **Control Messages**: Proper Airbyte protocol compliance for config changes via emit_control_message()
203
- **File Operations**: write_config() handles file writing errors gracefully through Airbyte CDK
204
- **Command Line Parsing**: migrate() method handles missing --config argument by checking if config_path exists