0
# Source Connector
1
2
The primary source connector functionality for creating and managing the HubSpot data integration. Provides connection checking, stream discovery, authentication scope validation, and custom object stream generation.
3
4
## Capabilities
5
6
### Connector Initialization
7
8
Creates a new SourceHubspot instance with configuration, catalog, and state management.
9
10
```python { .api }
11
class SourceHubspot(YamlDeclarativeSource):
12
def __init__(
13
catalog: Optional[ConfiguredAirbyteCatalog],
14
config: Optional[Mapping[str, Any]],
15
state: TState,
16
**kwargs
17
):
18
"""
19
Initialize the HubSpot source connector.
20
21
Parameters:
22
- catalog: Configured Airbyte catalog with stream selections
23
- config: Source configuration including credentials and settings
24
- state: Current synchronization state for incremental streams
25
- **kwargs: Additional keyword arguments
26
"""
27
```
28
29
### Connection Validation
30
31
Validates connectivity and authentication to the HubSpot API.
32
33
```python { .api }
34
def check_connection(self, logger: logging.Logger, config: Mapping[str, Any]) -> Tuple[bool, Optional[Any]]:
35
"""
36
Check connection to HubSpot API.
37
38
Parameters:
39
- logger: Logger instance for connection checking
40
- config: Configuration with credentials and settings
41
42
Returns:
43
- Tuple of (is_healthy: bool, error_message: Optional[Any])
44
"""
45
```
46
47
### Stream Discovery
48
49
Discovers and returns available streams based on configuration and OAuth scopes.
50
51
```python { .api }
52
def streams(self, config: Mapping[str, Any]) -> List[Stream]:
53
"""
54
Get list of available streams based on configuration and granted scopes.
55
56
Parameters:
57
- config: Source configuration including credentials and settings
58
59
Returns:
60
- List of available Stream instances
61
"""
62
```
63
64
### Scope Management
65
66
Retrieves granted OAuth scopes to determine stream availability.
67
68
```python { .api }
69
def get_granted_scopes(self, authenticator) -> List[str]:
70
"""
71
Get OAuth scopes granted to the application.
72
73
Parameters:
74
- authenticator: OAuth authenticator instance
75
76
Returns:
77
- List of granted scope strings
78
"""
79
```
80
81
### API Factory
82
83
Creates API client instances with proper configuration.
84
85
```python { .api }
86
@staticmethod
87
def get_api(config: Mapping[str, Any]) -> API:
88
"""
89
Factory method to create API client instance.
90
91
Parameters:
92
- config: Configuration with credentials
93
94
Returns:
95
- Configured API client instance
96
"""
97
```
98
99
### Common Parameters
100
101
Extracts and standardizes common parameters for stream initialization.
102
103
```python { .api }
104
def get_common_params(self, config) -> Mapping[str, Any]:
105
"""
106
Extract common parameters for stream initialization.
107
108
Parameters:
109
- config: Source configuration
110
111
Returns:
112
- Dictionary of common parameters (api, start_date, credentials, etc.)
113
"""
114
```
115
116
### Custom Object Stream Generation
117
118
Generates stream instances for HubSpot custom objects.
119
120
```python { .api }
121
def get_custom_object_streams(self, api: API, common_params: Mapping[str, Any]) -> Generator[CustomObject, None, None]:
122
"""
123
Generate custom object stream instances.
124
125
Parameters:
126
- api: API client instance
127
- common_params: Common parameters for stream initialization
128
129
Yields:
130
- CustomObject stream instances for each custom object type
131
"""
132
```
133
134
### Web Analytics Custom Object Streams
135
136
Generates web analytics stream instances for custom objects (experimental feature).
137
138
```python { .api }
139
def get_web_analytics_custom_objects_stream(
140
self,
141
custom_object_stream_instances: List[CustomObject],
142
common_params: Any
143
) -> Generator[WebAnalyticsStream, None, None]:
144
"""
145
Generate web analytics streams for custom objects.
146
147
Parameters:
148
- custom_object_stream_instances: List of custom object stream instances
149
- common_params: Common parameters for stream initialization
150
151
Yields:
152
- WebAnalyticsStream instances for each custom object
153
"""
154
```
155
156
## Usage Examples
157
158
### Basic Connector Setup
159
160
```python
161
from source_hubspot import SourceHubspot
162
import logging
163
164
# Configure OAuth credentials
165
config = {
166
"credentials": {
167
"credentials_title": "OAuth Credentials",
168
"client_id": "your_client_id",
169
"client_secret": "your_client_secret",
170
"refresh_token": "your_refresh_token"
171
},
172
"start_date": "2023-01-01T00:00:00Z"
173
}
174
175
# Create connector instance
176
source = SourceHubspot(catalog=None, config=config, state=None)
177
178
# Test connection
179
logger = logging.getLogger("test")
180
is_healthy, error = source.check_connection(logger, config)
181
182
if is_healthy:
183
print("Connection successful!")
184
# Get available streams
185
streams = source.streams(config)
186
print(f"Available streams: {[stream.name for stream in streams]}")
187
else:
188
print(f"Connection failed: {error}")
189
```
190
191
### Private App Authentication
192
193
```python
194
# Configure Private App credentials
195
config = {
196
"credentials": {
197
"credentials_title": "Private App Credentials",
198
"access_token": "your_private_app_token"
199
},
200
"start_date": "2023-01-01T00:00:00Z"
201
}
202
203
source = SourceHubspot(catalog=None, config=config, state=None)
204
streams = source.streams(config)
205
```
206
207
### With Experimental Streams
208
209
```python
210
config = {
211
"credentials": {
212
"credentials_title": "OAuth Credentials",
213
"client_id": "your_client_id",
214
"client_secret": "your_client_secret",
215
"refresh_token": "your_refresh_token"
216
},
217
"start_date": "2023-01-01T00:00:00Z",
218
"enable_experimental_streams": True
219
}
220
221
source = SourceHubspot(catalog=None, config=config, state=None)
222
streams = source.streams(config)
223
224
# Will include web analytics streams if experimental streams are enabled
225
web_analytics_streams = [s for s in streams if "WebAnalytics" in s.__class__.__name__]
226
```
227
228
## Constants
229
230
```python { .api }
231
DEFAULT_START_DATE = "2006-06-01T00:00:00Z" # Default start date if none provided
232
233
# OAuth scope requirements for each stream type
234
scopes: Dict[str, Set[str]] # Stream name to required scopes mapping
235
properties_scopes: Dict[str, Set[str]] # Property-specific scope requirements
236
```