0
# Custom Objects & Dynamic Schemas
1

2
Support for HubSpot custom objects with runtime schema discovery, automatic property mapping, and dynamic stream generation for extensible data models.
3

4
## Capabilities
5

6
### Dynamic Stream Generation
7

8
Custom object streams are generated dynamically based on HubSpot schema definitions discovered at runtime.
9

10
```yaml { .api }
11
custom_object_streams:
12
  type: DynamicDeclarativeStream
13
  stream_template:
14
    type: StateDelegatingStream
15
    name: "{{ name }}"  # Generated from custom object schema
16
    json_schema:
17
      $schema: "http://json-schema.org/draft-07/schema#"
18
      type: ["null", "object"]
19
      additionalProperties: true
20
      properties:
21
        id:
22
          type: ["null", "string"]
23
        createdAt:
24
          type: ["null", "string"]
25
          format: "date-time"
26
        updatedAt:
27
          type: ["null", "string"]
28
          format: "date-time"
29
        archived:
30
          type: ["null", "boolean"]
31
        properties:
32
          type: ["null", "object"]
33
          # Dynamic properties based on custom object schema
34
        
35
  components_resolver:
36
    type: HttpComponentsResolver
37
    retriever:
38
      path: "/crm/v3/schemas"
39
```
40

41
**Generated Stream Names:**
42
- Custom object schema name becomes stream name
43
- Follows pattern: `{custom_object_name}` (e.g., `vehicles`, `projects`, `invoices`)
44

45
### Custom Object Schema Discovery
46

47
Runtime schema discovery process for custom objects.
48

49
```yaml { .api }
50
schema_discovery_process:
51
  1_fetch_schemas:
52
    path: "/crm/v3/schemas"
53
    description: "Retrieve all custom object schemas"
54
    
55
  2_process_properties:
56
    source: "properties array from schema response"
57
    transformation: "Convert HubSpot property definitions to JSON schema"
58
    
59
  3_generate_streams:
60
    process: "Create stream definition for each custom object"
61
    naming: "Use custom object name as stream name"
62
    
63
  4_apply_configuration:
64
    incremental_sync: true
65
    cursor_field: "updatedAt"
66
    primary_key: ["id"]
67
```
68

69
### Custom Object Schema Structure
70

71
```yaml { .api }
72
CustomObjectSchema:
73
  type: object
74
  properties:
75
    id:
76
      type: string
77
      description: "Custom object schema identifier"
78
    name:
79
      type: string
80
      description: "Custom object name (used as stream name)"
81
    labels:
82
      type: object
83
      properties:
84
        singular:
85
          type: string
86
          description: "Singular label"
87
        plural:
88
          type: string
89
          description: "Plural label"
90
    primaryDisplayProperty:
91
      type: string
92
      description: "Primary display property name"
93
    secondaryDisplayProperties:
94
      type: array
95
      items:
96
        type: string
97
      description: "Secondary display properties"
98
    searchableProperties:
99
      type: array
100
      items:
101
        type: string
102
      description: "Properties that can be searched"
103
    properties:
104
      type: array
105
      items:
106
        $ref: "#/definitions/CustomObjectProperty"
107
      description: "Custom object property definitions"
108
    associatedObjects:
109
      type: array
110
      items:
111
        type: string
112
      description: "Object types this custom object can associate with"
113

114
CustomObjectProperty:
115
  type: object
116
  properties:
117
    name:
118
      type: string
119
      description: "Property name"
120
    label:
121
      type: string
122
      description: "Property display label"
123
    type:
124
      type: string
125
      enum: ["string", "number", "boolean", "datetime", "date", "enumeration"]
126
      description: "Property data type"
127
    fieldType:
128
      type: string
129
      description: "UI field type"
130
    description:
131
      type: string
132
      description: "Property description"
133
    options:
134
      type: array
135
      items:
136
        type: object
137
        properties:
138
          label:
139
            type: string
140
          value:
141
            type: string
142
      description: "Options for enumeration properties"
143
    hasUniqueValue:
144
      type: boolean
145
      description: "Whether property values must be unique"
146
    hidden:
147
      type: boolean
148
      description: "Whether property is hidden in UI"
149
    displayOrder:
150
      type: integer
151
      description: "Property display order"
152
```
153

154
### Schema Loader Implementation
155

156
```yaml { .api }
157
HubspotCustomObjectsSchemaLoader:
158
  type: SchemaLoader
159
  
160
  schema_generation_process:
161
    1_extract_properties:
162
      source: "parameters.schema_properties"
163
      description: "Property definitions injected by HttpComponentsResolver"
164
      
165
    2_map_types:
166
      string_types: ["string", "enumeration", "phone_number", "object_coordinates", "json"]
167
      datetime_types: ["datetime", "date-time"]
168
      date_types: ["date"]
169
      number_types: ["number"]
170
      boolean_types: ["boolean", "bool"]
171
      
172
    3_generate_schema:
173
      base_properties:
174
        - id: string identifier
175
        - createdAt: datetime
176
        - updatedAt: datetime  
177
        - archived: boolean
178
      dynamic_properties:
179
        - properties: nested object with custom properties
180
        - properties_{name}: flattened custom properties
181
        
182
    4_apply_nullability:
183
      all_types: ["null", "{actual_type}"]
184
      description: "All properties nullable for flexibility"
185
```
186

187
### Property Type Mapping
188

189
```yaml { .api }
190
type_mapping:
191
  hubspot_to_json_schema:
192
    string: 
193
      json_type: ["null", "string"]
194
      examples: ["text", "textarea", "select"]
195
      
196
    enumeration:
197
      json_type: ["null", "string"] 
198
      description: "Dropdown/select options as string values"
199
      
200
    phone_number:
201
      json_type: ["null", "string"]
202
      format: "phone number string"
203
      
204
    object_coordinates:
205
      json_type: ["null", "string"]
206
      description: "Geographic coordinates as string"
207
      
208
    json:
209
      json_type: ["null", "string"]
210
      description: "JSON data stored as string"
211
      
212
    datetime:
213
      json_type: ["null", "string"]
214
      format: "date-time"
215
      
216
    date:
217
      json_type: ["null", "string"]
218
      format: "date"
219
      
220
    number:
221
      json_type: ["null", "number"]
222
      description: "Numeric values"
223
      
224
    boolean:
225
      json_type: ["null", "boolean"]
226
      description: "True/false values"
227
      
228
    unknown_types:
229
      json_type: ["null", "string"]
230
      fallback: "Cast unknown types to string with warning"
231
```
232

233
### Custom Object Record Structure
234

235
```yaml { .api }
236
CustomObjectRecord:
237
  type: object
238
  properties:
239
    # Standard fields (all custom objects)
240
    id:
241
      type: string
242
      description: "Unique custom object record identifier"
243
    createdAt:
244
      type: string
245
      format: date-time
246
      description: "Record creation timestamp"
247
    updatedAt:
248
      type: string
249
      format: date-time
250
      description: "Last update timestamp"
251
    archived:
252
      type: boolean
253
      description: "Whether record is archived"
254
      
255
    # Dynamic custom properties (nested)
256
    properties:
257
      type: object
258
      additionalProperties: true
259
      description: "Custom object properties as nested object"
260
      
261
    # Dynamic custom properties (flattened)
262
    # Format: properties_{property_name}
263
    # Example: properties_vehicle_make, properties_project_status
264
```
265

266
**Example Custom Object Record:**
267
```json
268
{
269
  "id": "12345",
270
  "createdAt": "2024-01-15T10:30:00Z",
271
  "updatedAt": "2024-01-20T14:45:00Z", 
272
  "archived": false,
273
  "properties": {
274
    "vehicle_make": "Toyota",
275
    "vehicle_model": "Camry",
276
    "year": "2023",
277
    "color": "Blue",
278
    "price": "25000"
279
  },
280
  "properties_vehicle_make": "Toyota",
281
  "properties_vehicle_model": "Camry",
282
  "properties_year": "2023",
283
  "properties_color": "Blue",
284
  "properties_price": "25000"
285
}
286
```
287

288
### Stream Configuration Templates
289

290
```yaml { .api }
291
custom_object_stream_template:
292
  primary_key: ["id"]
293
  cursor_field: "updatedAt"
294
  sync_mode: incremental
295
  
296
  retriever:
297
    url_base: "https://api.hubapi.com"
298
    path: "/crm/v3/objects/{{ name }}"
299
    http_method: "GET"
300
    
301
  paginator:
302
    type: "DefaultPaginator"
303
    page_size: 100
304
    pagination_strategy:
305
      type: "CursorPagination"
306
      page_size: 100
307
      cursor_value: "{{ response.paging.next.after }}"
308
      
309
  incremental:
310
    type: "DatetimeBasedCursor"
311
    cursor_field: "updatedAt"
312
    datetime_format: "%Y-%m-%dT%H:%M:%S.%fZ"
313
    start_datetime:
314
      datetime: "{{ config['start_date'] }}"
315
      datetime_format: "%Y-%m-%dT%H:%M:%SZ"
316
```
317

318
### Association Support
319

320
Custom objects support associations with standard and other custom objects.
321

322
```yaml { .api }
323
custom_object_associations:
324
  supported_associations:
325
    - contacts
326
    - companies  
327
    - deals
328
    - tickets
329
    - other_custom_objects
330
    
331
  association_retrieval:
332
    path: "/crm/v4/associations/{{ custom_object_name }}/{{ association_type }}/batch/read"
333
    method: "POST"
334
    body:
335
      inputs: "{{ record_ids }}"
336
      
337
  association_flattening:
338
    format: "Array of associated object IDs"
339
    field_naming: "{{ association_type }}" 
340
    example:
341
      contacts: ["101", "102"]
342
      companies: ["201"]
343
      deals: ["301", "302"]
344
```
345

346
### Runtime Stream Registration
347

348
```yaml { .api }
349
dynamic_stream_registration:
350
  discovery_phase:
351
    1_fetch_schemas: "GET /crm/v3/schemas"
352
    2_filter_active: "Only include active custom object schemas"
353
    3_generate_streams: "Create stream definition for each schema"
354
    
355
  stream_registration:
356
    stream_name: "{{ schema.name }}"
357
    stream_class: "DynamicDeclarativeStream"
358
    schema_loader: "HubspotCustomObjectsSchemaLoader"
359
    
360
  configuration_injection:
361
    schema_properties: "Injected via HttpComponentsResolver"
362
    stream_parameters: "Include custom object schema metadata"
363
```
364

365
### Error Handling
366

367
```yaml { .api }
368
error_handling:
369
  schema_discovery_failures:
370
    - Log warning if schema endpoint unavailable
371
    - Continue with standard streams only
372
    - Retry schema discovery on subsequent syncs
373
    
374
  property_type_errors:
375
    - Cast unknown property types to string
376
    - Log warning for unrecognized types
377
    - Continue processing with fallback type
378
    
379
  association_failures:
380
    - Log warning if associations unavailable
381
    - Continue sync without association data
382
    - Preserve primary custom object data
383
```
384

385
### Performance Considerations
386

387
```yaml { .api }
388
performance_optimization:
389
  schema_caching:
390
    - Cache custom object schemas between syncs
391
    - Refresh schema cache on stream discovery
392
    - Minimize API calls during runtime
393
    
394
  property_chunking:
395
    - Handle large custom property lists
396
    - Chunk properties to stay within URL limits
397
    - Maintain property request consistency
398
    
399
  concurrent_processing:
400
    - Process multiple custom object streams in parallel
401
    - Respect rate limits across all streams
402
    - Balance throughput with API constraints
403
```
404

405
### Usage Examples
406

407
**Custom Object Stream Configuration:**
408
```yaml
409
source:
410
  type: airbyte/source-hubspot
411
  config:
412
    credentials:
413
      credentials_title: "Private App Credentials"
414
      access_token: "${HUBSPOT_ACCESS_TOKEN}"
415
    start_date: "2023-01-01T00:00:00Z"
416

417
# Custom object streams are automatically discovered and available
418
# Stream names match custom object schema names in HubSpot
419
```
420

421
**Querying Custom Object Data:**
422
```sql
423
-- Query custom vehicle object
424
SELECT 
425
  id,
426
  properties_vehicle_make,
427
  properties_vehicle_model,
428
  properties_year,
429
  properties_price,
430
  createdAt,
431
  updatedAt
432
FROM vehicles
433
WHERE properties_year >= '2020'
434
ORDER BY updatedAt DESC;
435

436
-- Query custom project object with associations
437
SELECT 
438
  p.id,
439
  p.properties_project_name,
440
  p.properties_status,
441
  p.contacts,  -- Associated contact IDs
442
  p.companies  -- Associated company IDs  
443
FROM projects p
444
WHERE p.properties_status = 'active';
445
```