0
# Data Streams
1
2
Access to 12 Mailchimp data streams including campaigns, lists, segments, email activity, and detailed analytics reports. All streams are configured declaratively through the manifest.yaml file with automatic pagination, error handling, and incremental sync capabilities.
3
4
## Stream Configuration
5
6
All streams inherit from declarative base configurations that provide:
7
8
- **Authentication**: Automatic selection between OAuth and API key methods
9
- **Pagination**: Offset-based pagination with 1000 records per page
10
- **Rate Limiting**: Built-in handling of Mailchimp API rate limits
11
- **Data Center Routing**: Automatic routing to correct Mailchimp data center
12
- **Incremental Sync**: Timestamp-based incremental updates where supported
13
14
## Available Streams
15
16
### Campaign Streams
17
18
#### Campaigns
19
Email marketing campaigns with comprehensive metadata and settings.
20
21
- **Primary Key**: `id`
22
- **Cursor Field**: `create_time`
23
- **Sync Mode**: Incremental
24
- **API Endpoint**: `/campaigns`
25
26
Contains campaign details, settings, content, recipients, and delivery statistics.
27
28
#### Email Activity
29
Detailed tracking of individual email interactions and engagement events.
30
31
- **Primary Key**: `["timestamp", "email_id", "action"]`
32
- **Cursor Field**: `timestamp`
33
- **Sync Mode**: Incremental
34
- **API Endpoint**: `/reports/{campaign_id}/email-activity`
35
- **Custom Extractor**: MailChimpRecordExtractorEmailActivity
36
37
Tracks opens, clicks, bounces, unsubscribes, and other email activities with timestamp precision.
38
39
#### Reports
40
Campaign performance analytics and detailed reporting metrics.
41
42
- **Primary Key**: `id`
43
- **Cursor Field**: `send_time`
44
- **Sync Mode**: Incremental
45
- **API Endpoint**: `/reports`
46
47
Includes open rates, click rates, bounce rates, unsubscribe rates, and detailed campaign statistics.
48
49
### Audience Management Streams
50
51
#### Lists
52
Mailing lists and audience segments with subscriber counts and settings.
53
54
- **Primary Key**: `id`
55
- **Cursor Field**: `date_created`
56
- **Sync Mode**: Incremental
57
- **API Endpoint**: `/lists`
58
59
Contains list metadata, subscriber counts, default settings, and contact information.
60
61
#### List Members
62
Individual subscribers within mailing lists with detailed profile information.
63
64
- **Primary Key**: `["id", "list_id"]`
65
- **Cursor Field**: `last_changed`
66
- **Sync Mode**: Incremental
67
- **API Endpoint**: `/lists/{list_id}/members`
68
69
Includes email addresses, subscriber status, merge fields, interests, and engagement statistics.
70
71
#### Segments
72
Audience segments within lists based on subscriber criteria and behavior.
73
74
- **Primary Key**: `id`
75
- **Cursor Field**: `updated_at`
76
- **Sync Mode**: Incremental
77
- **API Endpoint**: `/lists/{list_id}/segments`
78
79
Contains segment definitions, member counts, and segmentation criteria.
80
81
#### Segment Members
82
Individual subscribers within specific audience segments.
83
84
- **Primary Key**: `["id", "segment_id"]`
85
- **Cursor Field**: `last_changed`
86
- **Sync Mode**: Incremental
87
- **API Endpoint**: `/lists/{list_id}/segments/{segment_id}/members`
88
89
Links subscribers to segments with join timestamps and engagement data.
90
91
### Subscriber Interaction Streams
92
93
#### Tags
94
Contact tags for organizing and categorizing subscribers.
95
96
- **Primary Key**: `["id", "name"]`
97
- **Sync Mode**: Full Refresh
98
- **API Endpoint**: `/lists/{list_id}/tags`
99
100
Contains tag names, creation dates, and usage counts across subscribers.
101
102
#### Interest Categories
103
Interest categories for subscriber preference management and segmentation.
104
105
- **Primary Key**: `["id", "list_id"]`
106
- **Sync Mode**: Full Refresh
107
- **API Endpoint**: `/lists/{list_id}/interest-categories`
108
109
Defines interest categories with titles, types, and subscriber opt-in preferences.
110
111
#### Interests
112
Individual interests within categories for granular subscriber preferences.
113
114
- **Primary Key**: `id`
115
- **Sync Mode**: Full Refresh
116
- **API Endpoint**: `/lists/{list_id}/interest-categories/{interest_category_id}/interests`
117
118
Contains specific interest options subscribers can select within categories.
119
120
#### Unsubscribes
121
Unsubscribe events and opt-out history with detailed reasoning and timestamps.
122
123
- **Primary Key**: `["campaign_id", "email_id", "timestamp"]`
124
- **Cursor Field**: `timestamp`
125
- **Sync Mode**: Incremental
126
- **API Endpoint**: `/reports/{campaign_id}/unsubscribed`
127
128
Tracks unsubscribe events with campaign attribution and subscriber details.
129
130
### Automation Streams
131
132
#### Automations
133
Email automation workflows and drip campaigns with trigger conditions.
134
135
- **Primary Key**: `id`
136
- **Cursor Field**: `create_time`
137
- **Sync Mode**: Incremental
138
- **API Endpoint**: `/automations`
139
140
Contains automation settings, triggers, email sequences, and performance metrics.
141
142
## Stream Access
143
144
All streams are automatically available when using the SourceMailchimp connector. The declarative configuration handles:
145
146
- **Stream Discovery**: Automatic detection of available streams and their schemas
147
- **Data Extraction**: Efficient pagination and rate limit handling
148
- **Schema Evolution**: Automatic handling of API schema changes
149
- **Error Recovery**: Robust error handling and retry mechanisms
150
151
## Usage Examples
152
153
### Accessing Stream Data
154
155
```python
156
from source_mailchimp import SourceMailchimp
157
158
# Create connector instance
159
source = SourceMailchimp()
160
161
# Discover available streams
162
catalog = source.discover(logger, config)
163
164
# Available streams will include all 12 data streams:
165
# - automations, campaigns, email_activity, lists, list_members
166
# - tags, interest_categories, interests, reports, segments
167
# - segment_members, unsubscribes
168
```
169
170
### Stream Configuration
171
172
```json
173
{
174
"streams": [
175
{
176
"stream": {
177
"name": "campaigns",
178
"json_schema": {...},
179
"supported_sync_modes": ["full_refresh", "incremental"]
180
},
181
"sync_mode": "incremental",
182
"cursor_field": ["create_time"],
183
"destination_sync_mode": "append"
184
}
185
]
186
}
187
```