0
# Airbyte Source Pipedrive
1
2
A manifest-only Airbyte source connector that extracts data from Pipedrive CRM using their REST API. This connector operates as a declarative configuration-based data integration tool that runs within the base `source-declarative-manifest` Docker image, providing comprehensive data extraction from Pipedrive including deals, activities, persons, organizations, pipelines, stages, users, and custom fields.
3
4
## Package Information
5
6
- **Package Name**: airbyte-source-pipedrive
7
- **Package Type**: Docker container
8
- **Language**: YAML manifest with Python components
9
- **Installation**: `docker pull airbyte/source-pipedrive:2.4.0`
10
- **Base Image**: `docker.io/airbyte/source-declarative-manifest:6.33.1`
11
12
## Core Usage
13
14
```bash
15
# Get connector specification
16
docker run --rm airbyte/source-pipedrive:2.4.0 spec
17
18
# Test connection
19
docker run --rm -v $(pwd)/config:/config airbyte/source-pipedrive:2.4.0 check --config /config/config.json
20
21
# Discover available streams
22
docker run --rm -v $(pwd)/config:/config airbyte/source-pipedrive:2.4.0 discover --config /config/config.json
23
24
# Extract data
25
docker run --rm -v $(pwd)/config:/config -v $(pwd)/catalog:/catalog airbyte/source-pipedrive:2.4.0 read --config /config/config.json --catalog /catalog/configured_catalog.json
26
```
27
28
## Basic Usage
29
30
```json
31
{
32
"api_token": "your_pipedrive_api_token",
33
"replication_start_date": "2017-01-25 00:00:00Z"
34
}
35
```
36
37
## Architecture
38
39
The connector uses Airbyte's low-code connector development kit (CDK) with a declarative YAML-based architecture. It includes:
40
41
- **Manifest Configuration**: Declarative stream definitions in `manifest.yaml`
42
- **Custom Components**: Python classes for handling Pipedrive's inconsistent API responses
43
- **Stream-based Architecture**: 26 different data streams covering all Pipedrive entities
44
- **Incremental Sync**: Cursor-based incremental synchronization for most entity streams
45
- **Robust Pagination**: Handles Pipedrive's cursor-based pagination patterns
46
47
## Capabilities
48
49
### Configuration and Connection
50
51
Handles connector configuration, authentication, and connection testing.
52
53
```yaml { .api }
54
# Connection specification
55
spec:
56
type: Spec
57
connection_specification:
58
type: object
59
required:
60
- api_token
61
- replication_start_date
62
properties:
63
api_token:
64
type: string
65
description: The Pipedrive API Token
66
airbyte_secret: true
67
replication_start_date:
68
type: string
69
description: UTC date and time in the format 2017-01-25T00:00:00Z
70
```
71
72
[Configuration and Connection](./configuration.md)
73
74
### Core Entity Streams
75
76
Extracts primary business entities from Pipedrive with incremental sync capabilities.
77
78
```yaml { .api }
79
# Example core entity stream structure
80
deals:
81
type: DeclarativeStream
82
name: deals
83
primary_key: [id]
84
retriever:
85
type: SimpleRetriever
86
requester:
87
path: v1/recents
88
request_parameters:
89
api_token: "{{ config['api_token'] }}"
90
items: deal
91
incremental_sync:
92
type: DatetimeBasedCursor
93
cursor_field: update_time
94
```
95
96
Supported entities: deals, persons, activities, notes, files, products, leads
97
98
[Core Entity Streams](./core-entities.md)
99
100
### Metadata and Configuration Streams
101
102
Extracts Pipedrive configuration metadata and custom field definitions.
103
104
```yaml { .api }
105
# Example metadata stream structure
106
deal_fields:
107
type: DeclarativeStream
108
name: deal_fields
109
retriever:
110
type: SimpleRetriever
111
requester:
112
path: v1/dealFields
113
```
114
115
Includes: deal_fields, organization_fields, person_fields, activity_fields, activity_types, product_fields, pipelines, stages, users, roles, permission_sets, currencies, lead_labels, goals, filters
116
117
[Metadata and Configuration Streams](./metadata-streams.md)
118
119
### Relationship and Communication Streams
120
121
Handles relationships between entities and communication data.
122
123
```yaml { .api }
124
# Example relationship stream
125
deal_products:
126
type: DeclarativeStream
127
name: deal_products
128
retriever:
129
type: SimpleRetriever
130
requester:
131
path: v1/deals/{parent_id}/products
132
partition_router:
133
type: SubstreamPartitionRouter
134
parent_stream_configs:
135
- stream: deals
136
```
137
138
Includes: deal_products, mail, mailThreads
139
140
[Relationship and Communication Streams](./relationship-streams.md)
141
142
### Custom Components
143
144
Python components for handling Pipedrive's API inconsistencies.
145
146
```python { .api }
147
@dataclass
148
class NullCheckedDpathExtractor(RecordExtractor):
149
"""Custom extractor for Pipedrive's inconsistent API responses."""
150
151
field_path: List[Union[InterpolatedString, str]]
152
nullable_nested_field: Union[InterpolatedString, str]
153
config: Config
154
155
def extract_records(self, response: requests.Response) -> List[Mapping[str, Any]]:
156
"""Extract records, handling null data fields."""
157
```
158
159
[Custom Components](./custom-components.md)
160
161
## Types
162
163
```yaml { .api }
164
# Base stream configuration
165
DeclarativeStream:
166
type: object
167
properties:
168
type:
169
const: DeclarativeStream
170
name: string
171
primary_key: array
172
retriever: SimpleRetriever
173
incremental_sync: DatetimeBasedCursor
174
schema_loader: InlineSchemaLoader
175
176
# Request configuration
177
SimpleRetriever:
178
type: object
179
properties:
180
type:
181
const: SimpleRetriever
182
requester: HttpRequester
183
record_selector: RecordSelector
184
paginator: DefaultPaginator
185
186
# Authentication and pagination
187
HttpRequester:
188
type: object
189
properties:
190
url_base: string
191
path: string
192
http_method: string
193
request_parameters: object
194
195
# Incremental sync configuration
196
DatetimeBasedCursor:
197
type: object
198
properties:
199
type:
200
const: DatetimeBasedCursor
201
cursor_field: string
202
cursor_datetime_formats: array
203
datetime_format: string
204
start_datetime: MinMaxDatetime
205
```