0
# Airbyte Source Pipedrive
1

2
A manifest-only Airbyte source connector that extracts data from Pipedrive CRM using their REST API. This connector operates as a declarative configuration-based data integration tool that runs within the base `source-declarative-manifest` Docker image, providing comprehensive data extraction from Pipedrive including deals, activities, persons, organizations, pipelines, stages, users, and custom fields.
3

4
## Package Information
5

6
- **Package Name**: airbyte-source-pipedrive
7
- **Package Type**: Docker container
8
- **Language**: YAML manifest with Python components
9
- **Installation**: `docker pull airbyte/source-pipedrive:2.4.0`
10
- **Base Image**: `docker.io/airbyte/source-declarative-manifest:6.33.1`
11

12
## Core Usage
13

14
```bash
15
# Get connector specification
16
docker run --rm airbyte/source-pipedrive:2.4.0 spec
17

18
# Test connection
19
docker run --rm -v $(pwd)/config:/config airbyte/source-pipedrive:2.4.0 check --config /config/config.json
20

21
# Discover available streams
22
docker run --rm -v $(pwd)/config:/config airbyte/source-pipedrive:2.4.0 discover --config /config/config.json
23

24
# Extract data
25
docker run --rm -v $(pwd)/config:/config -v $(pwd)/catalog:/catalog airbyte/source-pipedrive:2.4.0 read --config /config/config.json --catalog /catalog/configured_catalog.json
26
```
27

28
## Basic Usage
29

30
```json
31
{
32
  "api_token": "your_pipedrive_api_token",
33
  "replication_start_date": "2017-01-25 00:00:00Z"
34
}
35
```
36

37
## Architecture
38

39
The connector uses Airbyte's low-code connector development kit (CDK) with a declarative YAML-based architecture. It includes:
40

41
- **Manifest Configuration**: Declarative stream definitions in `manifest.yaml`
42
- **Custom Components**: Python classes for handling Pipedrive's inconsistent API responses
43
- **Stream-based Architecture**: 26 different data streams covering all Pipedrive entities
44
- **Incremental Sync**: Cursor-based incremental synchronization for most entity streams
45
- **Robust Pagination**: Handles Pipedrive's cursor-based pagination patterns
46

47
## Capabilities
48

49
### Configuration and Connection
50

51
Handles connector configuration, authentication, and connection testing.
52

53
```yaml { .api }
54
# Connection specification
55
spec:
56
  type: Spec
57
  connection_specification:
58
    type: object
59
    required:
60
      - api_token
61
      - replication_start_date
62
    properties:
63
      api_token:
64
        type: string
65
        description: The Pipedrive API Token
66
        airbyte_secret: true
67
      replication_start_date:
68
        type: string
69
        description: UTC date and time in the format 2017-01-25T00:00:00Z
70
```
71

72
[Configuration and Connection](./configuration.md)
73

74
### Core Entity Streams
75

76
Extracts primary business entities from Pipedrive with incremental sync capabilities.
77

78
```yaml { .api }
79
# Example core entity stream structure
80
deals:
81
  type: DeclarativeStream
82
  name: deals
83
  primary_key: [id]
84
  retriever:
85
    type: SimpleRetriever
86
    requester:
87
      path: v1/recents
88
      request_parameters:
89
        api_token: "{{ config['api_token'] }}"
90
        items: deal
91
  incremental_sync:
92
    type: DatetimeBasedCursor
93
    cursor_field: update_time
94
```
95

96
Supported entities: deals, persons, activities, notes, files, products, leads
97

98
[Core Entity Streams](./core-entities.md)
99

100
### Metadata and Configuration Streams
101

102
Extracts Pipedrive configuration metadata and custom field definitions.
103

104
```yaml { .api }
105
# Example metadata stream structure
106
deal_fields:
107
  type: DeclarativeStream
108
  name: deal_fields
109
  retriever:
110
    type: SimpleRetriever
111
    requester:
112
      path: v1/dealFields
113
```
114

115
Includes: deal_fields, organization_fields, person_fields, activity_fields, activity_types, product_fields, pipelines, stages, users, roles, permission_sets, currencies, lead_labels, goals, filters
116

117
[Metadata and Configuration Streams](./metadata-streams.md)
118

119
### Relationship and Communication Streams
120

121
Handles relationships between entities and communication data.
122

123
```yaml { .api }
124
# Example relationship stream
125
deal_products:
126
  type: DeclarativeStream
127
  name: deal_products
128
  retriever:
129
    type: SimpleRetriever
130
    requester:
131
      path: v1/deals/{parent_id}/products
132
    partition_router:
133
      type: SubstreamPartitionRouter
134
      parent_stream_configs:
135
        - stream: deals
136
```
137

138
Includes: deal_products, mail, mailThreads
139

140
[Relationship and Communication Streams](./relationship-streams.md)
141

142
### Custom Components
143

144
Python components for handling Pipedrive's API inconsistencies.
145

146
```python { .api }
147
@dataclass
148
class NullCheckedDpathExtractor(RecordExtractor):
149
    """Custom extractor for Pipedrive's inconsistent API responses."""
150
    
151
    field_path: List[Union[InterpolatedString, str]]
152
    nullable_nested_field: Union[InterpolatedString, str]
153
    config: Config
154
    
155
    def extract_records(self, response: requests.Response) -> List[Mapping[str, Any]]:
156
        """Extract records, handling null data fields."""
157
```
158

159
[Custom Components](./custom-components.md)
160

161
## Types
162

163
```yaml { .api }
164
# Base stream configuration
165
DeclarativeStream:
166
  type: object
167
  properties:
168
    type: 
169
      const: DeclarativeStream
170
    name: string
171
    primary_key: array
172
    retriever: SimpleRetriever
173
    incremental_sync: DatetimeBasedCursor
174
    schema_loader: InlineSchemaLoader
175

176
# Request configuration  
177
SimpleRetriever:
178
  type: object
179
  properties:
180
    type:
181
      const: SimpleRetriever
182
    requester: HttpRequester
183
    record_selector: RecordSelector
184
    paginator: DefaultPaginator
185

186
# Authentication and pagination
187
HttpRequester:
188
  type: object
189
  properties:
190
    url_base: string
191
    path: string
192
    http_method: string
193
    request_parameters: object
194

195
# Incremental sync configuration
196
DatetimeBasedCursor:
197
  type: object
198
  properties:
199
    type:
200
      const: DatetimeBasedCursor
201
    cursor_field: string
202
    cursor_datetime_formats: array
203
    datetime_format: string
204
    start_datetime: MinMaxDatetime
205
```