creating-openlineage-extractors

Create custom OpenLineage extractors for Airflow operators. Use when the user needs lineage from unsupported or third-party operators, wants column-level lineage, or needs complex extraction logic beyond what inlets/outlets provide.

1.29x

Quality

82%

Does it follow best practices?

Impact

97%

1.29x

Average score across 3 eval scenarios

Securityby

Passed

No known issues

Evaluation results

95%

19%

Lineage Tracking for Vendor SFTP-to-S3 Transfers

Custom extractor for third-party operator

Criteria

Without context

With context

Inherits BaseExtractor

100%

get_operator_classnames implemented

100%

_execute_extraction implemented

100%

Local imports only

50%

SFTP namespace format

100%

S3 namespace format

100%

self.operator property access

100%

None handling

100%

OperatorLineage return

100%

airflow.cfg registration

100%

Env var registration

33%

100%

12%

Adding Lineage Tracking to an In-House Transform Operator

OpenLineage methods on owned operator

Criteria

Without context

With context

No extractor class

100%

get_openlineage_facets_on_start

100%

get_openlineage_facets_on_complete

100%

Output statistics facet

100%

Local imports in methods

100%

Postgres namespace format

100%

OperatorLineage import path

100%

Dataset import path

100%

Source and target in on_start

100%

96%

34%

Lineage Tracking for a Dynamic-Output Partition Loader

Dynamic lineage and extractor unit tests

Criteria

Without context

With context

_execute_extraction present

100%

extract_on_complete present

100%

get_operator_classnames implemented

100%

Source dataset in _execute_extraction

25%

100%

Dynamic outputs in extract_on_complete

100%

None guard in _execute_extraction

25%

100%

Redshift namespace format

100%

Tests use MagicMock for operator

100%

Test pre-run extraction

25%

100%

Test post-run extraction

100%

Test None case

25%

100%

Local imports in extractor

Repository: astronomer/agents
Commit: 535a040

Evaluated: 12 days ago
Agent: Claude Code
Model: Claude Sonnet 4.6

Table of Contents

Lineage Tracking for Vendor SFTP-to-S3 Transfers Adding Lineage Tracking to an In-House Transform Operator Lineage Tracking for a Dynamic-Output Partition Loader

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.