0
# Apache Flink HBase 2.2 SQL Connector
1

2
Apache Flink SQL connector that enables seamless integration with HBase 2.2.x databases through Flink's Table API and SQL interface. This shaded JAR bundles all necessary HBase client dependencies with proper relocation to avoid classpath conflicts in Flink deployments.
3

4
## Package Information
5

6
- **Package Name**: flink-sql-connector-hbase-2.2_2.11
7
- **Group ID**: org.apache.flink
8
- **Language**: Java
9
- **Maven Installation**: Add to your project dependencies:
10

11
```xml
12
<dependency>
13
    <groupId>org.apache.flink</groupId>
14
    <artifactId>flink-sql-connector-hbase-2.2_2.11</artifactId>
15
    <version>1.14.6</version>
16
</dependency>
17
```
18

19
For runtime usage (when running Flink applications):
20

21
```bash
22
# Copy to Flink lib directory
23
cp flink-sql-connector-hbase-2.2_2.11-1.14.6.jar $FLINK_HOME/lib/
24
```
25

26
## Core Usage
27

28
The connector integrates with Flink SQL through DDL statements. No explicit Java imports are needed for SQL usage.
29

30
## Basic Usage
31

32
```sql
33
-- Create HBase table mapping
34
CREATE TABLE hbase_table (
35
    rowkey STRING,
36
    info ROW<name STRING, age INT>,
37
    data ROW<value DOUBLE, timestamp BIGINT>,
38
    PRIMARY KEY (rowkey) NOT ENFORCED
39
) WITH (
40
    'connector' = 'hbase-2.2',
41
    'table-name' = 'my_hbase_table',
42
    'zookeeper.quorum' = 'localhost:2181'
43
);
44

45
-- Insert data (UPSERT operation)
46
INSERT INTO hbase_table VALUES (
47
    'user_001',
48
    ROW('Alice', 25),
49
    ROW(95.5, 1234567890)
50
);
51

52
-- Query data
53
SELECT rowkey, info.name, info.age, data.value 
54
FROM hbase_table 
55
WHERE info.age > 18;
56

57
-- Lookup join with another table
58
SELECT o.order_id, u.info.name, u.info.age
59
FROM orders o
60
JOIN hbase_table FOR SYSTEM_TIME AS OF o.proc_time AS u
61
ON o.user_id = u.rowkey;
62
```
63

64
## Architecture  
65

66
The connector architecture consists of several key components:
67

68
- **Factory System**: `HBase2DynamicTableFactory` provides the main entry point for Flink's connector discovery mechanism
69
- **Table Sources/Sinks**: Support both batch scanning and streaming lookup operations with UPSERT capabilities (INSERT, UPDATE_AFTER, DELETE)
70
- **Serialization Layer**: Handles type mapping between Flink's type system and HBase's byte storage format
71
- **Configuration System**: Comprehensive options for connection, caching, buffering, and performance tuning
72
- **Shaded Dependencies**: All HBase client libraries are relocated to prevent classpath conflicts
73

74
## Capabilities
75

76
### SQL DDL and Table Operations
77

78
Core SQL operations for creating HBase table mappings, defining column families and qualifiers, and managing table schemas with proper data type mappings.
79

80
```sql { .api }
81
CREATE TABLE table_name (
82
    rowkey STRING,
83
    family_name ROW<qualifier_name DATA_TYPE, ...>,
84
    PRIMARY KEY (rowkey) NOT ENFORCED
85
) WITH (
86
    'connector' = 'hbase-2.2',
87
    'table-name' = 'hbase_table_name',
88
    -- connection and configuration options
89
);
90
```
91

92
[SQL DDL and Operations](./sql-ddl.md)
93

94
### Connection and Configuration
95

96
Comprehensive configuration options for HBase connections, Zookeeper settings, performance tuning, and operational parameters.
97

98
```java { .api }
99
// Key configuration options available via WITH clause
100
'connector' = 'hbase-2.2'                           // Required: Connector identifier
101
'table-name' = 'hbase_table_name'                   // Required: HBase table name
102
'zookeeper.quorum' = 'host1:2181,host2:2181'       // HBase Zookeeper quorum
103
'zookeeper.znode.parent' = '/hbase'                 // Zookeeper root (default: '/hbase')
104
'null-string-literal' = 'null'                     // Null representation (default: 'null')
105
```
106

107
[Connection and Configuration](./connection-config.md)
108

109
### Lookup Operations and Caching
110

111
Temporal table join functionality with both synchronous and asynchronous lookup modes, including comprehensive caching strategies for performance optimization.
112

113
```java { .api }
114
// Lookup configuration options
115
'lookup.async' = 'false'                           // Enable async lookup (default: false)
116
'lookup.cache.max-rows' = '1000'                   // Cache size (default: -1, disabled)
117
'lookup.cache.ttl' = '60s'                         // Cache TTL (default: 0)
118
'lookup.max-retries' = '3'                         // Max retry attempts (default: 3)
119
```
120

121
[Lookup Operations](./lookup-operations.md)
122

123
### Sink Operations and Buffering
124

125
UPSERT sink operations with intelligent buffering strategies, exactly-once semantics through checkpointing, and comprehensive error handling.
126

127
```java { .api }
128
// Sink configuration options
129
'sink.buffer-flush.max-size' = '2mb'               // Buffer size threshold (default: 2MB)
130
'sink.buffer-flush.max-rows' = '1000'              // Buffer row count (default: 1000)
131
'sink.buffer-flush.interval' = '1s'                // Flush interval (default: 1s)
132
'sink.parallelism' = '1'                           // Sink parallelism
133
```
134

135
[Sink Operations](./sink-operations.md)
136

137
### Data Types and Schema Mapping
138

139
Complete data type mapping between Flink's type system and HBase storage format, including support for complex nested types and proper serialization handling.
140

141
```java { .api }
142
// Supported Flink data types mapping to HBase bytes
143
BOOLEAN, TINYINT, SMALLINT, INTEGER, BIGINT        // Integer types
144
FLOAT, DOUBLE, DECIMAL                             // Floating point types  
145
CHAR, VARCHAR                                      // String types
146
BINARY, VARBINARY                                  // Binary types
147
DATE, TIME, TIMESTAMP                              // Temporal types
148
ROW<field_name field_type, ...>                   // Nested row types for column families
149
```
150

151
[Data Types and Schema](./data-types.md)
152

153
## Error Handling
154

155
The connector provides robust error handling with:
156

157
- **Connection Retry Logic**: Configurable retry attempts for HBase connection failures
158
- **Graceful Degradation**: Proper handling of table not found and region server unavailability  
159
- **Buffer Management**: Overflow protection and timeout handling for sink operations
160
- **Checkpointing Support**: Exactly-once processing guarantees through Flink's checkpoint mechanism
161
- **Validation Errors**: Clear error messages for schema mismatches and configuration issues