0
# Flink SQL Parser Hive
1

2
Flink SQL Parser Hive is a specialized SQL parser component for Apache Flink that extends the core SQL parser to support Hive dialect syntax. It provides comprehensive parsing capabilities for Hive-specific DDL (Data Definition Language) and DML (Data Manipulation Language) statements, enabling seamless integration with Hive metastores and compatibility with existing Hive data warehouses within Flink's distributed stream and batch processing engine.
3

4
## Package Information
5

6
- **Package Name**: flink-sql-parser-hive
7
- **Package Type**: maven
8
- **Language**: Java
9
- **Installation**: Include as Maven dependency with coordinates `org.apache.flink:flink-sql-parser-hive:1.17.2`
10

11
## Core Imports
12

13
```java
14
import org.apache.flink.sql.parser.hive.impl.FlinkHiveSqlParserImpl;
15
import org.apache.flink.sql.parser.hive.ddl.*;
16
import org.apache.flink.sql.parser.hive.dml.*;
17
import org.apache.flink.sql.parser.hive.type.*;
18
```
19

20
## Basic Usage
21

22
```java
23
import org.apache.calcite.sql.SqlParser;
24
import org.apache.flink.sql.parser.hive.impl.FlinkHiveSqlParserImpl;
25

26
// Create a Hive SQL parser
27
SqlParser parser = SqlParser.create(sqlStatement,
28
    SqlParser.config()
29
        .withParserFactory(FlinkHiveSqlParserImpl.FACTORY)
30
        .withQuoting(Quoting.DOUBLE_QUOTE)
31
        .withUnquotedCasing(Casing.TO_UPPER)
32
        .withQuotedCasing(Casing.UNCHANGED));
33

34
// Parse Hive SQL statement
35
SqlNode sqlNode = parser.parseStmt();
36

37
// Example: Create Hive table with partitions
38
String createTableSql = """
39
    CREATE TABLE IF NOT EXISTS sales_data (
40
        id BIGINT,
41
        customer_name STRING,
42
        amount DECIMAL(10,2)
43
    )
44
    PARTITIONED BY (year INT, month INT)
45
    STORED AS PARQUET
46
    LOCATION '/data/sales'
47
    TBLPROPERTIES ('transactional'='true')
48
    """;
49
```
50

51
## Architecture
52

53
The Flink SQL Parser Hive is built around several key components:
54

55
- **Generated Parser**: `FlinkHiveSqlParserImpl` provides the main parsing entry point using JavaCC and FMPP code generation
56
- **DDL System**: Comprehensive support for Hive DDL operations including databases, tables, views, and partitions
57
- **DML Extensions**: Enhanced INSERT statements with static and dynamic partition support
58
- **Type System**: Extended type specifications for Hive-specific data types like STRUCT with field comments
59
- **Constraint Handling**: Three-dimensional constraint system (ENABLE/DISABLE, VALIDATE/NOVALIDATE, RELY/NORELY)
60
- **Property Management**: Reserved property validation and escaping for SQL client compatibility
61

62
## Capabilities
63

64
### Parser Integration
65

66
Core parser factory and integration point for creating Hive SQL parsers within Apache Calcite framework.
67

68
```java { .api }
69
/**
70
 * Main parser factory for creating Hive SQL parser instances
71
 */
72
public class FlinkHiveSqlParserImpl {
73
    public static final SqlParserImplFactory FACTORY;
74
}
75
```
76

77
[Parser Integration](./parser-integration.md)
78

79
### Database Operations
80

81
Complete database lifecycle management including creation, alteration, and property management for Hive databases.
82

83
```java { .api }
84
/**
85
 * CREATE DATABASE statement for Hive dialect
86
 */
87
public class SqlCreateHiveDatabase extends SqlCreateDatabase {
88
    public SqlCreateHiveDatabase(SqlParserPos pos, SqlIdentifier databaseName, 
89
                                SqlNodeList propertyList, SqlCharStringLiteral comment, 
90
                                SqlCharStringLiteral location, boolean ifNotExists) throws ParseException;
91
}
92

93
/**
94
 * Base class for ALTER DATABASE operations
95
 */
96
public abstract class SqlAlterHiveDatabase extends SqlAlterDatabase {
97
    public enum AlterHiveDatabaseOp { CHANGE_PROPS, CHANGE_LOCATION, CHANGE_OWNER }
98
}
99
```
100

101
[Database Operations](./database-operations.md)
102

103
### Table Operations
104

105
Comprehensive table management including creation with Hive-specific features, alteration, and column management.
106

107
```java { .api }
108
/**
109
 * CREATE TABLE statement for Hive dialect with full Hive table features
110
 */
111
public class SqlCreateHiveTable extends SqlCreateTable {
112
    public SqlCreateHiveTable(SqlParserPos pos, SqlIdentifier tableName, SqlNodeList columnList,
113
                             HiveTableCreationContext creationContext, SqlNodeList propertyList,
114
                             SqlNodeList partColList, SqlCharStringLiteral comment, boolean isTemporary,
115
                             boolean isExternal, HiveTableRowFormat rowFormat, 
116
                             HiveTableStoredAs storedAs, SqlCharStringLiteral location, 
117
                             boolean ifNotExists) throws ParseException;
118
}
119

120
/**
121
 * ROW FORMAT specification for Hive tables
122
 */
123
public static class HiveTableRowFormat {
124
    public static HiveTableRowFormat withDelimited(...) throws ParseException;
125
    public static HiveTableRowFormat withSerDe(...) throws ParseException;
126
}
127

128
/**
129
 * STORED AS specification for Hive tables
130
 */
131
public static class HiveTableStoredAs {
132
    public static HiveTableStoredAs ofFileFormat(...) throws ParseException;
133
    public static HiveTableStoredAs ofInputOutputFormat(...) throws ParseException;
134
}
135
```
136

137
[Table Operations](./table-operations.md)
138

139
### Partition Management
140

141
Partition operations for adding, renaming, and managing Hive table partitions.
142

143
```java { .api }
144
/**
145
 * ADD PARTITION statement for Hive tables
146
 */
147
public class SqlAddHivePartitions extends SqlCall {
148
    public SqlAddHivePartitions(SqlParserPos pos, SqlIdentifier tableName, boolean ifNotExists,
149
                               List<SqlNodeList> partSpecs, List<SqlCharStringLiteral> partLocations);
150
}
151

152
/**
153
 * PARTITION RENAME statement for Hive tables
154
 */
155
public class SqlAlterHivePartitionRename extends SqlAlterHiveTable {
156
    public SqlNodeList getNewPartSpec();
157
}
158
```
159

160
[Partition Management](./partition-management.md)
161

162
### View Operations
163

164
View creation and management with Hive-specific properties and syntax.
165

166
```java { .api }
167
/**
168
 * CREATE VIEW statement for Hive dialect
169
 */
170
public class SqlCreateHiveView extends SqlCreateView {
171
    public SqlCreateHiveView(SqlParserPos pos, SqlIdentifier viewName, SqlNodeList fieldList,
172
                            SqlNode query, boolean ifNotExists, SqlCharStringLiteral comment,
173
                            SqlNodeList properties);
174
}
175
```
176

177
[View Operations](./view-operations.md)
178

179
### Data Manipulation
180

181
Enhanced INSERT statements with comprehensive partition support for both static and dynamic partitioning.
182

183
```java { .api }
184
/**
185
 * Enhanced INSERT statement for Hive tables with partition support
186
 */
187
public class RichSqlHiveInsert extends RichSqlInsert {
188
    public RichSqlHiveInsert(SqlParserPos pos, SqlNodeList keywords, SqlNodeList extendedKeywords,
189
                            SqlNode targetTable, SqlNode source, SqlNodeList columnList,
190
                            SqlNodeList staticPartitions, SqlNodeList allPartKeys);
191
}
192
```
193

194
[Data Manipulation](./data-manipulation.md)
195

196
### Constraint System
197

198
Three-dimensional constraint system supporting ENABLE/DISABLE, VALIDATE/NOVALIDATE, and RELY/NORELY traits.
199

200
```java { .api }
201
/**
202
 * Complete constraint trait specification
203
 */
204
public class SqlHiveConstraintTrait {
205
    public SqlHiveConstraintTrait(SqlLiteral enable, SqlLiteral validate, SqlLiteral rely);
206
    public boolean isEnable();
207
    public boolean isValidate();
208
    public boolean isRely();
209
}
210

211
/**
212
 * Constraint enable/disable options
213
 */
214
public enum SqlHiveConstraintEnable { ENABLE, DISABLE }
215

216
/**
217
 * Constraint validate/no-validate options  
218
 */
219
public enum SqlHiveConstraintValidate { VALIDATE, NOVALIDATE }
220

221
/**
222
 * Constraint rely/no-rely options
223
 */
224
public enum SqlHiveConstraintRely { RELY, NORELY }
225
```
226

227
[Constraint System](./constraint-system.md)
228

229
### Type System Extensions
230

231
Extended type system supporting Hive-specific data types including enhanced STRUCT types with field comments.
232

233
```java { .api }
234
/**
235
 * STRUCT type specification with field names, types, and comments
236
 */
237
public class ExtendedHiveStructTypeNameSpec extends ExtendedSqlRowTypeNameSpec {
238
    public ExtendedHiveStructTypeNameSpec(SqlParserPos pos, List<SqlIdentifier> fieldNames,
239
                                         List<SqlDataTypeSpec> fieldTypes, 
240
                                         List<SqlCharStringLiteral> comments) throws ParseException;
241
}
242
```
243

244
[Type System Extensions](./type-system.md)
245

246
### Utilities and Helpers
247

248
Utility classes for property validation, data type conversion, and constraint trait management.
249

250
```java { .api }
251
/**
252
 * Utility methods for Hive DDL operations
253
 */
254
public class HiveDDLUtils {
255
    public static final String COL_DELIMITER;
256
    
257
    public static SqlNodeList checkReservedTableProperties(SqlNodeList props) throws ParseException;
258
    public static void convertDataTypes(SqlNodeList columns) throws ParseException;
259
    public static byte encodeConstraintTrait(SqlHiveConstraintTrait trait);
260
    public static SqlCharStringLiteral unescapeStringLiteral(SqlCharStringLiteral literal);
261
}
262
```
263

264
[Utilities and Helpers](./utilities.md)
265

266
## Exception Handling
267

268
All parsing operations may throw `ParseException` when SQL syntax is invalid or when validation fails:
269

270
```java
271
try {
272
    SqlNode node = parser.parseStmt();
273
} catch (ParseException e) {
274
    // Handle parsing errors
275
    System.err.println("SQL parsing failed: " + e.getMessage());
276
}
277
```
278

279
## Integration with Flink
280

281
This parser integrates seamlessly with Apache Flink's table ecosystem:
282

283
1. **SQL Gateway**: Use with Flink SQL Gateway for Hive compatibility
284
2. **Table API**: Parse Hive DDL statements in Table API programs  
285
3. **Catalog Integration**: Works with HiveCatalog for metadata management
286
4. **Streaming/Batch**: Supports both streaming and batch processing scenarios