Core server lifecycle management and initialization for the Spark Hive Thrift Server with Spark SQL integration.
Main entry point for starting and managing the Spark Hive Thrift Server.
/**
* The main entry point for the Spark SQL port of HiveServer2
*/
object HiveThriftServer2 {
/**
* Starts a new thrift server with the given SQL context
* @param sqlContext The Spark SQL context to use for query execution
* @return HiveThriftServer2 instance representing the running server
*/
def startWithContext(sqlContext: SQLContext): HiveThriftServer2
/**
* Command-line entry point for the thrift server
* @param args Command line arguments including Hive configuration options
*/
def main(args: Array[String]): Unit
/**
* Execution state enumeration for tracking operation states
* Note: This is private[thriftserver] - not part of public API
*/
private[thriftserver] object ExecutionState extends Enumeration {
type ExecutionState = Value
val STARTED, COMPILED, CANCELED, TIMEDOUT, FAILED, FINISHED, CLOSED = Value
}
}Usage Examples:
import org.apache.spark.sql.hive.thriftserver.{HiveThriftServer2, SparkSQLEnv}
// Initialize the SparkSQL environment first
SparkSQLEnv.init()
// Start the server with the initialized SQL context
val server = HiveThriftServer2.startWithContext(SparkSQLEnv.sqlContext)
// The server is now running and accepting connections
// It will automatically set up UI tabs and listeners# Start from command line with custom port
spark-submit --class org.apache.spark.sql.hive.thriftserver.HiveThriftServer2 \
--conf spark.sql.hive.thriftServer.singleSession=true \
spark-hive-thriftserver_2.12-3.5.6.jar \
--hiveconf hive.server2.thrift.port=10001The server instance class that extends HiveServer2 with Spark SQL capabilities.
/**
* The server instance that provides HiveServer2 compatibility
* Note: This class is private[hive] - not directly instantiated by users
*/
private[hive] class HiveThriftServer2(sqlContext: SQLContext) extends HiveServer2 {
/**
* Initialize the server with Hive configuration
* @param hiveConf Hive configuration object
*/
def init(hiveConf: HiveConf): Unit
/**
* Start the server services
*/
def start(): Unit
/**
* Stop the server services
*/
def stop(): Unit
}Singleton managing the global SparkContext and SQLContext lifecycle for the thrift server.
/**
* Singleton environment manager for SparkContext and SQLContext
*/
object SparkSQLEnv {
/**
* Initialize the Spark SQL environment
* Creates SparkContext and SQLContext if not already initialized
*/
def init(): Unit
/**
* Stop the Spark SQL environment
* @param exitCode Exit code for the application
*/
def stop(exitCode: Int): Unit
/**
* Global SQL context instance
*/
def sqlContext: SQLContext
/**
* Global Spark context instance
*/
def sparkContext: SparkContext
}Usage Examples:
import org.apache.spark.sql.hive.thriftserver.SparkSQLEnv
// Initialize the environment (creates SparkContext and SQLContext)
SparkSQLEnv.init()
// Access the global contexts
val sqlCtx = SparkSQLEnv.sqlContext
val sparkCtx = SparkSQLEnv.sparkContext
// Check if environment is initialized
if (SparkSQLEnv.sparkContext != null) {
println(s"Spark master: ${SparkSQLEnv.sparkContext.master}")
println(s"Application ID: ${SparkSQLEnv.sparkContext.applicationId}")
}
// Clean shutdown
SparkSQLEnv.stop(0)The server supports various configuration options through Hive configuration and Spark configuration.
// Common configuration options (set via --hiveconf or spark.conf)
// Transport mode
hive.server2.transport.mode = "binary" | "http"
// Port configuration
hive.server2.thrift.port = 10000 // Binary mode port
hive.server2.thrift.http.port = 10001 // HTTP mode port
// Authentication
hive.server2.authentication = "NONE" | "KERBEROS" | "CUSTOM"
hive.server2.authentication.kerberos.principal = "principal@REALM"
hive.server2.authentication.kerberos.keytab = "/path/to/keytab"
// Session management
hive.server2.session.check.interval = "6h"
hive.server2.idle.session.timeout = "7d"Server management includes comprehensive error handling for startup and lifecycle management.
// Common exceptions that may be thrown during server management
class HiveThriftServerErrors {
def taskExecutionRejectedError(rejected: RejectedExecutionException): Throwable
def runningQueryError(e: Throwable, format: ErrorMessageFormat.Value): Throwable
def failedToOpenNewSessionError(e: Throwable): Throwable
def cannotLoginToKerberosError(e: Exception): Throwable
def cannotLoginToSpnegoError(principal: String, keyTabFile: String, e: IOException): Throwable
}Common Error Scenarios:
// Server startup with error handling
try {
SparkSQLEnv.init()
val server = HiveThriftServer2.startWithContext(SparkSQLEnv.sqlContext)
// Check if SparkContext was stopped during startup
if (SparkSQLEnv.sparkContext.stopped.get()) {
throw new RuntimeException("SparkContext stopped during server startup")
}
} catch {
case e: Exception =>
println(s"Error starting HiveThriftServer2: ${e.getMessage}")
SparkSQLEnv.stop(-1)
System.exit(-1)
}The server automatically registers shutdown hooks for clean resource cleanup.
// Automatic shutdown hook registration
ShutdownHookManager.addShutdownHook { () =>
SparkSQLEnv.stop()
// UI tabs are automatically detached
// Server services are stopped
}