Abstract
As big data becomes popular, the data warehouse needs the ability to process massive data fast. Hadoop framework allows for the distributed processing of large data sets and provides a distributed file system (HDFS) that provides high throughput access to data. Currently, Hadoop framework is the most suitable platform for processing big data. Hive is a data warehouse built on Hadoop and uses MapReduce as the execution environment. It provides SQL-like query language called HiveQL for user to query data in the data warehouse. Although there is a fault tolerance mechanism in Hive, its response time is long. Impala is another SQL engine on Hadoop which is compatible with Hive. Its response time is short, but the data processed by Impala is constrained by memory size. Furthermore, it does not provide fault tolerance mechanism. In this thesis, we focus on designing a fusion SQL engine system that combines Hive and Impala. It provides a uniform interface and a fault tolerance mechanism. After the system parses the query from users, it leverages the preprocessed statistics to estimate the memory consumption, and chooses the SQL engine (Hive or Impala) to execute the query automatically. It makes Hive and Impala become complement of each other.