关键词: 大数据, 数据迁移, 关系型数据库, Hadoop平台, HiETL

Abstract: Current data management scheme in financial industry basically relies on traditional databases. However, due to the inherent limitation on the extension capability and storage performance, it is difficult to meet the rapid growing demands for massive data processing in big data era. Based on the characteristics of financial data, such as, large scale, diversity, cross region  and cross storage system, this paper proposes a big data migration management platform, called HiETL, the functions of which include the data migration from heterogeneous relational databases to Hadoop platform, centralized integration of massive data, expanding storage, and efficient query. The migration speed can reach 3 MB/s as long as the migration accuracy is ensured.

Key words: big data, data migration, relational databases, Hadoop platform, HiETL