Research on metadata driven ETL

Computer Engineering and Applications ›› 2012, Vol. 48 ›› Issue (6): 114-118.

• 数据库、信号与信息处理 • Previous Articles Next Articles

Research on metadata driven ETL

SU Fang, SHOU Yongxi, SU Yila

College of Information Engineering, Inner Mongolia University of Technology, Hohhot 010051, China

Received:1900-01-01 Revised:1900-01-01 Online:2012-02-21 Published:2012-02-21

元数据驱动ETL的研究

宿芳，寿永熙，苏依拉

内蒙古工业大学信息工程学院，呼和浩特 010051

Abstract

Abstract: With the development of data warehouse, ETL（Extract, Transformation and Load） are so important steps in processing data while building a data warehouse. A stable and reliable ETL system is depended on the overall design. Base on the traditional ETL, this paper is mainly about metadata driven ETL. Using SQL engine, package engine and related algorithm to solve the problem, the aim is to improve the ETL System’s stability and reliability, and enable users to control the process by maintaining metadata.

Key words: Extract, Transformation and Load（ETL）, data warehouse, metadata, SQL engine, package engine

摘要： 对原始数据的提取、转换、加载（ETL）是数据仓库开发的重要步骤。一个稳定而可靠的ETL系统取决于它的总体设计。在原有ETL基础上，提出了元数据驱动ETL的思想，利用包引擎和SQL引擎以及相关算法，对传统ETL的方法进行改进，目的是提高ETL系统的稳定性和可靠性，使用户通过对元数据的维护来控制ETL的执行过程。

关键词: 抽取、转换和加载（ETL）, 数据仓库, 元数据, SQL引擎, 包引擎

SU Fang, SHOU Yongxi, SU Yila. Research on metadata driven ETL[J]. Computer Engineering and Applications, 2012, 48(6): 114-118.

宿芳，寿永熙，苏依拉. 元数据驱动ETL的研究[J]. 计算机工程与应用, 2012, 48(6): 114-118.

[1]	BAO Zhiqiang, XING Yu, LYU Shaoqing, HUANG Qiongdan. Improved YOLO V2 6D Object Pose Estimation Algorithm [J]. Computer Engineering and Applications, 2021, 57(9): 148-153.
[2]	XU Degang, WANG Lu, LI Fan. Review of Typical Object Detection Algorithms for Deep Learning [J]. Computer Engineering and Applications, 2021, 57(8): 10-25.
[3]	HU Wentao, CHEN Xiuhong. Low-Rank Projection Learning Based on Neighbor Graph [J]. Computer Engineering and Applications, 2021, 57(7): 209-214.
[4]	CHEN Wei, XU Yun. Research on Extraction of Biomedical Entity Relation Based on Literature Mining [J]. Computer Engineering and Applications, 2021, 57(7): 115-120.
[5]	ZHANG Xiaoli, ZHANG Kuixing, JIANG Mei, WEI Benzheng, CONG Jinyu. Review of Image Classification Technology for Lymphoma [J]. Computer Engineering and Applications, 2021, 57(6): 1-9.
[6]	LIU Bowen, FAN Chunxiao. Relation Extraction Based on CapsuleNet via Position Perception [J]. Computer Engineering and Applications, 2021, 57(6): 101-107.
[7]	XIONG Jian, QIN Renchao, HE Mengyi, LIU Jianlan, TANG Fengyang. Application of Improved Random Forest Algorithm in Android Malware Detection [J]. Computer Engineering and Applications, 2021, 57(3): 130-136.
[8]	XU Zhijing, WANG Yi. Glaucoma Fundus Images Classification Method Based on Transfer Learning [J]. Computer Engineering and Applications, 2021, 57(3): 144-149.
[9]	LI Daidong, HE Xiaohui, LI Panle, TIAN Zhihui, ZHOU Guangsheng. Road Extraction Network of Remote Sensing Image Based on SPUD-ResNet [J]. Computer Engineering and Applications, 2021, 57(23): 240-247.
[10]	ZHAO Lihua, WANG Chunli, CHU Yufeng. Attention-Based Double BiReGU Model for Aspect Term Extraction [J]. Computer Engineering and Applications, 2021, 57(22): 160-165.
[11]	HUANG Meigen, LIU Jiale, LIU Chuan. Research on Improved BERT’s Chinese Multi-relation Extraction Method [J]. Computer Engineering and Applications, 2021, 57(21): 234-240.
[12]	WEI Hao, ZHOU Ai, ZHANG Yijia, CHEN Fei, QU Wen, LU Mingyu. Review of Deep Learning-Based Biomedical Entity Relation Extraction Research [J]. Computer Engineering and Applications, 2021, 57(21): 14-23.
[13]	LI Longlong, HE Dongjian, WANG Meili. Study of Plant Leaf Image Recognition Based on Improved Local Binary Pattern Algorithm [J]. Computer Engineering and Applications, 2021, 57(19): 228-234.
[14]	LI Jie, LI Miao, YUAN Xiguo. Detection Algorithm?of Pathogenic Microbes from Next-Generation Sequencing Data [J]. Computer Engineering and Applications, 2021, 57(19): 282-289.
[15]	GUO Hengguang, LIU Wenbiao, YU Renbo. Shape Feature Extraction Using Spike Function [J]. Computer Engineering and Applications, 2021, 57(18): 220-226.

Research on metadata driven ETL

元数据驱动ETL的研究

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics