Chapter 3. ETL features

Table of Contents

1. Loaders, readers and writers
1.1. Choosing the right loader
2. The DB loaders, readers and writers
2.1. The DB reader
2.2. The DB writer
3. The standard DB loader (TEDWStandardDBLoader)
3.1. The DB cursor reader (TEDWDBCursorReader)
3.2. Introduction to data maps (TEDWDataMap)
3.2.1. The PascalScript data pipe
3.2.2. Data map item attributes
3.3. The DB record writer (TEDWDBRecordWriter)
3.4. Using table writers (TEDWDBTableWriter)
3.4.1. The base strategy
3.4.2. The Insert strategy
3.4.3. The Update strategy
3.4.4. The Merge strategy
3.4.5. The Delete strategy
4. The hierarchy DB loader (TEDWHierarchyDBLoader)
4.1. The DB cursor hierarchy reader (TEDWDBCursorHirearchyReader)
5. The standard XML DB loader (TEDWStandardXMLDBLoader)
5.1. The SAX structured CDS reader (TEDWSAXStructuredCDSReader)
6. The standard dataset DB loaders (TEDWStandardADORecordsetDBLoader and TEDWStandardCDSDBLoader)
6.1. The dataset readers (TEDWADORecordSetReader and TEDWCDSReader)
7. The text loaders (TEDWStandardFixedWidthTextLoader and TEDWStandardCSVTextLoader)
7.1. The fixed-width text reader (TEDWFixedWidthTextReader)
7.2. The comma-separated-values reader (TEDWCSVTextReader)
8. Commonly used data packet types and transformers
8.1. The basic data item class
8.2. The data item data packet
8.3. The data row data packet
8.4. The DB params data packet
8.5. The dataset-based data packets
9. The SQL agents
9.1. The general-purpose SQL agent (TEDWSQLAgent)
9.2. The empty table SQL agent (TEDWEmptyTableAgent)
9.3. The SQL select agent (TEDWSelectAgent)
10. Miscellaneous agents
10.1. The shell agent (TEDWShellAgent)

ETL (Extract/Transform/Load) comprises a set of techniques and processes that help getting data from various kinds of sources into the Data Warehouse. This chapter explains EDW's ETL features in some detail, documents all the pre-defined agents and their parameters, and gives usage examples.