Products > ETL Framework > ETL Scenario Examples
Toolsverse Data Integration

ETL Framework

ETL Scenario Examples

  1. Basic
    1. New Scenario
    2. Move data
    3. Migrate tables
    4. Stream data
    5. Automatically commit transaction on each executed SQL block
    6. Map Source to Destination if names are different
    7. Destination Table name different from Destination name
    8. Map source fields to destination fields. Exclude some fields
    9. Use specific ETL driver
    10. Using generic jdbc ETL driver together with specific database ETL driver. This ETL code wiil not generate stored procedures
    11. Destination scope
    12. Scenario Variables
    13. Variable substitution
    14. Allowed Scenario Actions
    15. Use attributes to force required connections
    16. Disable creattion of the temporary tables in the destination database
    17. Merge
    18. Merge using generic jdbc ETL driver (destination = PostgreSQL)
    19. Update
    20. Delete
    21. Inline SQL
    22. Source and Destination Conditions (JavaScript)
    23. Source and Destination Conditions (SQL)
    24. Destination Exceptions
    25. If/Then/Else for Destination
    26. Debuging
    27. Multiple source connections
    28. Multiple destination connections
    29. Multiple destination connections, each in parallel thread
    30. Parallel extract and load
    31. Parallel extract and load with synchronization
    32. Inner scenarios
    33. Inner Scenarios with individual actions
    34. Inner + Main Scenarios
    35. Parallel Inner scenarios
    36. Conditional Inner Scenarios (JavaScript)
    37. Conditional Inner Scenarios (SQL)
    38. Execute inner scenarios in the loop (SQL)
    39. Execute inner scenarios in the loop (JavaScript)
  2. Default Functions
    1. Get new column value for the destination table and keep referential integrity
    2. Use default functions instead of driver specific
    3. Calculate field value (JavaScript)
    4. Exclude fields
    5. Assign scenario variable value to the field value
  3. Transformations
    1. Order Data Set
    2. Transpose Matrix
    3. Filter Data Set
    4. Remove Duplicates
    5. Minus one data set from another
    6. Union of the Data Sets
    7. Join Data Sets
    8. Pivot Table
    9. Denormalize Data Set
  4. Common Tasks
    1. Validation (JavaScript)
    2. Transformation (JavaScript)
    3. Regex replacement
    4. SQL task
    5. Operating System command
    6. IO tasks
    7. XSL transformation
    8. Before and after tasks
  5. XML Files
    1. Export from xml to database using pre-configured xml connection
    2. Export from xml to database using hard coded file name
    3. Export from xml to database with xsl transformation using hard coded xml and xsl file names
    4. Save database table as xml using xsl transformation
    5. Extract data from database table and load into xml file using xsl transformation
    6. Load data from xml into database using hard coded file name
    7. Using properties to configure date+time formats
  6. Text Files
    1. Save database table as delimited text file
    2. Save database table as fixed lenth text file
    3. Export from delimited text file to database using pre-configured text connection
    4. Export from text file to database using hard coded file name
    5. Export from database to text file using hard coded file name
    6. Export from database to text file using pre configured text connection
    7. Export text file to database using pre-configured text connection. Calculate and exclude some fields
    8. Using properties to configure file format
  7. Excel
    1. Export from database to Excel using pre-configured excel connection
    2. Export from Excel to database using pre-configured Excel connection
    3. Export from Excel (1997-2003) to database using hard coded file name
    4. Export from Excel (2007) to database using hard coded file name
    5. Export from Excel (1997-2003) to database using ODBC
    6. Export from database to Excel (1997-2003) using ODBC
    7. Export from database to Excel (1997-2003) file using hard coded file name
    8. Export from database to Excel (2007) file using hard coded file name
    9. Save database table as Excel (1997-2003) file
    10. Save database table as Excel (2007) file
  8. Readers and Writers
    1. Xml to Excel 2003 to Excel 2007 to Text to Database
  9. Oracle
    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new column value for the destination table using Oracle sequence and keep referential integrity
    3. PL/SQL in the etl scenario
    4. Create PL/SQL function and use it in the etl scenario. Use Oracle global variable
    5. Use Oracle cursors in the etl scenario
    6. Use sqlplus COPY command to copy data from one table to another. Possible in different databases
    7. Oracle Extract using sqlplus and Load using sql*loader
    8. Oracle merge, update, delete
  10. DB2
    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new column value using Db2 autoincrement column and sequence and keep referential integrity
    3. SQL PL in the etl scenario
    4. Create SQL PL function and use it in the etl scenario. Use DB2 global variable
    5. Use Db2 cursors in the etl scenario
    6. Db2 Extract and Load using ADMIN_CMD stored procedure
    7. Db2 merge, update, delete
  11. MySQL
    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new column value using MySql autoincrement column and keep referential integrity
    3. MySQL procedural language in the etl scenario
    4. Create MySQL function and use it in the etl scenario. Use Scenario global variable
    5. Use MySQL cursors in the etl scenario
    6. MySQL Extract using select INTO OUTFILE and Load using LOAD DATA
    7. MySQL merge, update, delete
  12. PostgreSQL
    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new primary key using PostgreSQL sequence and serial column and keep referential integrity
    3. PL/pgSQL in the etl scenario
    4. Create pgSQL function and use it in the etl scenario. Use Scenario global variable
    5. Use PostgreSQL cursors in the etl scenario
    6. PostgreSQL Extract and Load using COPY
    7. PostgreSQL merge, update, delete
  13. MS SQL Server
    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new column value using Ms Sql autoincrement and SQL and keep referential integrity
    3. TransactSQL in the etl scenario
    4. Create TransactSQL function and use it in the etl scenario. Use Scenario global variable
    5. Use MS SQL cursors in the etl scenario
    6. MS SQL Extract using bcp and Load using bulk insert and bcp
    7. MS SQL merge, update, delete
  14. Sybase Adaptive Server
    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new column value using Sybase autoincrement and SQL and keep referential integrity
    3. Sybase TSQL in the etl scenario
    4. Create TSQL function and use it in the etl scenario. Use Sybase app_context
    5. Use Sybase cursors in the etl scenario
    6. Sybase Extract and load using bcp
    7. Sybase, merge, update, delete
  15. Informix
    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new column value using Informix serial column and sequence and keep referential integrity
    3. Informix SPL in the ETL scenario
    4. Create SPL function and use it in the etl scenario. Use Informix global variable
    5. Use Informix cursors in the etl scenario
    6. Informix Extract and Load using DbAccess
    7. Informix merge, update, delete