Toolsverse-Data management tools

ETL Scenario Examples

Basic stuff

    1. Migrate data from the source to destination
    2. Migrate data from the source to the specified destination table
    3. Stream data
    4. High performance data migration
    5. Automatically create tables and indexes
    6. Conrol how to commit transaction
    7. Map source columns to destination columns. Exclude some columns
    8. Map column with spaces in the name to the variable name
    9. Create variables with spaces in the name
    10. Add columns to the destination on the fly using SQL
    11. Add column to the destination on the fly using JavaScript
    12. Change column type
    13. Map Source to Destination if names are different
    14. Migrate data from source to destination when names are different
    15. Migrate data using specific ETL driver
    16. Using generic ETL driver together with specific database ETL driver
    17. Control flow of extracts and loads
    18. Scenario Variables
    19. Variable substitution
    20. Allowed ETL actions
    21. Use attributes to force required connections
    22. Disable creation of the temporary tables in the destination database
    23. Merge data
    24. Merge data using generic ETL driver
    25. Update data
    26. Delete data
    27. Conditional action (insert, delete or update)
    28. Inline SQL
    29. Conitionaly include sources and destinations using JavaScript
    30. Conitionaly include sources and destinations using SQL
    31. Destination Exceptions
    32. If/Then/Else
    33. On (event) actions
    34. Migrate data from multiple source connections
    35. Multiple destination connections
    36. Migrate data to the multiple destination connections in parallel threads
    37. Extract and load data in parallel threads
    38. Parallel extract and load with synchronization
    39. Inner scenarios
    40. Inner Scenarios with individual actions
    41. Inner + Main Scenarios
    42. Execute inner scenarios in parallel threads
    43. Conditionaly execute inner scenarios using JavaScript
    44. Conditionaly executes inner scenarios using SQL
    45. Execute inner scenarios in the loop using SQL
    46. Execute inner scenarios in the loop using JavaScript
    47. Using stored procedure and reference cursor to extract data

Column level transformations

    1. Column level transformations using JavaScript
    2. Column level transformations using SQL

Dataset transformations

    1. Order Data Set
    2. Transpose Matrix
    3. Filter Data Set
    4. Remove duplicates
    5. Intersect datasets
    6. Minus data sets
    7. Union data sets
    8. Join Data Sets
    9. Join multiple data sets
    10. Diff datasets
    11. Group by
    12. Denormalize data set
    13. Matrix transformation

Dimension transformations

    1. Add dimension
    2. Extract dimension

Data Migration

    1. High performance data migration
    2. Automatically create tables and indexes
    3. Migrate data from one db to another using diffrent table names
    4. Extract data from database and create files in any supported format
    5. Use named connections
    6. Migrate data using specific ETL driver
    7. Simple data migration
    8. Migrate data using specific parent ETL driver
    9. Migrate data to the multiple destination connections in parallel threads
    10. Migrate data using specific source ETL driver
    11. Migrate data in parallel threads with synchronization
    12. Stream data from source to destination

Common functions

    1. Get new column value for the destination table and keep referential integrity
    2. Use default functions instead of driver specific
    3. Calculate column value using JavaScript
    4. Exclude fields
    5. Assign scenario variable value to the column value

Common Tasks

    1. Validation using JavaScript
    2. Transformation using JavaScript
    3. Regex replacement
    4. SQL task
    5. Operating System command
    6. IO tasks
    7. XSL transformation
    8. Send email
    9. Before and after tasks

XML Files in the Toolsverse data set format

    1. Export from xml to database using pre-configured xml connection
    2. Export from xml to database using hard coded file name
    3. Export from xml to database with xsl transformation using hard coded xml and xsl file names
    4. Export from database table to xml using xsl transformation
    5. Extract data from database table and load into xml file using xsl transformation
    6. Load data from xml into database using hard coded file name
    7. Export data from database into XML data set file

XML Files

    1. Export from xml to database using pre-configured xml connection
    2. Export from xml to database using hard coded file name
    3. Load data from xml into database using hard coded file name
    4. Export data from database into XML file

Delimited and Fixed Length Text Files

    1. Export data from database table to delimited text file
    2. Export data from database table to the fixed lenth text file
    3. Export from delimited text file to the database using pre-configured connection
    4. Export from text file to database using hard coded file name
    5. Export from database to text file using hard coded file name
    6. Export from database to text file using pre configured text connection
    7. Export text file to database using pre-configured text connection. Calculate\exclude fields.
    8. Using properties to configure file format

JSON files

    1. Export from json to database using pre-configured json connection
    2. Export from json to database using hard coded file name
    3. Load data from json into database using hard coded file name
    4. Export data from database into JSON file

HTML files

    1. Export data from database into HTML file

PDF files

    1. Export data from database into PDF file

Excel

    1. Export from database to Excel using pre-configured excel connection
    2. Export from Excel to database using pre-configured Excel connection
    3. Export from Excel (1997-2003) to database using hard coded file name
    4. Export from Excel (2007) to database using hard coded file name
    5. Export from Excel (1997-2003) to database using ODBC
    6. Export from database to Excel (1997-2003) using ODBC
    7. Export from database to Excel (1997-2003) file using hard coded file name
    8. Export from database to Excel (2007) file using hard coded file name
    9. Export from database table to Excel (1997-2003) file
    10. Export from database table to Excel (2007) file

Readers and Writers

    1. Xml to Excel 2003 to Excel 2007 to Text to Database

Oracle specific functions and transformations

    1. Migrate data, including CLOBs and BLOBs
    2. Get new column value for the destination table using Oracle sequence and keep referential integrity
    3. Using PL/SQL in the etl scenario
    4. Create PL/SQL function and use it in the etl scenario. Use Oracle global variable
    5. Using Oracle cursors in the etl scenario
    6. Using sqlplus COPY command to copy data from one table to another
    7. Oracle Extract using slplus and Load using sql*loader
    8. Oracle merge, update, delete

DB2 specific functions and transformations

    1. Migrate data including BLOBs and CLOBs
    2. Get new column value using DB2 autoincrement column and sequence and keep referential integrity
    3. Using SQL/PL in the etl scenario
    4. Create SQL/PL function and use it in the etl scenario. Use DB2 global variable
    5. Using DB2 cursors in the etl scenario
    6. DB2 Extract and Load using ADMIN_CMD stored procedure
    7. Db2 merge, update, delete

MySql specific functions and transformations

    1. Migrate data including CLOBs and BLOBs
    2. Get new column value using MySql autoincrement column and keep referential integrity
    3. Using MySql procedural language in the etl scenario
    4. Create MySql function and use it in the etl scenario. Use Scenario global variable
    5. Using MySql cursors in the etl scenario
    6. MySql Extract using select INTO OUTFILE and Load using LOAD DATA
    7. MySql merge, update, delete

PostgreSQL specific functions and transformations

    1. Migrate data including BLOBs and CLOBs
    2. Get new column value using PostgreSQL sequence and serial column and keep referential integrity
    3. Using PL/pgSQL in the etl scenario
    4. Create pgSQL function and use it in the etl scenario. Use Scenario global variable
    5. Using PostgreSQL cursors in the etl scenario
    6. PostgreSQL Extract and Load using COPY
    7. PostgreSQL merge, update, delete

MS SQL Server specific functions and transformations

    1. Migrate data including BLOBs and CLOBs
    2. Get new column value using Ms Sql autoincrement and SQL and keep referential integrity
    3. Using TransactSQL in the etl scenario
    4. Create TransactSQL function and use it in the etl scenario. Use Scenario global variable
    5. Using Ms Sql cursors in the etl scenario
    6. Ms Sql Extract using bcp and Load using bulk insert and bcp
    7. Ms Sql merge, update, delete

Sybase SQL Server specific functions and transformations

    1. Migrate data including BLOBs and CLOBs
    2. Get new column value using Sybase autoincrement and SQL and keep referential integrity
    3. Using Sybase TSQL in the etl scenario
    4. Create TSQL function and use it in the etl scenario. Use Sybase app_context
    5. Using Sybase cursors in the etl scenario
    6. Sybase Extract and load using bcp
    7. Sybase, merge, update, delete

Informix specific functions and transformations

    1. Move blobs, clobs and other types of data. Create tables if needed
    2. Get new column value using Informix serial column and sequence and keep referential integrity
    3. Using Informix SPL in the etl scenario
    4. Create SPL function and use it in the etl scenario. Use Informix global variable
    5. Using Informix cursors in the etl scenario
    6. Informix Extract and Load using DbAccess
    7. Informix merge, update, delete