Installing and Configuring ETL Framework

From Toolsverse Knowledge Base
Revision as of 23:33, 8 February 2015 by Maxlmus (Talk | contribs)

Jump to: navigation, search

Downloading

Look for clearly marked ETL Framework downloads. On Windows and OS X ETL Framework can be downloaded as an installer. Use archive downloads for other operating systems.

Installing

There are two ways to install ETL Framework: using installer or extracting files from the archive.

When using installer just execute it and follow instructions on the screen.

OS X installer is offered in a form of self-extracting DMG file. When prompted drag and drop ETL Framework folder to the Applications.

When installing from archive you will need to manually extract files using utility provided by operating system. All files are contained in the enclosing folder named etl-version.

Installing from archive will not add any entries to the Start menu, add desktop launchers or register the software in the registry.

Configuring

ETL Framework comes pre-configured to provide highest levels of performance and compatibility.

Framework requires a valid not expired license. Licenses are located under ETL_FRAMEWORK_HOME\license folder (*.lic files) or under ETL_FRAMEWORK_HOME\lib (*.jar files).

Note: ETL Framework comes with a 20-days evaluation license. It will expire after 20 days since the first use. You can a buy a commercial license which will never expire. License can be delivered as a lic or jar file. Using jars for licenses decreases dependency on external folders.

You can read more about licensing here.

Changing configuration properties

ETL framework uses etl.properties file located under ETL_FRAMEWORK_HOME/config folder to load configuration properties from. You can change default name of the configuration file by setting property config.file.name. Example:
-Dconfig.file.name=c:/etl/config/test.properties.

If file does not exist framework will use default values.

You can add any property to the file using format property_name=value. You can also completely ignore configuration file and set properties programmatically using System.setProperty(name, value) or by passing -D props to the Java program.

Some of the properties:

  • network.appupdateurl - the URL used to call auto-update Web service
  • network.appserverurl - the URL used to call ETL Web service

Example:

network.appserverurl=http://localhost:8080/dataexplorer/ide
network.appupdateurl=http://www.toolsverse.com/api/services/CheckForUpdates
# never change properties below
app.update.key=etl
app.name=etlprocess
app.title=ETL Framework

Configuring HTTP Proxy

Use your properties file (the default is ETL_FRAMEWORK_HOME/config/etl.properties) to configure HTTP proxy:

network.httpproxyhost=
network.httpproxyport=
network.httpproxyuser=
network.httpproxypassword=
# none, http, https, socks
network.httpproxytype=
# comma separated list of hosts to exclude, for example localhost,198.168.1.1. Can leave it blank 
network.httpproxyexclude=

Configuring EMail Notifications

If you are going to use email notifications you need to configure mailer. Use your properties file (the default is ETL_FRAMEWORK_HOME/config/etl.properties):

# example: smtp.mail.host=smtp.gmail.com                    
smtp.mail.host=
# example: smtp.mail.port=587 
smtp.mail.port=
# example: mailer@marketo.com
smtp.mail.user=
# example: passwd
smtp.mail.password=
# example: ops@marketo.com
smtp.mail.to=
# example: robot@marketo.com
smtp.mail.from=

Changing JVM Parameters

  1. Open ETL_FRAMEWORK_HOME\etlappstart.properties in your favorite text editor
  2. Change line containing app.vm.options
  3. Save

Changing Logging Parameters

  1. Open ETL_FRAMEWORK_HOME\config\log4j.properties in your favorite text editor
  2. Make a change
  3. Save

Adding JDBC drivers

Each copy of the ETL Framework comes with the following JDBC drivers:

  • DB2
  • Informix
  • Microsoft SQL Server
  • Sybase Advanced Server
  • MySQL
  • Oracle
  • PostgreSQL
  • Java DB (Derby)

They are located in the ETL_FRAMEWORK_HOME\jdbc folder. ETL Framework automatically recognizes (on start-up) and dynamically loads (when needed) jdbc drivers under this folder.

To install new jdbc driver just add a sub folder under ETL_FRAMEWORK_HOME\jdbc and put all required files there. Restart Data Explorer.

Alternatively, since JDBC drivers are just jar files you can simply put them in the ETL_FRAMEWORK_HOME\lib folder and they will be automatically loaded and added to the classpath.

Redistribution and managing dependencies

Configure ETL framework the way you like, exclude all unneeded components. You can find redistribution list and recommendations how to manage dependencies here.

Checking for updates

ETL Framework can check for new version and update itself if new version exists.

To check for new new version run ETL framework executable with -v parameter.

Example:

C:\projects\toolsverse\java\dist\etl>etl.exe -v
Toolsverse Etl Framework 4.0-19862. Use -? for help.

Checking for updates...
Update status: new version available

To auto update ETL framework run executable with -u parameter.

Example:

C:\projects\toolsverse\java\dist\etl>etl.exe -u
Toolsverse Etl Framework 4.0-19862. Use -? for help.

Manually updating

You can update framework to the new version manually:

  1. Download update
  2. Replace exiting files on files from the update