Difference between revisions of "Installing and Configuring ETL Framework"
(38 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
== Downloading == | == Downloading == | ||
− | Look for clearly marked ETL Framework downloads. On Windows and OS X | + | Look for clearly marked ETL Framework downloads. On Windows and OS X ETL Framework can be downloaded as an installer. Use archive downloads for other operating systems. |
== Installing == | == Installing == | ||
Line 18: | Line 18: | ||
ETL Framework comes pre-configured to provide highest levels of performance and compatibility. | ETL Framework comes pre-configured to provide highest levels of performance and compatibility. | ||
+ | |||
+ | Framework requires a valid not expired license. Licenses are located under ETL_FRAMEWORK_HOME\license folder (*.lic files) or under ETL_FRAMEWORK_HOME\lib (*.jar files). | ||
+ | |||
+ | '''Note''': ETL Framework comes with a 20-days evaluation license. It will expire after 20 days since the first use. You can a buy a commercial license which will never expire. License can be delivered as a lic or jar file. Using jars for licenses decreases dependency on external folders. | ||
+ | |||
+ | You can read more about licensing [http://www.toolsverse.com/support/faq/ here]. | ||
+ | |||
+ | === Changing configuration properties === | ||
+ | |||
+ | ETL framework uses etl.properties file located under ETL_FRAMEWORK_HOME/config folder to load configuration properties from. You can change default name of the configuration file by setting property config.file.name. Example: <pre>-Dconfig.file.name=c:/etl/config/test.properties.</pre> | ||
+ | |||
+ | If file does not exist framework will use default values. | ||
+ | |||
+ | You can add any property to the file using format property_name=value. You can also completely ignore configuration file and set properties programmatically using System.setProperty(name, value) or by passing -D props to the Java program. | ||
+ | |||
+ | Some of the properties: | ||
+ | * network.appupdateurl - the URL used to call auto-update Web service | ||
+ | * network.appserverurl - the URL used to call ETL Web service | ||
+ | |||
+ | Example: | ||
+ | |||
+ | <pre> | ||
+ | network.appserverurl=http://localhost:8080/dataexplorer/ide | ||
+ | network.appupdateurl=http://www.toolsverse.com/api/services/CheckForUpdates | ||
+ | # never change properties below | ||
+ | app.update.key=etl | ||
+ | app.name=etlprocess | ||
+ | app.title=ETL Framework | ||
+ | </pre> | ||
+ | |||
+ | You can pass XML configuration file name (or any other -D property) as a command line argument: | ||
+ | <pre>-Detl.config.name=import_data_config.xml</pre> | ||
+ | |||
+ | === Configuring HTTP Proxy === | ||
+ | |||
+ | Use your properties file (the default is ETL_FRAMEWORK_HOME/config/etl.properties) to configure HTTP proxy: | ||
+ | |||
+ | <pre> | ||
+ | network.httpproxyhost= | ||
+ | network.httpproxyport= | ||
+ | network.httpproxyuser= | ||
+ | network.httpproxypassword= | ||
+ | # none, http, https, socks | ||
+ | network.httpproxytype= | ||
+ | # comma separated list of hosts to exclude, for example localhost,198.168.1.1. Can leave it blank | ||
+ | network.httpproxyexclude= | ||
+ | </pre> | ||
+ | |||
+ | === Configuring EMail Notifications === | ||
+ | |||
+ | If you are going to use email notifications you need to configure mailer. Use your properties file (the default is ETL_FRAMEWORK_HOME/config/etl.properties): | ||
+ | |||
+ | <pre> | ||
+ | # example: smtp.mail.host=smtp.gmail.com | ||
+ | smtp.mail.host= | ||
+ | # example: smtp.mail.port=587 | ||
+ | smtp.mail.port= | ||
+ | # example: mailer@gmail.com | ||
+ | smtp.mail.user= | ||
+ | # example: passwd | ||
+ | smtp.mail.password= | ||
+ | # example: ops@gmail.com | ||
+ | smtp.mail.to= | ||
+ | # example: robot@gmail.com | ||
+ | smtp.mail.from= | ||
+ | </pre> | ||
=== Changing JVM Parameters === | === Changing JVM Parameters === | ||
− | # Open ETL_FRAMEWORK_HOME\ | + | # Open ETL_FRAMEWORK_HOME\etlappstart.properties in your favorite text editor |
# Change line containing app.vm.options | # Change line containing app.vm.options | ||
# Save | # Save | ||
+ | |||
+ | === Changing Logging Parameters === | ||
+ | |||
+ | # Open ETL_FRAMEWORK_HOME\config\log4j.properties in your favorite text editor | ||
+ | # Make a change | ||
+ | # Save | ||
+ | |||
+ | === Adding JDBC drivers === | ||
+ | |||
+ | Each copy of the ETL Framework comes with the following JDBC drivers: | ||
+ | |||
+ | * DB2 | ||
+ | * Informix | ||
+ | * Microsoft SQL Server | ||
+ | * Sybase Advanced Server | ||
+ | * MySQL | ||
+ | * Oracle | ||
+ | * PostgreSQL | ||
+ | * Java DB (Derby) | ||
+ | |||
+ | They are located in the ETL_FRAMEWORK_HOME\jdbc folder. ETL Framework automatically recognizes (on start-up) and dynamically loads (when needed) jdbc drivers under this folder. | ||
+ | |||
+ | To install new jdbc driver just add a sub folder under ETL_FRAMEWORK_HOME\jdbc and put all required files there. Restart Data Explorer. | ||
+ | |||
+ | Alternatively, since JDBC drivers are just jar files you can simply put them in the ETL_FRAMEWORK_HOME\lib folder and they will be automatically loaded and added to the classpath. | ||
+ | |||
+ | == Redistribution and managing dependencies == | ||
+ | |||
+ | Configure ETL framework the way you like, exclude all unneeded components. You can find redistribution list and recommendations how to manage dependencies [[Redistribution and managing dependencies|here]]. | ||
+ | |||
+ | == Checking for updates == | ||
+ | |||
+ | ETL Framework can check for new version and update itself if new version exists. | ||
+ | |||
+ | To check for new new version run ETL framework executable with -v parameter. | ||
+ | |||
+ | Example: | ||
+ | |||
+ | <pre> | ||
+ | C:\projects\toolsverse\java\dist\etl>etl.exe -v | ||
+ | Toolsverse Etl Framework 4.0-19862. Use -? for help. | ||
+ | |||
+ | Checking for updates... | ||
+ | Update status: new version available | ||
+ | </pre> | ||
+ | |||
+ | To auto update ETL framework run executable with -u parameter. | ||
+ | |||
+ | Example: | ||
+ | |||
+ | <pre> | ||
+ | C:\projects\toolsverse\java\dist\etl>etl.exe -u | ||
+ | Toolsverse Etl Framework 4.0-19862. Use -? for help. | ||
+ | </pre> | ||
+ | |||
+ | == Manually updating == | ||
+ | |||
+ | You can update framework to the new version manually: | ||
+ | |||
+ | # Download update | ||
+ | # Replace exiting files on files from the update |
Latest revision as of 13:40, 9 February 2015
Contents
Downloading
Look for clearly marked ETL Framework downloads. On Windows and OS X ETL Framework can be downloaded as an installer. Use archive downloads for other operating systems.
Installing
There are two ways to install ETL Framework: using installer or extracting files from the archive.
When using installer just execute it and follow instructions on the screen.
OS X installer is offered in a form of self-extracting DMG file. When prompted drag and drop ETL Framework folder to the Applications.
When installing from archive you will need to manually extract files using utility provided by operating system. All files are contained in the enclosing folder named etl-version.
Installing from archive will not add any entries to the Start menu, add desktop launchers or register the software in the registry.
Configuring
ETL Framework comes pre-configured to provide highest levels of performance and compatibility.
Framework requires a valid not expired license. Licenses are located under ETL_FRAMEWORK_HOME\license folder (*.lic files) or under ETL_FRAMEWORK_HOME\lib (*.jar files).
Note: ETL Framework comes with a 20-days evaluation license. It will expire after 20 days since the first use. You can a buy a commercial license which will never expire. License can be delivered as a lic or jar file. Using jars for licenses decreases dependency on external folders.
You can read more about licensing here.
Changing configuration properties
ETL framework uses etl.properties file located under ETL_FRAMEWORK_HOME/config folder to load configuration properties from. You can change default name of the configuration file by setting property config.file.name. Example:-Dconfig.file.name=c:/etl/config/test.properties.
If file does not exist framework will use default values.
You can add any property to the file using format property_name=value. You can also completely ignore configuration file and set properties programmatically using System.setProperty(name, value) or by passing -D props to the Java program.
Some of the properties:
- network.appupdateurl - the URL used to call auto-update Web service
- network.appserverurl - the URL used to call ETL Web service
Example:
network.appserverurl=http://localhost:8080/dataexplorer/ide network.appupdateurl=http://www.toolsverse.com/api/services/CheckForUpdates # never change properties below app.update.key=etl app.name=etlprocess app.title=ETL Framework
You can pass XML configuration file name (or any other -D property) as a command line argument:
-Detl.config.name=import_data_config.xml
Configuring HTTP Proxy
Use your properties file (the default is ETL_FRAMEWORK_HOME/config/etl.properties) to configure HTTP proxy:
network.httpproxyhost= network.httpproxyport= network.httpproxyuser= network.httpproxypassword= # none, http, https, socks network.httpproxytype= # comma separated list of hosts to exclude, for example localhost,198.168.1.1. Can leave it blank network.httpproxyexclude=
Configuring EMail Notifications
If you are going to use email notifications you need to configure mailer. Use your properties file (the default is ETL_FRAMEWORK_HOME/config/etl.properties):
# example: smtp.mail.host=smtp.gmail.com smtp.mail.host= # example: smtp.mail.port=587 smtp.mail.port= # example: mailer@gmail.com smtp.mail.user= # example: passwd smtp.mail.password= # example: ops@gmail.com smtp.mail.to= # example: robot@gmail.com smtp.mail.from=
Changing JVM Parameters
- Open ETL_FRAMEWORK_HOME\etlappstart.properties in your favorite text editor
- Change line containing app.vm.options
- Save
Changing Logging Parameters
- Open ETL_FRAMEWORK_HOME\config\log4j.properties in your favorite text editor
- Make a change
- Save
Adding JDBC drivers
Each copy of the ETL Framework comes with the following JDBC drivers:
- DB2
- Informix
- Microsoft SQL Server
- Sybase Advanced Server
- MySQL
- Oracle
- PostgreSQL
- Java DB (Derby)
They are located in the ETL_FRAMEWORK_HOME\jdbc folder. ETL Framework automatically recognizes (on start-up) and dynamically loads (when needed) jdbc drivers under this folder.
To install new jdbc driver just add a sub folder under ETL_FRAMEWORK_HOME\jdbc and put all required files there. Restart Data Explorer.
Alternatively, since JDBC drivers are just jar files you can simply put them in the ETL_FRAMEWORK_HOME\lib folder and they will be automatically loaded and added to the classpath.
Redistribution and managing dependencies
Configure ETL framework the way you like, exclude all unneeded components. You can find redistribution list and recommendations how to manage dependencies here.
Checking for updates
ETL Framework can check for new version and update itself if new version exists.
To check for new new version run ETL framework executable with -v parameter.
Example:
C:\projects\toolsverse\java\dist\etl>etl.exe -v Toolsverse Etl Framework 4.0-19862. Use -? for help. Checking for updates... Update status: new version available
To auto update ETL framework run executable with -u parameter.
Example:
C:\projects\toolsverse\java\dist\etl>etl.exe -u Toolsverse Etl Framework 4.0-19862. Use -? for help.
Manually updating
You can update framework to the new version manually:
- Download update
- Replace exiting files on files from the update