Difference between revisions of "Redistribution and managing dependencies"
From Toolsverse Knowledge Base
m (Maxlmus moved page Redistribution and Dependencies to Redistribution and managing dependencies) |
|||
Line 1: | Line 1: | ||
− | == | + | == Redistribution list == |
{| class="wikitable" | {| class="wikitable" | ||
Line 322: | Line 322: | ||
toolsverse-etl-sqlserver.jar | toolsverse-etl-sqlserver.jar | ||
</pre> | </pre> | ||
+ | |||
+ | == Managing Dependencies == | ||
+ | |||
+ | Most jars under '''lib''' are required unless you don't need a particular functionality, for example reading and writing Excel files. All jars under '''lib''' folder are automatically added to the classpath on the start-up. Jars under '''plugin''' folder are optional and loaded based on rules and available licenses. | ||
+ | |||
+ | You can reduce number of folders ETL framework depends on: | ||
+ | * '''lib''' folder is required | ||
+ | * Move all files from the '''plugin''' folder to '''lib'''. | ||
+ | * Package existing licenses (*.lic files) in the '''license''' folder into jar file and copy jar into '''lib''' folder. Name of the jar file doesn't matter but lic files must be located in the '''com/toolsverse/license''' folder inside jar. | ||
+ | * Files in the '''config''' folder and '''config''' folder itself are not required. You can pass properties to the framework using -Dproperty_name=value. | ||
+ | * '''data''' folder is used for staging temporary files, as a root for '''scenario''' folder, etc. | ||
+ | |||
+ | Look at the src/com/toolsverse/config/SystemConfig.java how ETL frameworks |
Revision as of 16:14, 5 August 2014
Redistribution list
Folder | File | Required | Needed for | Depends on |
---|---|---|---|---|
home | Yes | Root folder | ||
etl.exe | No | Windows executable | ||
etl.app | No | Mac application bundle | ||
etl.sh | No | Unix/Linux shell script | ||
etlappstart.properties | No | JVM properties | ||
etl.jar | Yes | Main executable jar | ||
home\config | No | Default configuration folder | ||
etl.properties | No | Configuration properties | ||
etl_config.xml | No | Default ETL configuration file | ||
log4j.properties | No | Log configuration | ||
home\license | No | Folder for licenses | ||
*.lic | No | Licenses (ETL framework will also look at license.jar under home\lib folder) |
| |
home\data | No | Default folder for data | ||
home\data\scenario | No | Default folder for ETL scenarios | ||
home\data\schema | No | Default folder for XLT schema files | ||
dataset2webrowset.xsl | No | XSL schema file for Toolseverse XML dataset to WebRowSet XML transformation | ||
webrowset2dataset.xsl | No | XSL schema file for WebRowSet XML to Toolseverse XML dataset transformation | ||
xmldataset.xsd | No | Schema file for Toolsverse XML dataset file format | ||
home\lib | No | Folder for core and third party jar files. Files from this folder automatically added to the classpath | ||
antlr-3.0.1.jar | Yes | Core third party file | ||
antlr.jar | Yes | Core third party file | ||
commons-beanutils-1.9.1.jar | Yes | Core third party file | ||
commons-codec-1.6.jar | Yes | Core third party file | ||
commons-collections4-4.0.jar | Yes | Core third party file | ||
commons-dbcp-1.4.jar | Yes | Core third party file | ||
commons-logging-1.1.3.jar | Yes | Core third party file | ||
commons-math3-3.0.jar | Yes | Core third party file | ||
commons-net-2.0.jar | Yes | Core third party file | ||
commons-pool-1.5.4.jar | Yes | Core third party file | ||
dom4j-1.6.1.jar | Yes | Core third party file | ||
httpclient-4.3.3.jar | Yes | Core third party file | ||
httpclient-cache-4.3.3.jar | Yes | Core third party file | ||
httpcore-4.3.2.jar | Yes | Core third party file | ||
httpmime-4.3.3.jar | Yes | Core third party file | ||
javassist.jar | Yes | Core third party file | ||
json-io.jar | Yes | Core third party file | ||
log4j-1.2.16.jar | Yes | Core third party file | ||
rsyntaxtextarea.jar | Yes | Core third party file | ||
RSyntaxTextArea.License.txt | Yes | Core third party file | ||
saxon9he.jar | Yes | Core third party file | ||
xercesImpl.jar | Yes | Core third party file | ||
xmlbeans-2.3.0.jar | Yes | Core third party file | ||
toolsverse-core.jar | Yes | Toolsverse core file | ||
toolsverse-etl-common.jar | Yes | Toolsverse core file | ||
toolsverse-etl-connector-sql.jar | Yes | Toolsverse core file | ||
toolsverse-etl-connector.jar | Yes | Toolsverse core file | ||
toolsverse-etl-core.jar | Yes | Toolsverse core file | ||
toolsverse-etl-driver.jar | Yes | Toolsverse core file | ||
toolsverse-etl-metadata.jar | Yes | Toolsverse core file | ||
toolsverse-io.jar | Yes | Toolsverse core file | ||
toolsverse-license.jar | Yes | Toolsverse core file | ||
toolsverse-mvc.jar | Yes | Toolsverse core file | ||
toolsverse-security.jar | Yes | Toolsverse core file | ||
toolsverse-service.jar | Yes | Toolsverse core file | ||
toolsverse-storage.jar | Yes | Toolsverse core file | ||
toolsverse-updater.jar | Yes | Toolsverse core file |
| |
derby.jar | No | Derby DB JDBC driver | ||
iText-4.2.0-com.itextpdf.jar | No | Read and write PDF files | ||
javax.json-1.0.4.jar | No | Read and write JSON files | ||
javax.mail.jar | No | Send emails | ||
jsch-0.1.43.jar | No | SFTP client | ||
poi-3.8-20120326.jar | No | Read and write Excel files | ||
poi-ooxml-3.8-20120326.jar | No | Read and write Excel files | ||
poi-ooxml-schemas-3.8-20120326.jar | No | Read and write Excel files | ||
servlet-api.jar | No | Required if you are going to call Web services | ||
socks.jar | No | Socks proxy used together with FTP | ||
home\plugin | No | Folder for plugins. In other to use files in this folder you must have a valid not expired license | ||
toolsverse-etl-connector-excel.jar | No | Required if you are going to read and write Excel files | toolsverse-etl-connector-json.jar | |
toolsverse-etl-connector-html.jar | No | Required if you are going to read and write HTML files | ||
toolsverse-etl-connector-json.jar | No | Required if you are going to read and write JSON files | ||
toolsverse-etl-connector-pdf.jar | No | Required if you are going to read and write PDF files | ||
toolsverse-etl-connector-text.jar | No | Required if you are going to read and write CVS and fixed-length files | toolsverse-etl-connector-json.jar, toolsverse-etl-connector-xml.jar | |
toolsverse-etl-connector-xml.jar | No | Required if you are going to read and write XML files | toolsverse-etl-connector-json.jar | |
toolsverse-etl-db2.jar | No | Required if you are going to work with DB2 databases | ||
toolsverse-etl-informix.jar | No | Required if you are going to work with Informix databases | ||
toolsverse-etl-oracle.jar | No | Required if you are going to work with Oracle databases | ||
toolsverse-etl-mysql.jar | No | Required if you are going to work with MySQL databases | ||
toolsverse-etl-postgres.jar | No | Required if you are going to work with Postgres databases | ||
toolsverse-etl-sybase.jar | No | Required if you are going to work with Sybase ASE databases | ||
home\jdbc | No | Default folder for JDBC drivers |
|
Example
home etl.exe etl.app etl.jar etl.sh etlappstart.properties config etl.properties etl_config.xml log4j.properties license *.lic data scenario schema dataset2webrowset.xsl webrowset2dataset.xsl xmldataset.xsd lib antlr-3.0.1.jar antlr.jar commons-beanutils-1.9.1.jar commons-codec-1.6.jar commons-collections4-4.0.jar commons-dbcp-1.4.jar commons-logging-1.1.3.jar commons-math3-3.0.jar commons-net-2.0.jar commons-pool-1.5.4.jar derby.jar dom4j-1.6.1.jar httpclient-4.3.3.jar httpclient-cache-4.3.3.jar httpcore-4.3.2.jar httpmime-4.3.3.jar iText-4.2.0-com.itextpdf.jar javassist.jar javax.json-1.0.4.jar javax.mail.jar jsch-0.1.43.jar json-io.jar log4j-1.2.16.jar poi-3.8-20120326.jar poi-ooxml-3.8-20120326.jar poi-ooxml-schemas-3.8-20120326.jar rsyntaxtextarea.jar RSyntaxTextArea.License.txt saxon9he.jar servlet-api.jar socks.jar toolsverse-core.jar toolsverse-etl-aliasreader.jar toolsverse-etl-common.jar toolsverse-etl-connector-sql.jar toolsverse-etl-connector.jar toolsverse-etl-core.jar toolsverse-etl-driver.jar toolsverse-etl-metadata.jar toolsverse-io.jar toolsverse-license.jar toolsverse-mvc.jar toolsverse-security.jar toolsverse-service.jar toolsverse-storage.jar toolsverse-updater.jar xercesImpl.jar xmlbeans-2.3.0.jar plugin toolsverse-etl-connector-excel.jar toolsverse-etl-connector-html.jar toolsverse-etl-connector-json.jar toolsverse-etl-connector-pdf.jar toolsverse-etl-connector-text.jar toolsverse-etl-connector-xml.jar toolsverse-etl-db2.jar toolsverse-etl-informix.jar toolsverse-etl-mysql.jar toolsverse-etl-oracle.jar toolsverse-etl-postgres.jar toolsverse-etl-sqlserver.jar
Managing Dependencies
Most jars under lib are required unless you don't need a particular functionality, for example reading and writing Excel files. All jars under lib folder are automatically added to the classpath on the start-up. Jars under plugin folder are optional and loaded based on rules and available licenses.
You can reduce number of folders ETL framework depends on:
- lib folder is required
- Move all files from the plugin folder to lib.
- Package existing licenses (*.lic files) in the license folder into jar file and copy jar into lib folder. Name of the jar file doesn't matter but lic files must be located in the com/toolsverse/license folder inside jar.
- Files in the config folder and config folder itself are not required. You can pass properties to the framework using -Dproperty_name=value.
- data folder is used for staging temporary files, as a root for scenario folder, etc.
Look at the src/com/toolsverse/config/SystemConfig.java how ETL frameworks