The ETL Framework natively supports SOAP Web services over HTTP. You will need to:

# Set source or destination connection to the one linked to the SOAP endpoint
# Define SOAP request in the connection CDATA attribute

<syntaxhighlight lang="xml">
<connection alias="Cars">
<url>http://www.vendor.com/soapendpoint</url>
<CDATA><![CDATA[<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.vendor.com/soapendpoint/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SOAP-ENV:Header>
<ns1:AuthenticationHeader>
<vendorUserId>{requestUserId}</mktowsUserId>
<vendorSignature>{requestSignature}</requestSignature>
<vendorTimestamp>{requestTimestamp}</requestTimestamp>
</ns1:AuthenticationHeader>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<ns1:paramsGetMultipleCars>
<leadSelector xsi:type="ns1:KeySelector">
<keyType>MAKE</keyType>
<keyValues>
<stringItem>Ford</stringItem>
<stringItem>Acura</stringItem>
</keyValues>
</leadSelector>
<batchSize>100</batchSize>
</ns1:paramsGetMultipleCars>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

]]></CDATA>
<params>method=post;exceptions=500;file=false;header_Content-Type=text/xml</params>
<connector>com.toolsverse.etl.connector.xml.XmlObjectConnector</connector>
<transport>com.toolsverse.io.HttpProcessor</transport>
</connection>
</syntaxhighlight>

You can tokenize any part of the CDATA or URL using {token}. In the example above there are 3 tokens:
<pre>
<vendorUserId>{requestUserId}</mktowsUserId>
<vendorSignature>{requestSignature}</requestSignature>
<vendorTimestamp>{requestTimestamp}</requestTimestamp>
</pre>

Simple example below demonstrates how to set tokens, link source to the SOAP and point and transform result web service execution into the destination:

<syntaxhighlight lang="xml">
<?xml version="1.0" encoding="UTF-8" ?>
<scenario>
<name>SOAP</name>
<script>soap</script>
<driver name="com.toolsverse.etl.driver.GenericFileDriver" />
<beforetasks>
<task noconnection="true">
<name>security</name>
<class>com.toolsverse.etl.core.task.common.ScriptTask</class>
<code>
<![CDATA[
importPackage(java.text);
importPackage(java.util);

importPackage(javax.crypto);
importPackage(javax.crypto.spec);

importPackage(org.apache.commons.codec.binary);

importPackage(com.toolsverse.config);

var vendorUserId = SystemConfig.instance().getSystemProperty('requestUserId');
var vendorSecretKey = SystemConfig.instance().getSystemProperty('vendorSecretKey');

var df = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ssZ");
var text = df.format(new java.util.Date());
var requestTimestamp = text.substring(0, 22) + ":" + text.substring(22);
var encryptString = requestTimestamp + vendorUserId;

var secretKey = new SecretKeySpec(new java.lang.String(vendorSecretKey).getBytes(), "HmacSHA1");
var mac = Mac.getInstance("HmacSHA1");
mac.init(secretKey);
var rawHmac = mac.doFinal(new java.lang.String(encryptString).getBytes());
var hexChars = Hex.encodeHex(rawHmac);
var signature = new java.lang.String(hexChars);

SystemConfig.instance().setSystemProperty('requestSignature', signature);
SystemConfig.instance().setSystemProperty('requestTimestamp', requestTimestamp);
SystemConfig.instance().setSystemProperty('requestUserId', vendorUserId);
]]>
</code>
<variables>
<LANG value="JavaScript" />
</variables>
</task>
</beforetasks>
<sources>
<source>
<name>cars</name>
</source>
</sources>
<destinations>
<destination >
<name>cars</name>
</destination>
</destinations>
</scenario>
</syntaxhighlight>

Calling SOAP Web services

2015-02-10T00:24:40Z

Maxlmus:

Calling SOAP Web services

2015-02-10T00:22:57Z

Maxlmus:

Calling SOAP Web services

2015-02-10T00:22:04Z

Maxlmus:

Calling SOAP Web services

2015-02-10T00:21:15Z

Maxlmus: Created page with "The ETL Framework natively supports SOAP Web services over HTTP. You will need to: # Set source or destination connection to the one linked to the SOAP endpoint # Define SOAP..."

The ETL Framework natively supports SOAP Web services over HTTP. You will need to:

# Set source or destination connection to the one linked to the SOAP endpoint
# Define SOAP request in the connection CDATA attribute

<pre>
<connection alias="Cars">
<url>{soapendpoint}</url>
<CDATA><![CDATA[<?xml version="1.0" encoding="UTF-8"?>
<SOAP-ENV:Envelope xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ns1="http://www.vendor.com/soapendpoint/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<SOAP-ENV:Header>
<ns1:AuthenticationHeader>
<vendorUserId>{requestUserId}</mktowsUserId>
<vendorSignature>{requestSignature}</requestSignature>
<vendorTimestamp>{requestTimestamp}</requestTimestamp>
</ns1:AuthenticationHeader>
</SOAP-ENV:Header>
<SOAP-ENV:Body>
<ns1:paramsGetMultipleCars>
<leadSelector xsi:type="ns1:KeySelector">
<keyType>MAKE</keyType>
<keyValues>
<stringItem>Ford</stringItem>
<stringItem>Acura</stringItem>
</keyValues>
</leadSelector>
<batchSize>100</batchSize>
</ns1:paramsGetMultipleCars>
</SOAP-ENV:Body>
</SOAP-ENV:Envelope>

]]></CDATA>
<params>method=post;exceptions=500;file=false;header_Content-Type=text/xml</params>
<connector>com.toolsverse.etl.connector.xml.XmlObjectConnector</connector>
<transport>com.toolsverse.io.HttpProcessor</transport>
</connection>
</pre>

Calling REST based Web services

2015-02-09T13:44:35Z

Maxlmus:

The ETL Framework natively supports REST based Web services. You don't need to do anything special, just make sure sources are linked to the GET endpoints and destinations - to the PUT or POST endpoints.

Use JSON, XML or text to serialize datasets and objects.

You can include access tokens into the endpoint URLs. Example: https://host/rest/v1/data/?filter=123&access_token={access_token}. You will need to set access_token as a system property. Please see examples below:

[https://www.toolsverse.com/products/etl-framework/examples/web_services/http_get.shtml Example of the GET request]

[https://www.toolsverse.com/products/etl-framework/examples/web_services/http_put.shtml Example of the POST request]

Main Page

2015-02-09T13:44:16Z

Maxlmus:

== Toolsverse knowledge base ==

=== ETL and Data Integration ===

* [[ETL scenario language]]
* [[Writing data integration and data migration scenarios]]
* [https://www.toolsverse.com/products/etl-framework/examples ETL Scenario Examples]
* [https://www.toolsverse.com/products/etl-framework/java-examples Java Examples]

=== Support Resources ===
* [https://www.toolsverse.com/products/video-tutorials/ Video Tutorials]
* [https://www.toolsverse.com/products/case-studies/ Case Studies]

== Products ==

=== ETL Framework ===
* [https://www.toolsverse.com/products/etl-framework/ Product page]
* [https://www.toolsverse.com/products/etl-framework/feature-list.shtml Features]
* [[Installing and Configuring ETL Framework]]
* [[Redistribution and managing dependencies]]
* [[Building Source Code]]
* [[Using Eclipse]]
* [[Running ETL Scenarios]]
* [[Configuration file]]
* [[Calling REST based Web services]]
* [[Calling SOAP Web services]]
* [https://www.toolsverse.com/products/etl-framework/java-examples/ Embedding ETL Engine]
* [https://www.toolsverse.com/javadoc/ JavaDoc]

=== Data Explorer ===
* [https://www.toolsverse.com/products/data-explorer/ Product Page]
* [https://www.toolsverse.com/products/data-explorer/feature-list.shtml Features and Editions]
* [https://www.toolsverse.com/products/data-explorer/docs/ User Guide]

2014-12-22T16:39:30Z

Maxlmus:

2014-08-05T17:08:15Z

Maxlmus:

== Downloading ==

Look for clearly marked ETL Framework downloads. On Windows and OS X ETL Framework can be downloaded as an installer. Use archive downloads for other operating systems.

== Installing ==

There are two ways to install ETL Framework: using installer or extracting files from the archive.

When using installer just execute it and follow instructions on the screen.

OS X installer is offered in a form of self-extracting DMG file. When prompted drag and drop ETL Framework folder to the Applications.

When installing from archive you will need to manually extract files using utility provided by operating system. All files are contained in the enclosing folder named etl-version.

Installing from archive will not add any entries to the Start menu, add desktop launchers or register the software in the registry.

== Configuring ==

ETL Framework comes pre-configured to provide highest levels of performance and compatibility.

Framework requires a valid not expired license. Licenses are located under ETL_FRAMEWORK_HOME\license folder (*.lic files) or under ETL_FRAMEWORK_HOME\lib (*.jar files).

'''Note''': ETL Framework comes with a 20-days evaluation license. It will expire after 20 days since the first use. You can a buy a commercial license which will never expire. License can be delivered as a lic or jar file. Using jars for licenses decreases dependency on external folders.

You can read more about licensing [http://www.toolsverse.com/support/faq/ here].

=== Changing configuration properties ===

ETL framework uses etl.properties file located under ETL_FRAMEWORK_HOME/config folder to load configuration properties from. You can change default name of the configuration file by setting property config.file.name. Example: <pre>-Dconfig.file.name=c:/etl/config/test.properties.</pre>

If file does not exist framework will use default values.

You can add any property to the file using format property_name=value. You can also completely ignore configuration file and set properties programmatically using System.setProperty(name, value) or by passing -D props to the Java program.

Some of the properties:
* network.appupdateurl - the URL used to call auto-update Web service
* network.appserverurl - the URL used to call ETL Web service

Example:

<pre>
network.appserverurl=http://localhost:8080/dataexplorer/ide
network.appupdateurl=http://www.toolsverse.com/api/services/CheckForUpdates
# never change properties below
app.update.key=etl
app.name=etlprocess
app.title=ETL Framework
</pre>

=== Changing JVM Parameters ===

# Open ETL_FRAMEWORK_HOME\etlappstart.properties in your favorite text editor
# Change line containing app.vm.options
# Save

=== Changing Logging Parameters ===

# Open ETL_FRAMEWORK_HOME\config\log4j.properties in your favorite text editor
# Make a change
# Save

=== Adding JDBC drivers ===

Each copy of the ETL Framework comes with the following JDBC drivers:

* DB2
* Informix
* Microsoft SQL Server
* Sybase Advanced Server
* MySQL
* Oracle
* PostgreSQL
* Java DB (Derby)

They are located in the ETL_FRAMEWORK_HOME\jdbc folder. ETL Framework automatically recognizes (on start-up) and dynamically loads (when needed) jdbc drivers under this folder.

To install new jdbc driver just add a sub folder under ETL_FRAMEWORK_HOME\jdbc and put all required files there. Restart Data Explorer.

Alternatively, since JDBC drivers are just jar files you can simply put them in the ETL_FRAMEWORK_HOME\lib folder and they will be automatically loaded and added to the classpath.

== Redistribution and managing dependencies ==

Configure ETL framework the way you like, exclude all unneeded components. You can find redistribution list and recommendations how to manage dependencies [[Redistribution and managing dependencies|here]].

== Checking for updates ==

ETL Framework can check for new version and update itself if new version exists.

To check for new new version run ETL framework executable with -v parameter.

Example:

<pre>
C:\projects\toolsverse\java\dist\etl>etl.exe -v
Toolsverse Etl Framework 4.0-19862. Use -? for help.

Checking for updates...
Update status: new version available
</pre>

To auto update ETL framework run executable with -u parameter.

Example:

<pre>
C:\projects\toolsverse\java\dist\etl>etl.exe -u
Toolsverse Etl Framework 4.0-19862. Use -? for help.
</pre>

== Manually updating ==

You can update framework to the new version manually:

# Download update
# Replace exiting files on files from the update