public class Extractor
extends Object
Constructor and Description |
---|
Extractor(TransactionMonitor transactionMonitor)
Instantiates a new extractor.
|
Modifier and Type | Method and Description |
---|---|
void |
extract(EtlConfig config,
Scenario scenario)
Extracts data from all sources.
|
void |
extract(EtlConfig config,
Scenario scenario,
Loader loader,
int index)
Extracts data from all sources.
|
void |
extract(EtlConfig config,
Scenario scenario,
Source source,
Destination destination,
AddRecordCallback addRecordCallback)
Extracts data from the source.
|
void |
extractMandatory(EtlConfig config,
Scenario scenario)
Extracts mandatory data sets.
|
boolean |
hasMixOfStreamAndNotStream(EtlConfig config,
Scenario scenario)
Checks if the scenario contains mix of stream and not stream
transformations.
|
static Object |
lookup(EtlConfig config,
Scenario scenario,
String connectionName,
String name,
String fieldName,
String sql,
TypedKeyValue<String,Object>... args)
Extract data set, execute provided SQL statement and return the value of
the provided field.This method is thread safe.
|
static DataSet |
lookup(EtlConfig config,
Scenario scenario,
String connectionName,
String name,
String sql,
TypedKeyValue<String,Object>... args)
Extract data set, and optionally execute provided SQL statement.
|
static Object |
lookupNoCache(EtlConfig config,
Scenario scenario,
String connectionName,
String name,
String fieldName,
String sql,
TypedKeyValue<String,Object>... args)
Extract data set, execute provided SQL statement and return the value of
the provided field.
|
static DataSet |
lookupNoCache(EtlConfig config,
Scenario scenario,
String connectionName,
String name,
String sql,
TypedKeyValue<String,Object>... args)
Extract data set, and optionally execute provided SQL statement.
|
public Extractor(TransactionMonitor transactionMonitor)
transactionMonitor
- the transaction monitorpublic void extract(EtlConfig config, Scenario scenario, Loader loader, int index) throws Exception
config
- the etl configscenario
- the scenarioloader
- the loaderindex
- the indexException
- in case of any errorpublic void extract(EtlConfig config, Scenario scenario) throws Exception
config
- the etl configscenario
- the scenarioException
- in case of any errorpublic boolean hasMixOfStreamAndNotStream(EtlConfig config, Scenario scenario)
config
- the configscenario
- the scenariopublic static DataSet lookup(EtlConfig config, Scenario scenario, String connectionName, String name, String sql, TypedKeyValue<String,Object>... args) throws Exception
"stored_dataset_" + scenario.getName() + "_" + connectionName + "_" + name
config
- the configscenario
- the scenarioconnectionName
- the connection namename
- the data set name, also can be used as a file name for file
based connectionssql
- the sqlargs
- the arguments for SQL query in the format key/value.Exception
- in case of any errorpublic static DataSet lookupNoCache(EtlConfig config, Scenario scenario, String connectionName, String name, String sql, TypedKeyValue<String,Object>... args) throws Exception
config
- the configscenario
- the scenarioconnectionName
- the connection namename
- the data set name, also can be used as a file name for file
based connectionssql
- the sqlargs
- the arguments for SQL query in the format key/value.Exception
- in case of any errorpublic static Object lookup(EtlConfig config, Scenario scenario, String connectionName, String name, String fieldName, String sql, TypedKeyValue<String,Object>... args) throws Exception
config
- the configscenario
- the scenarioconnectionName
- the connection namename
- the data set name, also can be used as a file name for file
based connectionsfieldName
- the field namesql
- the sqlargs
- the arguments for SQL query in the format key/value.Exception
- in case of any errorpublic static Object lookupNoCache(EtlConfig config, Scenario scenario, String connectionName, String name, String fieldName, String sql, TypedKeyValue<String,Object>... args) throws Exception
"stored_dataset_" + scenario.getName() + "_" + connectionName + "_" + name
This method is thread safe.config
- the configscenario
- the scenarioconnectionName
- the connection namename
- the data set name, also can be used as a file name for file
based connectionsfieldName
- the field namesql
- the sqlargs
- the arguments for SQL query in the format key/value.Exception
- in case of any errorpublic void extract(EtlConfig config, Scenario scenario, Source source, Destination destination, AddRecordCallback addRecordCallback) throws Exception
config
- the etl configscenario
- the scenariosource
- the sourcedestination
- the destinationaddRecordCallback
- the add record callbackException
- in case of any errorCopyright © 2010-2020 Toolsverse. All Rights Reserved.