com.toolsverse.etl.core.engine

Interface CodeGen

All Known Implementing Classes:

DefaultCodeGen

public interface CodeGen

CodeGen is an interface for code generators. ETL framework generates and executes code for the etl scenario. Code is a usually vendor specific SQL such as Oracle PL/SQL, Microsoft Transact SQL, etc. The call sequence for the typical scenario is the following: prepare(...) is called for each destination. The resulting code saved internally, assembleCode(...) assembles final code by concatenating code for all destinations with added declarations, etc so it can be be executed in the designated database, execute(...) executes it.

Since:

1.0

Version:

2.0

Field Summary

Fields

Modifier and Type	Field and Description
static int	CREATED_STATUS The code generator created .
static int	EXECUTED_STATUS The code is executed .
static int	PREPARED_STATUS The code is prepared .

Method Summary

Modifier and Type	Method and Description
void	addScriptToCleanOnException(String script) Adds the script to clean on exception.
void	addScriptToCleanUpAfter(String script) Adds the script to clean up after.
void	addScriptToCleanUpBefore(String script) Adds the script to clean up before.
void	addScriptToCreate(Destination destination, String script) Adds the script to create.
void	addScriptToInit(Destination destination, String script, boolean cleanUpOnException, String type, boolean ignoreError) Adds the script to init.
void	assembleCode(EtlConfig config, Scenario scenario, Driver driver, int loadIndex, boolean silent) Assembles prepared code.
void	cleanUp(EtlConfig config, Scenario scenario, Driver driver, Connection conn) Clean up after last block of code is executed.
void	cleanUpOnException(EtlConfig config, Scenario scenario, Driver driver, Connection cleanUpConn) Clean up database objects on exception.
void	copy(CodeGen codeGen) Copy parameters from other code generator.
void	execute(EtlConfig config, Scenario scenario, Driver driver, boolean silent, Connection conn, Connection cleanUpConn, Destination destination) Executes code.
List<String>	getScriptsToCleanOnException() Gets the list of scripts which should be executed to clean up database object on exception.
int	getStatus() Gets the status of the code generator.
EtlUnit	getUnit() Gets the current unit.
void	prepare(EtlConfig config, Scenario scenario, Destination destination, boolean silent, boolean onlyInit, boolean isStreaming) Prepares code for the destination using given config and scenario.
void	reorderColumns(Destination destination, DataSetConnector writer, DataSetConnectorParams writerParams) Reorder columns.
void	reset() Resets the status.
void	runRemainingBatch(EtlConfig config, Scenario scenario, Connection conn, Destination destination) Run remaining batch statements.
void	setTransactionMonitor(TransactionMonitor transactionMonitor) Sets the transaction monitor.
void	setUnit(EtlUnit unit) Sets the current unit.

Field Detail

CREATED_STATUS

static final int CREATED_STATUS

The code generator created .

See Also:

Constant Field Values

PREPARED_STATUS

static final int PREPARED_STATUS

The code is prepared .

See Also:

Constant Field Values

EXECUTED_STATUS

static final int EXECUTED_STATUS

The code is executed .

See Also:

Constant Field Values

Method Detail

setTransactionMonitor

void setTransactionMonitor(TransactionMonitor transactionMonitor)

Sets the transaction monitor.

Parameters:

transactionMonitor - the new transaction monitor

assembleCode

void assembleCode(EtlConfig config,
                  Scenario scenario,
                  Driver driver,
                  int loadIndex,
                  boolean silent)
           throws Exception

Assembles prepared code. The idea is the following: prepare(...) is called for each destination. The resulting code saved internally. assembleCode(...) assembles final code by concatenating code for all destinations with added declarations, etc so it can be be executed in the designated database. Also it splits code on chunks if necessary. For example Oracle limits size of the PL\SQL block to approximately 3000 lines. @param config The config @param scenario The scenario @param driver The driver @param loadIndex If equals to 0 this chunk will be executed first.

Parameters:

silent - If true logging is disabled for everything but exceptions

Throws:

Exception - in case of any error

cleanUp

void cleanUp(EtlConfig config,
             Scenario scenario,
             Driver driver,
             Connection conn)
      throws Exception

Clean up after last block of code is executed.

Parameters:

config - the config

scenario - the scenario

driver - the driver

conn - the connection

Throws:

Exception - the exception in case of any error

cleanUpOnException

void cleanUpOnException(EtlConfig config,
                        Scenario scenario,
                        Driver driver,
                        Connection cleanUpConn)
                 throws Exception

Clean up database objects on exception.

Parameters:

config - the config

scenario - the scenario

driver - the driver

cleanUpConn - the clean up connection

Throws:

Exception - the exception

copy

void copy(CodeGen codeGen)

Copy parameters from other code generator.

Parameters:

codeGen - the code generator

execute

void execute(EtlConfig config,
             Scenario scenario,
             Driver driver,
             boolean silent,
             Connection conn,
             Connection cleanUpConn,
             Destination destination)
      throws Exception

Executes code. Code for each destination should be already prepared and stored internally.

Parameters:

config - The etl config

scenario - The scenario

driver - The driver

silent - If true logging is disabled for everything but exceptions

conn - The JDBC connection

cleanUpConn - the clean up connection

destination - the destination

Throws:

Exception - in case of any error

getScriptsToCleanOnException

List<String> getScriptsToCleanOnException()

Gets the list of scripts which should be executed to clean up database object on exception.

Returns:

the list of scripts to clean up on exception

getStatus

int getStatus()

Gets the status of the code generator.

Returns:

the status

getUnit

EtlUnit getUnit()

Gets the current unit.

Returns:

the unit

See Also:

EtlUnit

prepare

void prepare(EtlConfig config,
             Scenario scenario,
             Destination destination,
             boolean silent,
             boolean onlyInit,
             boolean isStreaming)
      throws Exception

Prepares code for the destination using given config and scenario.

Parameters:

config - The etl config

scenario - The scenario

destination - The destination

silent - If true logging is disabled for everything but exceptions

onlyInit - the only init

isStreaming - true if is streaming

Throws:

Exception - in case of any error

reset

void reset()

Resets the status.

runRemainingBatch

void runRemainingBatch(EtlConfig config,
                       Scenario scenario,
                       Connection conn,
                       Destination destination)
                throws Exception

Run remaining batch statements.

Parameters:

config - the ETL config

scenario - the scenario

conn - the connection

destination - the destination

Throws:

Exception - in case of any error

setUnit

void setUnit(EtlUnit unit)

Sets the current unit.

Parameters:

unit - the new unit

See Also:

EtlUnit

addScriptToCleanOnException

void addScriptToCleanOnException(String script)

Adds the script to clean on exception.

Parameters:

script - the script

addScriptToCleanUpAfter

void addScriptToCleanUpAfter(String script)

Adds the script to clean up after.

Parameters:

script - the script

addScriptToCleanUpBefore

void addScriptToCleanUpBefore(String script)

Adds the script to clean up before.

Parameters:

script - the script

addScriptToCreate

void addScriptToCreate(Destination destination,
                       String script)

Adds the script to create.

Parameters:

destination - the destination

script - the script

addScriptToInit

void addScriptToInit(Destination destination,
                     String script,
                     boolean cleanUpOnException,
                     String type,
                     boolean ignoreError)

Adds the script to init.

Parameters:

destination - the destination

script - the script

cleanUpOnException - the "clean up on exception" flag

type - the type

ignoreError - if true ignore error when executing script

reorderColumns

void reorderColumns(Destination destination,
                    DataSetConnector writer,
                    DataSetConnectorParams writerParams)

Reorder columns.

Parameters:

destination - the destination