Application structure

An application consists of the following main components:

• Sources - contains source definitions. Each source definition has a name, type and a list of source-type-specific attributes used to access data in the source. For example, PgSQLSource type (i.e. a source type to connect to PostgreSQL database) has connection sting (connStr), user, and password as source-type-specific parameters. Creating a source means only storing metadata about the source (namely, name, type, and source-type-specific parameters) - it does not initiate any jobs to extract data form the source. Data is extracted/loaded from/to source by various operations that takes source definitions (referred by name) as their parameters.

• Specifications - define rules of various data data integration tasks. Each specification has name, type, and type-specific parameters. Specification name must be unique across all types. In the InforLink’s user interface specifications are organized by type into sub-folders of the application navigation tree. For example, Matching type specification contains rules to match and consolidate duplicate records in a table. For each specification type there is usually a corresponding operation to execute a specification of the type. Such operation takes the name of a specification as a parameter and executes the specification. It can also take other parameters to execute a part of the specification or to execute it in a specific mode. For example, MatchOneSource is an operation that executes a Matching specification. Additional parameters of MatchOneSource allows to execute only matching rules (without consolidation rule) that can be useful during development.

• Scenarios - define procedures that execute operations. More precisely, a scenario is a procedure consisting of a sequence of operation calls. Operation implements a basic data integration task and is called from scenarios. For example, LoadTableFromSource operation allows to load any table from any source into another source. Transform is an universal operation to implement various types of data transformation.