DAPI

Metadata For Execute Data Logistic Load Process Step

Purpose The DAPI Metadata For Execute Data Logistic Load Process Step provides data logistic information to orchestrate any single Load…
Purpose

The DAPI Metadata For Execute Data Logistic Load Process Step provides data logistic information to orchestrate any single Load Process Step (LPS).

Motivation

This DAPI provides each LPS with the information necessary for execution. The information is the time period with which the LPS extracts the data from the source.

Applicability

The DAPI plays a role in LPS loading the data from the source interfaces into the stage layer.

Content

All LPS which has generated jobs.

Attribute Description
ProcessStepId Unique Id to identify a defined LoadProcessStep in table LoadProcessStep.
ProcessStepLabel Label to identify a defined LoadProcessStep. Defined during data model design process.
ProcessStepType Short name of different load types:
  • DEF: Default - No specific process type
  • ILO: Initial Load - Used for special initial load processes
  • FUL: Full Load processes - incrementatl interface
  • INL: Incremental load processes - full load interface
  • WIL: Window load processes - selection of window of records to process
LoadBatchId Unique Id to identify a defined Batch in table LoadBatch.
AuditTrailId Unique Id for each single LoadProcessStep. This is the atomic building block.
AssertionTimeFrom Current running LPS start timestamp.
HighEndTimestamp High End Timestamp as defined in table TEDAMOHConfigExMeXFramework.
DeltaExportTimeStampFrom

Time period begin timestamp. It is defined as

  1. If override timestamp exist use it; else
  2. If LPS never executed us Low End Timestamp; else
  3. Use recent sucessful batch start timestamp of associated batch; else
  4. Use recent sucessful LPS start timestamp; else
  5. Low End Timestamp
DeltaExportTimeStampBefore

Time period before timestamp. It is defined as

  1. Use current running batch start timestamp of associated batch; else
  2. Use current running LPS start timestamp; else
  3. High End Timestamp
Sample

In the following table are shown two example LPS to load data from a source interface into the Stage Layer target tables:

  • ArtworkInMuseum
  • Supllier

To extract data from the source interface the two LPS will use the time periode (Closed - Open notation) which are stated in the columns

  • DeltaExportTimeStampFrom
  • DeltaExportTimeStampBefore

LPS 9 will extract and load data with the time period [2019-09-24 13:54:29.6989653  -  2019-09-24 15:49:57.8288227[

LPS 10 will extract and load data with the time period [2019-09-24 13:54:29.6989653  -  2019-09-24 15:49:57.8288227[

ProcessStepId  ProcessStepLabel ProcessStepType LoadBatchId AuditTrailId AssertionTimeFrom [...] DeltaExportTimeStampFrom DeltaExportTimeStampBefore
9 Source2StageArtworkInMuseum DEF 4 16107 2019-09-24 15:49:57.8734702 [...] 2019-09-24 13:54:29.6989653 2019-09-24 15:49:57.8288227
10 Source2StageSupplier DEF 4 16108 2019-09-24 15:49:58.1498235 [...] 2019-09-24 13:54:29.6989653 2019-09-24 15:49:57.8288227
Posted 2 years agoby admin.tedamoh
DAPI Data Logistic