Skip to main content

Good afternoon,

 

among the best practices suggested by Celonis, there are:

 

  1. Scheduling Full Loads if delta loads are not possible or if source tables' metadata are changed frequently (i.e., a column is added or deleted).
  2. Extracting only the relevant columns of the source tables (i..e, those actually used in the transformations).

 

If the relevant columns can't be deleted or changed in the source systems (because, for instance, they are crucial for the business and it would be a disaster to change them for the data recovery!), why should I schedule full extractions to consider metadata changes?

 

Shouldn't I launch them only manually when columns need to be added to the extracted tables?

 

Moreover, if the answer is in the documentation, could you share the link?

 

Thanks and regards,

Simone

you can schedule delta load, and enable the feature include "meta data change" under the extraction setting tab, so in case of new column got added to source table, it will bring the new column only for new records.

 

hope it helps


you can schedule delta load, and enable the feature include "meta data change" under the extraction setting tab, so in case of new column got added to source table, it will bring the new column only for new records.

 

hope it helps

Hi Devesh,

 

I knew that that feature is not recommended by Celonis (just because of the inconsistency of the new and old records).

 

Anyway, thanks for your help, I guess that, even if it is very unrealistic that the structure of the extracted columns will be changed, a full extraction must always be scheduled just in case.

 

Simone


Reply