Hi Experts,
Could you please advise what's the best way to load Master Data tables like Vendor, Material etc. In my opinion, since the tables are not huge, can be fully loaded each time. Any different view suggesting delta load etc. from a best practice perspective? Please guide. Thanks!
Best answer by janpeter.van.d
Hi @dinesh.kumar35,
If the tables have a change date available, a delta load would make sense since you would be sure that changes are extracted. If you setup the primary keys in the extraction settings, Celonis will remove the old values automatically.
If no change date is available, I would suggest a full load based on the same reasons as you state.
Note that whatever method you take, it is advisable to full load your entire data set once in a while (once per week/2 weeks/month) to ensure it is still 100% in sync with your source data.
Kind regards,
Jan-peter
+26Hi @dinesh.kumar35,
If the tables have a change date available, a delta load would make sense since you would be sure that changes are extracted. If you setup the primary keys in the extraction settings, Celonis will remove the old values automatically.
If no change date is available, I would suggest a full load based on the same reasons as you state.
Note that whatever method you take, it is advisable to full load your entire data set once in a while (once per week/2 weeks/month) to ensure it is still 100% in sync with your source data.
Kind regards,
Jan-peter
Hi, if the tables are not large, there is nothing against subtracting them in the full. Large tables such as the KONV in SAP I would subtract once on the weekend in full and if necessary in delta during the week.
Hi @dinesh.kumar35,
If the tables have a change date available, a delta load would make sense since you would be sure that changes are extracted. If you setup the primary keys in the extraction settings, Celonis will remove the old values automatically.
If no change date is available, I would suggest a full load based on the same reasons as you state.
Note that whatever method you take, it is advisable to full load your entire data set once in a while (once per week/2 weeks/month) to ensure it is still 100% in sync with your source data.
Kind regards,
Jan-peter
Hi @janpeter.van.d Our SAP team does not allow to full loads as it impacts the SAP transactional system.
Have you faced this challenge or how do you handle full loads in your projects with SAP?
+26Hi @janpeter.van.d Our SAP team does not allow to full loads as it impacts the SAP transactional system.
Have you faced this challenge or how do you handle full loads in your projects with SAP?
Hi Ravi,
I have not to be honest. It is possible to only keep you data up to date with delta loads to limit the extraction loads, but it might get out-of-sync over time for various reasons. It always an trade-off between risks that systems get overloaded and risks that data gets out of sync. Depending your use case you should consider this together with the use case owners and system owners (here: SAP Team).
Hi @janpeter.van.d Our SAP team does not allow to full loads as it impacts the SAP transactional system.
Have you faced this challenge or how do you handle full loads in your projects with SAP?
@janpeter.van.d Thanks!
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
