Skip to main content
After accumulating 2 years of data in our data O2C and P2P data models, Id like to setup a regular cleanup job to remove all data older than 2 years from the data model. Are there any best practices to do this you could share with me?

Assuming you have running delta loads, just add a transformation job in the beginning that deletes older data

Example: delete data older than CURRENT year

 

DELETE

FROM TABLE

WHERE YEAR(TABLE.DATECOLUMN)<YEAR(TODAY())

 

 


Another alternative is to extract only the relevant data in a full load again and rerun the transformations and Data model load. Therefore you are also saving APC within the EMS.

 

Best,

Kevin


Running full loads on O2C tables such as CDPOS and VBFA can be tricky as they are heavy tables and might fail before completion.

The other reason to avoid full loads is that obviously, they take way longer to complete (sometimes >24h from extraction to transformation on O2C models).

Still, if you opt to run full loads, you need to make some adjustments as detailed here

https://www.celopeers.com/s/question/0D50700000dY6kMCAS/hi-allis-there-any-way-to-put-data-pool-parameters-for-just-rolling-12-months-so-that-it-can-extract-data-for-last-12-months-only-every-time-can-we-define-such-parameter-for-extraction-and-what-will-be-the-syntax

 

Best,

Gabriel


Reply