Skip to main content
Question

Archiving of old data

  • June 15, 2020
  • 3 replies
  • 20 views

oliver.joere
Level 2
After accumulating 2 years of data in our data O2C and P2P data models, Id like to setup a regular cleanup job to remove all data older than 2 years from the data model. Are there any best practices to do this you could share with me?

3 replies

gabriel.okaba11
Celonaut
Forum|alt.badge.img+2

Assuming you have running delta loads, just add a transformation job in the beginning that deletes older data

Example: delete data older than CURRENT year

 

DELETE

FROM TABLE

WHERE YEAR(TABLE.DATECOLUMN)<YEAR(TODAY())

 

 


  • Level 0
  • November 27, 2021

Another alternative is to extract only the relevant data in a full load again and rerun the transformations and Data model load. Therefore you are also saving APC within the EMS.

 

Best,

Kevin


gabriel.okaba11
Celonaut
Forum|alt.badge.img+2

Running full loads on O2C tables such as CDPOS and VBFA can be tricky as they are heavy tables and might fail before completion.

The other reason to avoid full loads is that obviously, they take way longer to complete (sometimes >24h from extraction to transformation on O2C models).

Still, if you opt to run full loads, you need to make some adjustments as detailed here

https://www.celopeers.com/s/question/0D50700000dY6kMCAS/hi-allis-there-any-way-to-put-data-pool-parameters-for-just-rolling-12-months-so-that-it-can-extract-data-for-last-12-months-only-every-time-can-we-define-such-parameter-for-extraction-and-what-will-be-the-syntax

 

Best,

Gabriel