Data Loads option : when I should use 'Reload from Cache'?

Dear Community member,

In IBC Event Collection, frequently I used Data Loads but still I am not sure the difference between ‘Reload from Cache’ and ‘Force Complete Reload’.
Looking at log (below picture), I found that ‘Reload from Cache’ ran every Saturday automatically. Previously I heard that it is relevant to system reboot and Cache (snapshot) is used to reload data automatically. Except for this case, when should I use ‘Reload from Cache’ ?

Why I ask it is, ‘Force Complete Reload’ is always slower than ‘Reload from Cache’, so I would like to understand the difference and use faster option as much as possible.

Best regards,
Kazuhiko

Hi Kazuhiko,

“Reload from Cache” does not update the data in the Data Model tables so it can be useful in cases where you did not change the underlying data, but configurations of the Data Model.
Examples include updating a foreign key relationship or the activity column configurations.

In all cases where data has changed, you should either do a complete reload to update the data or use the API to perform a partial reload of the tables which have changed. See https://help.celonis.cloud/help/display/CIBC/Process+Data+Model+API for details on the last point.

Best regards,
Leonid

Dear Leonid,

Thanks for your detail answer. I am clear.

Best regards,
Kazuhiko

Hi Team,

In “Reload From Cache” does it mean it will load data model from already extracted tables in IBC ?
In “Force Complete Reload” mean it will pull data from database directly rather using data from already loaded extracts ?
please clarify .

Thanks,
Satish.M

Hi Satish,

You can find more information on this here: https://help.celonis.cloud/help/display/CIBC/Data+Model+Loading

In order to pull the newest data from your source system, you will need to use Data Jobs for example.

Does this answer your question?

Best,
Leonid

Hi Leonid,

Thank You for the reply.
Let me make my question more elaborate.

we have Data Extraction and Data Model Loads 2 concepts.
Run 1:
Step 1. Say for example I am extraction 10 Million records table A in extraction
Step 2. Table A is part of Datamodel and it got load using " Force comlete reload" option.
Though data is already in IBC data model load is taking huge time to refresh.

Run 2: In by backend DB this table A has new 2 million records added.
Step 1: Table A extracted using Delta Load mode and now Table A in IBC has 12Million records
Step 2: Now Data model has to be refreshed againand it is trying to reload all 12Million records and taking time.
why is it taking time though data is already in celonis IBC ?
How can we improve the data model load performance?

Thanks,
Satish

Hi Satish,

Ok, I understand your point. Unfortunately, tables have to be loaded completely if they were changed. However, 10 million records is not that many so maybe the load is slow because of something else? Typical Data Models use views and if the SQL code that creates these views is not performant, the Data Model load takes a lot of time. We have defined best practices for transformations in the help space which might help you to improve the speed of the Data Model load and of transformations as well.

Regards,
Leonid