Best practice of integration with on-premise custom systems

Dear Community member,

Now I am planning how to integrate on-premise custom systems with Celonis, based on new client’s expectation.
In case I cannot use standard app connection (like SAP ECC) via Extractor, next choice is Data push API or file upload.

In these scenarios, I need to decide transformation task is performed, either in Ceonis or outside of Celonis like below flow.

(Source systems = on-premise custom systems) -> (transformation system = ?) -> (Load system = Celonis IBC)

Could you share your experience of,
1.whether transformation task is performed in/outside of Celonis, and why ?
2.do you reuse Activity name of SAP ECC to smoothly unite transformations of multiple source systems ?
3.if your answer of #1 is outside of Celonis, did you use some ETL tool (Talend, KNIME etc.) for managing transformation ?

For the moment, I will choose to perform transformation outside of Celonis, due to optimizing human resource (transformation script can be developed by client’s developers not familiar with Celonis).
In case of O2C/P2P processes, I will provide list of SAP O2C/P2P Activities for client’s developer to map to client systems.
OSS ETL tools are available to filter/join tables in source systems, and may be possible to call API or just download files.

Kazuhiko

Hello, kaztakata.

In my projects I choose between two scenarios based on data size.

If number of cases/activities is relatively small (let’s say below 1-2 mln activities) do it outside Celonis and use PUSH API. In this case Telentd or Pentaho Kettle is your open source options. It will take your client several iterations to create proper event log, however with small data size reloading is not a big deal

If we are talking of bigger datasets split it. Ask client to export raw table (similar to EKKO/EKPO, EBAN, etc. in SAP ECC) and do transformation jobs in Celonis. In case if you find error in mapping of activities you can always adjust and rerun transformation without uploading gigabytes of data and involving customer IT staff.

Related to your question #2 - it could be regional specifics, but we’ve NEVER used standard activity names as customer always have extra activities which are important for them.

1 Like

Dear nicks.si,

Thanks for your reply.
It is good guidance for me, because I am not sure how much volume is acceptable for OSS ETL tools.

Dear all,

Just FYI, I implemented ETL program by KNIME (as below).
Also I measured throughput time of pushing excel file from my laptop to IBC.

10K record : 20 sec
100K record : 60 sec
1000K record : 240 sec

In case of 1000K record, KNIME process consumed 2.5GB memory.
I feel it is upper limit per one data push as nicks.si mentioned.

Best regards,
Kazuhiko

1 Like