Skip to main content

Data Ingestion & Infrastructure

Discuss and ask questions about data integration, task mining, admin settings, and PQL extractors to optimize your infrastructure.

  • 805 Topics
  • 2,121 Replies
805 Topics
emilia.stepi
Level 2
emilia.stepiLevel 2
asked in Data Ingestion & Infrastructure
Restricting access to specific data model column

Hello everyone,I want to restrict access to a specific column in a data model so that non-admins cannot see the values in that column or the column itself. Is there a way to achieve this using data permissions? Thank you!Best,Emilia

252
benjamin.delay
Level 2
carmen.kaluz
Level 3
carmen.kaluzLevel 3
asked in Data Ingestion & Infrastructure
Use App Key for content-cli in ML workbench

Hi all,I would like to export datapools as json using content-cli in the ML workbench in order to backup them in Git. I have created an API key in my profile and used it in a python script with content-cli pull data-pool --id data-pool-id, which successfully generated a json of the datapool.However, I don't want to do this with a user-based API key but with an Application key (created under admin&settings).When I run the same command using the Application key, I get the error "error: The provided team or api key is wrong."With get_celonis(api_token=“applicationkey”, key_type="APP_KEY") however, I can see the Application key and all its permissions.Is content-cli possible with an Application key and if so, how can I use it?

293
carmen.kaluz
Level 3
thomas.keim
Level 4
thomas.keimLevel 4
asked in Data Ingestion & Infrastructure
Usage of PQL Function QUANTITY_CONVERT

Hello Community,I need you help in using the PQL Function QUANTITY_CONVERTMy objective is simple: I want to convert the SAP Field LIPS.BTGEW to Tons (TO). The original Unit is stored in field LIPS.GEWEI.With this try I get an errorQUANTITY_CONVERT("LIPS"."BRGEW", FROM ("LIPS"."GEWEI"), TO ('TO'))A PQL error is preventing this query from executing.Syntax error near [)] after reading [QUANTITY_CONVERT("LIPS"."BRGEW", FROM ("LIPS"."GEWEI"), TO ('TO')] at line 32. Please refer to PQL documentation for available syntax. What's the problem? If I understand the docu (https://docs.celonis.com/en/quantity_convert.html) correctly, I must not provide a conversation rates table?This operator converts quantity units based on a QUANTITY_CONVERSION_RATES table, which is optionally provided by the user. If not provided by the user, a default QUANTITY_CONVERSION_RATES table will be used. Where can I see the contents of the default QUANTITY_CONVERSION_RATES table?Thanks for helpBest, Thomas 

251
thomas.keim
Level 4
douglas.souza
Level 3
douglas.souzaLevel 3
asked in Data Ingestion & Infrastructure
Filter Statments Extractions

I need to use a filter in my extraction, but the field in the table I'm bringing it from has a separate name "Date Archived" Example.When I use it like this "Date Archived" >= '2024-01-01 00:00:00', it doesn't load, I believe it's because of the space between the column name. Does anyone know how to solve this?I've tried to put the column as a parameter, but I still can't. 

305
douglas.souza
Level 3
harshsh
Level 2
harshshLevel 2
asked in Data Ingestion & Infrastructure
Replication Cockpit

How do I see the replication cockpit also along with the data job. How do i set the change table and trigger for it?

281
thomas.guerr
Level 4
gauri.pai11
Level 3
gauri.pai11Level 3
asked in Data Ingestion & Infrastructure
Data Extraction filter help

Hi team,I need help in understanding how to apply filter for KONP table in data extraction joining another table.My requirement is:1.For A018 table - I need all the records where A018.MATNR =  EKPO.MATNR.2.For KONP table, all the records from KONP table where A018.KSCHL = KONP.KSCHL If i am trying to implement 1st point in filter condition of A018 extraction as screenshot below,Data extraction fails with error : Error loading resource A017. The multi-part identifier "ts1.EKPO.MATNR" could not be bound.Can you please help in this? Thank you in advance.

91
thomas.guerr
Level 4
M
Level 2
mel.schic11Level 2
asked in Data Ingestion & Infrastructure
I would like to connect two fields that have an indirect N:N relation. Is this something which I can do via a PU_Function?

Within an OLAP table, I would like to use both the Delivery Line Item Date ("VBEP"."EDATU") and the shipment gross weight ("LIPS"."BRGEW") Does anyone know how I can apply a PQL function to combine these tables? Please find the error below. I looked into the PQL documentation but I can't find the solution. Error in formulaNo common table could be found. The tables ["VBEP"] and ["LIPS"] are connected, but have no common table. This means that they do not have a direct (or indirect) 1:N or N:1 relationship. Join path: [VBEP]N <-- 1![VBAP]!1 --> N[VBFA]N <-- 1[LIPS]. For more information on the join path, search for "Join functionality" in PQL documentation.

272
kandi.naidu
Level 4
janpeter.van.d
Level 8
janpeter.van.dLevel 8
asked in Data Ingestion & Infrastructure
What does the Eventlog Automerge option in Data Model overview do?

Hi all, Since a short while we see the option eventlog automerge in the data model overview (see attached screenshot). However, this is not mentioned in the Release Notes or online documentation. Does somebody knows what this option is for? Thanks in advance! Bests, Jan-peter! Celonis_Automerge|690x150

133
Ushmita Patil
Level 2
thomas.guerr
Level 4
thomas.guerrLevel 4
asked in Data Ingestion & Infrastructure
Extractor configuration for data transfer from MS Sharepoint to Celonis

Hello,I am trying to get data from a Microsoft sharepoint list to the Celonis Platform.According to the Flowchart provided by the API-Team - When to use the Standard Data Ingestion API (and other methods) - the extractor builder should be used for a Sharepoint data transfer, since the REST API is exposed.I tried to set up the connection, but I fail when configuring the endpoint, since I get xml-data even if I specify that I would like to have a json structure (Request Parameter - Accept: application/json;odata=verbose)Does anyone have experience with connecting the MS universe with the Celonis universe?I also found a workaround that works (Push CSV to Data Integration), but I would rather orchestrate all the data loads in the Data Integration environment instead of using Process Automation modules to make a data transfer.Thanks for your support!BR THomas

812
thomas.guerr
Level 4
swapnil.karad12
Level 2
swapnil.karad12Level 2
asked in Data Ingestion & Infrastructure
Linking tables for O2C and P2P in SAP

I want to visualize end to end process of a buisness; order received → purchasing begin→ production → payments to vendors → order shipped to customer → cutomer payment receivedhow i can connect in SAP all this O2C to P2P; what are common tables to link all this

170
swapnil.karad12
Level 2
gauri.pai11
Level 3
gauri.pai11Level 3
asked in Data Ingestion & Infrastructure
Content-cli Profile creation

Hi team,I am using this command content-cli profile create.I am asked for name,url and api key.I have added all the three, but it gives me error as - The provided team or api key is wrong.Name of the profile: XYZYour team (please provide the full url): https://XYZ.eu-5.celonis.cloudProfile type: OAuth Device Code (1), OAuth Client Credentials (2) or Application Key / API Key (3): 3Your api token: XYZ*API token is directly fetched from celonis-edit profile-api key.Is it necessary to have admin role for this.Currently i have analyst access.Please help me with this.

271
abdulnour.duale11
Celonaut
haruka.yano
Level 2
haruka.yanoLevel 2
asked in Data Ingestion & Infrastructure
Object transformation preview once showed and disappers

HiI’m curently working on OCPM Data Pool and have trouble in an object transformation.In the Currecny Conversion object(o_celonis_CurrencyConversion),I checked that I can see data record in Preview,and executed publish to development.Then I ran the Data Job Transformation(full load), succeeded,ran the Data Model Load, succeeded,but at the end o_celonis_CurrencyConversion has no data.I don’t know why but when I go see the object preview again, the preview doesn’t show anything.Has anyone had this kind of experience ever?If yes please kindly help. FYI as the prerequisite, - the relevant sap table data is sycronized from another data pool - the object and its transformation script is celonis default one​​​​​​​ - Data job is done by the duplicate of test:ocpm-data-job(celonis default test data job) - the reason why I use the duplicate data job is that the original one returns an error for schema.

150
haruka.yano
Level 2
PhuongBHN
Level 3
PhuongBHNLevel 3
asked in Data Ingestion & Infrastructure
How to enable the ‘Push data into Celonis’ Data connection option within my Data pool

Hello everyone, I am unable to locate the ‘Push data into Celonis’ Data connection option within my Data pool.Could you guide me on how to enable this feature?!--endfragment>My Celonis  I want the ‘Push data into Celonis’ as the image below 

503
abdulnour.duale11
Celonaut
Suveesh_adessi
Level 2
Suveesh_adessiLevel 2
asked in Data Ingestion & Infrastructure
Celonis Audit API - Group & Group Members

I’ve been extracting Celonis audit logs API for user-platform analysis. I couldn’t find the right field for group name and group members in the extracted data. If you are someone done this already, please share your knowledge to get these data.Also, please mention if there is any dedicated API endpoint is available to extract those data.

211
jonas.hess
Level 3
MrAj
Level 3
MrAjLevel 3
asked in Data Ingestion & Infrastructure
How to Trigger a Machine Learning Script from a Button in Celonis?

Hi everyone,I need help figuring out how to execute a Machine Learning script directly from a button in my Celonis app in views (dashboard). I believe I might need to use Action Flows, but I’m unsure about the exact steps to make this work.Here’s what I want to achieve: When a user clicks a button in the Celonis app, it should trigger an ML script. The script could be hosted externally (e.g., a REST API) or executed within Celonis.( atm its not hosted, there is just a file in ipynb format, stored in ml workbench in  analysis. I need guidance on how to set up the Action Flow and link it to the button. Has anyone implemented something similar? If so, could you share the best approach and any potential challenges I should be aware of?Thank you in advance!

240
MrAj
Level 3
MrAj
Level 3
MrAjLevel 3
posted in Data Ingestion & Infrastructure
How to Trigger a Machine Learning Script from a Button in Celonis?

 Hi everyone,I need help figuring out how to execute a Machine Learning script directly from a button in my Celonis app in views (dashboard). I believe I might need to use Action Flows, but I’m unsure about the exact steps to make this work.Here’s what I want to achieve: When a user clicks a button in the Celonis app, it should trigger an ML script. The script could be hosted externally (e.g., a REST API) or executed within Celonis.( atm its not hosted, there is just a file in ipynb format, stored in ml workbench in  analysis. I need guidance on how to set up the Action Flow and link it to the button. Has anyone implemented something similar? If so, could you share the best approach and any potential challenges I should be aware of?Thank you in advance!

170
MrAj
Level 3
MrAj
Level 3
MrAjLevel 3
asked in Data Ingestion & Infrastructure
Configuration Management for Process Flow in Celonis by Gui or flat files

Current State:Our Celonis process mining implementation includes configuration settings for process flow and status updates in our data transformation pipeline Currently, only developers can modify these configurations through SQL queries and code changes End users (procurement specialists/buyers) have no way to modify process flow configurations without technical knowledgeRequirement Question: What is the most effective way to enable non-technical end users to update process flow configurations in Celonis without requiring knowledge of SQL, PQL, YAML, or coding?Specifically:What data format should be used for configuration (CSV, Excel, UI form, etc.)? Where should this configuration be stored in the Celonis architecture? How will the configuration files be uploaded/modified by end users? How can we ensure the uploaded configurations are properly integrated with our data transformation steps?The solution should accommodate users with no programming knowledge while maintaining data inte

60
MrAj
Level 3
.
Level 4
.14600833Level 4
asked in Data Ingestion & Infrastructure
Source und Target with PROCESS_ORDER

Hello everyone,i need a throughput between two consequitive activities and want to implement it via source and target.Die idee wre source(activity_table.activity), target(activity_table.activity, process_order(target)=process_order(source)+1))would it work?Best,Maria

155
sudharani.perum
Level 2
andres.naran12
Level 4
andres.naran12Level 4
asked in Data Ingestion & Infrastructure
How to get the timestamp for an specific activity?

Hi community, I have the following challenge that I’m trying to figure out.I need to calculate how many hours were spent on a holiday, since the connection is not through SAP, I cannot use the functions that PQL provides for that.My idea is to get the timestamp related to the last activity that happened on holiday and calculate the throughput time between 9:00 AM and that specific time in order to get how many hours were spent.Example:Activity 1 → Day 1 Activity 2 → Day 2 (holiday) Activity 3 → Day 3 Activity 4 → Day 4In this example let’s suppose activity 2 occurred at 13:00, so in this case I should do the following calculation. 13 - 9 = 4 hours → This is the number of hours that were spent on holiday. Does anyone have a similar experience?

502
andres.naran12
Level 4
farouk.abdel
Level 2
farouk.abdelLevel 2
asked in Data Ingestion & Infrastructure
CPM4 sizing

what is the celonis sizing guidelines for CPM4?

121
A
Celonaut
maxime.psbpm
Level 3
maxime.psbpmLevel 3
asked in Data Ingestion & Infrastructure
Issue with Many-to-Many (N:N) Relationships in Celonis

Hi everyone,I'm facing an issue with how Celonis handles many-to-many (N:N) relationships. It seems like Celonis treats these relationships as 1:N → 1:N instead of supporting 0:N → 0:N, which leads to missing data when filtering.Example Issue:I have a many-to-many relationship between contracts and assets, modeled with an intermediate table:Contract Asset C1 A2 C2 A1 C3 NULL C4 NULL NULL A3 NULL A4 The issue occurs when I apply a filter like "Asset ≠ A4". Instead of just removing A4, I also lose contracts C3 and C4, which should still be visible since they are not linked to any asset.My Understanding of the Issue:Celonis seems to implicitly treat N:N relationships as 1:N → 1:N, enforcing a strict link between contracts and assets. It does not seem to support a true N:N model, where both sides can have optional (0:N) relationships.Question:Has anyone faced this issue before?Is there a recommended modeling approach to ensure that filtering on assets does not remove u

1075
maxime.psbpm
Level 3
Bharath
Level 3
BharathLevel 3
asked in Data Ingestion & Infrastructure
PQL For distinct activities in a case, happening at the same time.

I have a table, for CaseIDs (1,2,3...) and some Activities (A,B,C….). The activities might be repeated in a case or they can be unique. and a TimeStamp column corresponding to the caseid and activity. i need to write a query where in a case there are 7 distinct activities and all of them must happen in the same timestamp. kindly help me with this.

100
Bharath
Level 3
llllliz
Level 2
lllllizLevel 2
asked in Data Ingestion & Infrastructure
How to use data pool parameter?

Dear community, I would like to set up a data pool parameter to allow me to switch between test and prod environment? So I would need two global variables for my data jobs:var_data_source: to allow me switch between test schema and prod schema var_table_name: to allow me when creating table to add _test at the end of the table nameI have only found the way to create parameters in EACH data job under transformation and refer to the data pool parameter. Is there a smarter way? I am still using the data integration (not OCPM). Thanks a lot!   

280
llllliz
Level 2
lize.erasm11
Level 3
lize.erasm11Level 3
asked in Data Ingestion & Infrastructure
Adding attributes to a event in a OCPM model

Hi, has anyone found a easy or quick way to add a large list of attributes to a event? 

331
abdulnour.duale11
Celonaut
thomas.keim
Level 4
thomas.keimLevel 4
asked in Data Ingestion & Infrastructure
Check if a Variable (New Views) is null

Hello Community,how can I check if a variable from the new views is NULL? My challenge is to not use an filter in a PU_FUNCTION when the variable is empty.Variable: VAR_CONDITION_FILTERExample Value with multiple selection: ‘Y001’, ‘Y020’, ‘Y300’I want to use the variable in a PU_SUM Filter statement like in this way.PU_SUM("VTTK", "FREIGHT"."FREIGHT_CONVERTED", CASE WHEN '${var_condition_filter}' IS NOT NULL THEN "FREIGHT"."CONDITION" IN (${var_condition_filter}) END )But this don’t work. Error:A PQL error is preventing this query from executing.Syntax error near [IN] after reading [PU_SUM("VTTK", "FREIGHT"."FREIGHT_CONVERTED", CASE WHEN '${var_conditon_filter}' IS NOT NULL THEN "FREIGHT"."CONDITION" ] at line 9. Please refer to PQL documentation for available syntax. This here works, but I need the check if the variable is empty to avoid errors and to avoid that users has to select all the possible values.PU_SUM("VTTK", "FREIGHT"."FREIGHT_CONVERTED", "FREIGHT"."CONDITION" IN (${var_c

581
thomas.keim
Level 4

Badge Winners

  • Celonis Certified Data Analyst
    misri.hetalkumar.parikhas earned the badge Celonis Certified Data Analyst
  • Qualified to Create and Deliver Demos
    darko.duric11has earned the badge Qualified to Create and Deliver Demos
  • Celonis Technical Expert
    Pranavnhas earned the badge Celonis Technical Expert
  • Qualified to Get Data Into Celonis
    Pranavnhas earned the badge Qualified to Get Data Into Celonis
  • Celonis Technical Expert
    Gero Busatishas earned the badge Celonis Technical Expert
VIEW ALL
Community Code of Conduct