Skip to main content

Data Ingestion & Infrastructure

Discuss and ask questions about data integration, task mining, admin settings, and PQL extractors to optimize your infrastructure.

  • 812 Topics
  • 2,131 Replies
812 Topics
Leslie Lei
Level 2
Leslie LeiLevel 2
asked in Data Ingestion & Infrastructure
Cross tables filtering doesn't work in N to 1 to N data model

Hello Celonis users, Good day. There is no chance for me to change anything in the data model and I mainly work in PQL Editor. My goal is to use fields from N-side tables to filter data and visualize it. More details:Data model: Table A [N] → [1] Table B [1] ← [N] Table C Table A:Material code            Warehouse            Procurement type1                                  Australia                    External1                                     USA                        Own produced2                                      UK                          Own produced2                                      USA                         Others3                                   Austrialia                   External3                                       UK                          Own produced Table BMaterial code              Responsible1                                        Martin2                                         Mary3                                         Julia Table CMaterial

685
Leslie Lei
Level 2
Leslie Lei
Level 2
Leslie LeiLevel 2
asked in Data Ingestion & Infrastructure
How to restrict the scope of filters

Dear Celonis users, I only know how to create a filter in KM and apply it to a table.  I would like to restrict the scope of filters, so that they will only have impact on one tab, when I put those filters in the filter bar. Is it possible to achieve this goal on Celonis web version?

30
Leslie Lei
Level 2
jan.hagem
Level 3
jan.hagemLevel 3
asked in Data Ingestion & Infrastructure
Automating Data Ingestion of External Statistics (Destatis) into Celonis

Hi everyone,in our Procurement analysis we are evaluating price developments and also compare them with external reference data provided by the German Federal Statistical Office (Destatis). The data is available as CSV flat files, which we currently download and manually integrate into our data model.I was wondering:👉 Has anyone already built an automated data ingestion pipeline for this kind of use case (regularly pulling official statistics or external CSV-based sources into Celonis)?👉 If yes, what approach or tools did you use (APIs, connectors, scheduled pipelines, etc.)?Any insights or best practices would be highly appreciated!Thanks in advance 🙌

80
jan.hagem
Level 3
Chethan Shetty
Level 3
Chethan ShettyLevel 3
asked in Data Ingestion & Infrastructure
How to upload new dataset to a table

Hey, I’ve an table I hope that is being creating initially using excel import. Now that I want to update the data in the same table. Can anyone please guide me to do that. Thanks in advance RegardsChethan

622
Chethan Shetty
Level 3
michael.hamme13
Level 2
michael.hamme13Level 2
asked in Data Ingestion & Infrastructure
Connect MongoDB – Database Type Issue

Hi everyone,I’m trying to connect MongoDB to Celonis. At first, I assumed that I’d need to use a custom JDBC driver (based on a thread that I’ve read here and the Celonis documentation).However, when I reached the step to select the “Database Type” in the connector, I couldn’t find a fitting option for MongoDB.Has anyone successfully set this up?Should I use a specific Database Type in this case? Or is there another workaround to get MongoDB connected?Thanks for any tips or guidance! 

271
janpeter.van.d
Level 8
kazuki.sugiy12
Level 2
kazuki.sugiy12Level 2
asked in Data Ingestion & Infrastructure
Silent Installation and execution of Task Mining

Hi, we’re planning to use task mining for one of our customers.Would it be possible to install a desktop application on employees’ laptops without them noticing?Also, once the application has been installed, can it be run silently so that employees are not aware that they are being monitored?

241
julia.bauer
Level 5
mukesh.gupta11
Level 4
mukesh.gupta11Level 4
asked in Data Ingestion & Infrastructure
Does anyone have the steps to do the data integration with the Dynamics 365 for order to cash process?

Does anyone have the steps to do the data integration with the Dynamics 365 for order to cash process?

73
asdfg
Level 3
andres.naran12
Level 4
andres.naran12Level 4
asked in Data Ingestion & Infrastructure
How to get KPIs values through PyCelonis?

Hi everyone, some months ago, I faced a challenge related to getting variables from a view using PyCelonis. Today I have another challenge where I need to get the result from multiple KPIs and send it back to the data pool.At the beginning, I used the module “Get Rows” from action flows, and I was able to get the values from each KPI. However, in some KPIs, the values that I’m getting inside actions flows, are completely different from the ones that exist in an OLAP’s table where those KPIs are being used. I don’t understand the reason for this, so I was thinking of a workaround solution with PyCelonis, where I would get the values from each KPI and create a dataframe that can be sent to the datapool.I can see in the documentation that we can get the values from records, but only of those records that exist in a data model. But I cannot see in the documentation how to deal with KPIs.Does anyone know if is there a way to get the value of multiple KPIs with PyCelonis, pls?

140
andres.naran12
Level 4
emilia.stepi
Level 2
emilia.stepiLevel 2
asked in Data Ingestion & Infrastructure
Restricting access to specific data model column

Hello everyone,I want to restrict access to a specific column in a data model so that non-admins cannot see the values in that column or the column itself. Is there a way to achieve this using data permissions? Thank you!Best,Emilia

322
benjamin.delay
Level 2
carmen.kaluz
Level 3
carmen.kaluzLevel 3
asked in Data Ingestion & Infrastructure
Use App Key for content-cli in ML workbench

Hi all,I would like to export datapools as json using content-cli in the ML workbench in order to backup them in Git. I have created an API key in my profile and used it in a python script with content-cli pull data-pool --id data-pool-id, which successfully generated a json of the datapool.However, I don't want to do this with a user-based API key but with an Application key (created under admin&settings).When I run the same command using the Application key, I get the error "error: The provided team or api key is wrong."With get_celonis(api_token=“applicationkey”, key_type="APP_KEY") however, I can see the Application key and all its permissions.Is content-cli possible with an Application key and if so, how can I use it?

353
carmen.kaluz
Level 3
thomas.keim
Level 4
thomas.keimLevel 4
asked in Data Ingestion & Infrastructure
Usage of PQL Function QUANTITY_CONVERT

Hello Community,I need you help in using the PQL Function QUANTITY_CONVERTMy objective is simple: I want to convert the SAP Field LIPS.BTGEW to Tons (TO). The original Unit is stored in field LIPS.GEWEI.With this try I get an errorQUANTITY_CONVERT("LIPS"."BRGEW", FROM ("LIPS"."GEWEI"), TO ('TO'))A PQL error is preventing this query from executing.Syntax error near [)] after reading [QUANTITY_CONVERT("LIPS"."BRGEW", FROM ("LIPS"."GEWEI"), TO ('TO')] at line 32. Please refer to PQL documentation for available syntax. What's the problem? If I understand the docu (https://docs.celonis.com/en/quantity_convert.html) correctly, I must not provide a conversation rates table?This operator converts quantity units based on a QUANTITY_CONVERSION_RATES table, which is optionally provided by the user. If not provided by the user, a default QUANTITY_CONVERSION_RATES table will be used. Where can I see the contents of the default QUANTITY_CONVERSION_RATES table?Thanks for helpBest, Thomas 

291
thomas.keim
Level 4
douglas.souza
Level 3
douglas.souzaLevel 3
asked in Data Ingestion & Infrastructure
Filter Statments Extractions

I need to use a filter in my extraction, but the field in the table I'm bringing it from has a separate name "Date Archived" Example.When I use it like this "Date Archived" >= '2024-01-01 00:00:00', it doesn't load, I believe it's because of the space between the column name. Does anyone know how to solve this?I've tried to put the column as a parameter, but I still can't. 

485
douglas.souza
Level 3
harshsh
Level 2
harshshLevel 2
asked in Data Ingestion & Infrastructure
Replication Cockpit

How do I see the replication cockpit also along with the data job. How do i set the change table and trigger for it?

331
thomas.guerr
Level 4
gauri.pai11
Level 3
gauri.pai11Level 3
asked in Data Ingestion & Infrastructure
Data Extraction filter help

Hi team,I need help in understanding how to apply filter for KONP table in data extraction joining another table.My requirement is:1.For A018 table - I need all the records where A018.MATNR =  EKPO.MATNR.2.For KONP table, all the records from KONP table where A018.KSCHL = KONP.KSCHL If i am trying to implement 1st point in filter condition of A018 extraction as screenshot below,Data extraction fails with error : Error loading resource A017. The multi-part identifier "ts1.EKPO.MATNR" could not be bound.Can you please help in this? Thank you in advance.

121
thomas.guerr
Level 4
M
Level 2
mel.schic11Level 2
asked in Data Ingestion & Infrastructure
I would like to connect two fields that have an indirect N:N relation. Is this something which I can do via a PU_Function?

Within an OLAP table, I would like to use both the Delivery Line Item Date ("VBEP"."EDATU") and the shipment gross weight ("LIPS"."BRGEW") Does anyone know how I can apply a PQL function to combine these tables? Please find the error below. I looked into the PQL documentation but I can't find the solution. Error in formulaNo common table could be found. The tables ["VBEP"] and ["LIPS"] are connected, but have no common table. This means that they do not have a direct (or indirect) 1:N or N:1 relationship. Join path: [VBEP]N <-- 1![VBAP]!1 --> N[VBFA]N <-- 1[LIPS]. For more information on the join path, search for "Join functionality" in PQL documentation.

342
kandi.naidu
Level 4
janpeter.van.d
Level 8
janpeter.van.dLevel 8
asked in Data Ingestion & Infrastructure
What does the Eventlog Automerge option in Data Model overview do?

Hi all, Since a short while we see the option eventlog automerge in the data model overview (see attached screenshot). However, this is not mentioned in the Release Notes or online documentation. Does somebody knows what this option is for? Thanks in advance! Bests, Jan-peter! Celonis_Automerge|690x150

153
Ushmita Patil
Level 2
thomas.guerr
Level 4
thomas.guerrLevel 4
asked in Data Ingestion & Infrastructure
Extractor configuration for data transfer from MS Sharepoint to Celonis

Hello,I am trying to get data from a Microsoft sharepoint list to the Celonis Platform.According to the Flowchart provided by the API-Team - When to use the Standard Data Ingestion API (and other methods) - the extractor builder should be used for a Sharepoint data transfer, since the REST API is exposed.I tried to set up the connection, but I fail when configuring the endpoint, since I get xml-data even if I specify that I would like to have a json structure (Request Parameter - Accept: application/json;odata=verbose)Does anyone have experience with connecting the MS universe with the Celonis universe?I also found a workaround that works (Push CSV to Data Integration), but I would rather orchestrate all the data loads in the Data Integration environment instead of using Process Automation modules to make a data transfer.Thanks for your support!BR THomas

972
thomas.guerr
Level 4
swapnil.karad12
Level 2
swapnil.karad12Level 2
asked in Data Ingestion & Infrastructure
Linking tables for O2C and P2P in SAP

I want to visualize end to end process of a buisness; order received → purchasing begin→ production → payments to vendors → order shipped to customer → cutomer payment receivedhow i can connect in SAP all this O2C to P2P; what are common tables to link all this

190
swapnil.karad12
Level 2
gauri.pai11
Level 3
gauri.pai11Level 3
asked in Data Ingestion & Infrastructure
Content-cli Profile creation

Hi team,I am using this command content-cli profile create.I am asked for name,url and api key.I have added all the three, but it gives me error as - The provided team or api key is wrong.Name of the profile: XYZYour team (please provide the full url): https://XYZ.eu-5.celonis.cloudProfile type: OAuth Device Code (1), OAuth Client Credentials (2) or Application Key / API Key (3): 3Your api token: XYZ*API token is directly fetched from celonis-edit profile-api key.Is it necessary to have admin role for this.Currently i have analyst access.Please help me with this.

311
abdulnour.duale11
Celonaut
haruka.yano
Level 2
haruka.yanoLevel 2
asked in Data Ingestion & Infrastructure
Object transformation preview once showed and disappers

HiI’m curently working on OCPM Data Pool and have trouble in an object transformation.In the Currecny Conversion object(o_celonis_CurrencyConversion),I checked that I can see data record in Preview,and executed publish to development.Then I ran the Data Job Transformation(full load), succeeded,ran the Data Model Load, succeeded,but at the end o_celonis_CurrencyConversion has no data.I don’t know why but when I go see the object preview again, the preview doesn’t show anything.Has anyone had this kind of experience ever?If yes please kindly help. FYI as the prerequisite, - the relevant sap table data is sycronized from another data pool - the object and its transformation script is celonis default one​​​​​​​ - Data job is done by the duplicate of test:ocpm-data-job(celonis default test data job) - the reason why I use the duplicate data job is that the original one returns an error for schema.

170
haruka.yano
Level 2
PhuongBHN
Level 3
PhuongBHNLevel 3
asked in Data Ingestion & Infrastructure
How to enable the ‘Push data into Celonis’ Data connection option within my Data pool

Hello everyone, I am unable to locate the ‘Push data into Celonis’ Data connection option within my Data pool.Could you guide me on how to enable this feature?!--endfragment>My Celonis  I want the ‘Push data into Celonis’ as the image below 

553
abdulnour.duale11
Celonaut
Suveesh_adessi
Level 2
Suveesh_adessiLevel 2
asked in Data Ingestion & Infrastructure
Celonis Audit API - Group & Group Members

I’ve been extracting Celonis audit logs API for user-platform analysis. I couldn’t find the right field for group name and group members in the extracted data. If you are someone done this already, please share your knowledge to get these data.Also, please mention if there is any dedicated API endpoint is available to extract those data.

241
jonas.hess
Level 3
MrAj
Level 3
MrAjLevel 3
asked in Data Ingestion & Infrastructure
How to Trigger a Machine Learning Script from a Button in Celonis?

Hi everyone,I need help figuring out how to execute a Machine Learning script directly from a button in my Celonis app in views (dashboard). I believe I might need to use Action Flows, but I’m unsure about the exact steps to make this work.Here’s what I want to achieve: When a user clicks a button in the Celonis app, it should trigger an ML script. The script could be hosted externally (e.g., a REST API) or executed within Celonis.( atm its not hosted, there is just a file in ipynb format, stored in ml workbench in  analysis. I need guidance on how to set up the Action Flow and link it to the button. Has anyone implemented something similar? If so, could you share the best approach and any potential challenges I should be aware of?Thank you in advance!

280
MrAj
Level 3
MrAj
Level 3
MrAjLevel 3
posted in Data Ingestion & Infrastructure
How to Trigger a Machine Learning Script from a Button in Celonis?

 Hi everyone,I need help figuring out how to execute a Machine Learning script directly from a button in my Celonis app in views (dashboard). I believe I might need to use Action Flows, but I’m unsure about the exact steps to make this work.Here’s what I want to achieve: When a user clicks a button in the Celonis app, it should trigger an ML script. The script could be hosted externally (e.g., a REST API) or executed within Celonis.( atm its not hosted, there is just a file in ipynb format, stored in ml workbench in  analysis. I need guidance on how to set up the Action Flow and link it to the button. Has anyone implemented something similar? If so, could you share the best approach and any potential challenges I should be aware of?Thank you in advance!

200
MrAj
Level 3
MrAj
Level 3
MrAjLevel 3
asked in Data Ingestion & Infrastructure
Configuration Management for Process Flow in Celonis by Gui or flat files

Current State:Our Celonis process mining implementation includes configuration settings for process flow and status updates in our data transformation pipeline Currently, only developers can modify these configurations through SQL queries and code changes End users (procurement specialists/buyers) have no way to modify process flow configurations without technical knowledgeRequirement Question: What is the most effective way to enable non-technical end users to update process flow configurations in Celonis without requiring knowledge of SQL, PQL, YAML, or coding?Specifically:What data format should be used for configuration (CSV, Excel, UI form, etc.)? Where should this configuration be stored in the Celonis architecture? How will the configuration files be uploaded/modified by end users? How can we ensure the uploaded configurations are properly integrated with our data transformation steps?The solution should accommodate users with no programming knowledge while maintaining data inte

90
MrAj
Level 3

Badge Winners

  • Qualified to Deliver Business Value with Celonis
    awanish.kumarhas earned the badge Qualified to Deliver Business Value with Celonis
  • Solution Creation Expert
    zaved ahmedhas earned the badge Solution Creation Expert
  • Celonis Technical Expert
    zaved ahmedhas earned the badge Celonis Technical Expert
  • Qualified to Build Views
    zaved ahmedhas earned the badge Qualified to Build Views
  • Celonis Certified Data Analyst
    hazrul.hashi11has earned the badge Celonis Certified Data Analyst
VIEW ALL
Community Code of Conduct