Data Ingestion & Infrastructure

I need to create a connection from Celonis - Data Integration to AWS S3 without using an access key, as our policy requires password rotation every 90 days, and I prefer to avoid manual updates. Do you have any suggestions on how to achieve this?

Hi Celonis Community, I’m working on optimizing data jobs in Celonis Data Integration for a client with an extremely heavy SAP system (Saturn). Our data jobs currently take around 1.5 hours to process, and the client wants to reduce this to 0.5 hours. Despite optimizing SQL queries (e.g., early filtering, minimizing joins) and disabling unnecessary objects, the massive data volume seems to be the main bottleneck. Here are some examples of the volumes we’re handling: - SalesOrderScheduleLine Transformation: 119M rows in CDPOS, 59M in CDHDR, cost of 24M.- CreditMemoItem Transformation: 123M in VBAP, 824M in VBFA, 178M in VBRP, cost of 83M.- CustomerInvoice Transformation: Similar volumes, 824M in VBFA, 178M in VBRP, cost of 83M. We’ve explored strategies like incremental loads and partitioning, but the sheer volume from Saturn (e.g., 824M rows in VBFA) still slows us down. I found in the documentation that the Payment Term Checker slows with 100M+ invoices, suggesting infrastructure migh

Hello, I'm working on connecting serviceNow to Celonis, I did Task Extraction (OK) → Task Transformation (OK) and when I loaded data it works (loaded rows OK), but when I press "set as activity table" it gives me the message :  "Preview not available". A table preview is currently not available. The cause may be due to a missing data pool or connectivity issues” without columns to select “Case ID”, “brief description”, “date”.I need to know why the columns are not showing up in views (after transformation task) but showing up in the original tables….for information: I tested the data in task extraction, task transformation and loaded data model and they contain all data of the views. I also execute data job for all, but the columns still don't appear.Thank you in advance for your help,

Hi Team,I am facing issue while creating dropdown, I have used below PQL QueryCASE WHENRUNNING_TOTAL(SUM("PO_LINE_ITEM"."NET_VALUE_LINE_ITEM_IN_REPORTING_CURRENCY")) /GLOBAL(SUM("PO_LINE_ITEM"."NET_VALUE_LINE_ITEM_IN_REPORTING_CURRENCY")) <= 0.8 THEN 'Top 20% Vendors' ELSE 'Remaining Vendors'ENDwhile using this query in the dropdown i am getting only one option either 'Top 20% Vendors' or  'Remaining Vendors' but i want both the options in the dropdown. So pls kindly revert ASAP.Thanks,Kiranmayi

Hi everyone,I received the following error message during the last few data uploads:The load failed: Load failed. Error Message: Could not execute foreign key join: there are duplicates on both sides of the specified key relationship. Table "PM_AFRU": key columns ("MANDT", "AUFPL", "APLZL"), key ('100', '1016228353', '00000001'), rows 0 and 1. Table "AFVC": key columns ("MANDT", "AUFPL", "APLZL"), key ('100', '1018097598', '00000001'), rows 1603274 and 1608762.Everything was working fine about a year ago, and I have checked the table relationships—they are correct. What could be the issue?Can anyone help?THX, M

HI Team,I have One table which has Vehicle ID and other Table which has PartDetails. I want to count claim number from this but both columns are coming from different tables. When I tried using this with DOMAIN TABLE it is giving me following error message:-Pull-Up-function could not be executed. PU_COUNT_DISTINCT: Cannot execute PU-function on tables [<DOMAIN_TABLE ( "O_CUSTOM_CLAIM"."VEHICLE_ID" , "O_CUSTOM_CLAIMLINEDETAILS"."PVTXT" )>] and ["o_custom_Claim"], since they do not have a 1:N relationship. You might want to make use of the BIND function. Join path: [DOMAIN_TABLE ( "O_CUSTOM_CLAIM"."VEHICLE_ID" , "O_CUSTOM_CLAIMLINEDETAILS"."PVTXT" )]1 --> N[o_custom_ClaimLineDetails]N <-- 1[o_custom_ClaimLines]N <-- 1[o_custom_Claim]. For more information on the join path, search for "Join functionality" in PQL documentation.Can anyone assist?

Out of the blue, every single Excel module in Action Flows has stopped working. Has anyone gone through something similar recently? Has any announcement on this topic been done by Celonis?The screenshot of the error is meant to help better understand the issue as it seems to be a simple ‘access’ problem, but every account that’s been used within those modules has the same issue and all of then started at the same time, so it feels something structural is going on…Either way, if anyone has some insight on this it would be greatly appreciated! Thanks! 

I am trying to retrieve a count of a specific column based on a **CASE WHEN** condition using PQL. However, I am encountering the following error:   *"A PQL error is preventing this query from executing. No common table could be found. The tables ["LINK"] and ["JS_PGT_Resolved"] are connected, but have no common table. This means that they do not have a direct (or indirect) 1:N or N:1 relationship. Join path: [LINK]N <-- 1![SUPPORTCASE]!1 --> N[JS_PGT_Resolved]. For more information on the join path, search for 'Join functionality' in PQL documentation."*   I would appreciate any guidance on how to resolve this issue. Thank you! 

Hi,I was wondering the following: I want to export my entire data pool (extraction, transformations and data models) using the pyCelonis API. I have been trying to code it myself but there doesn't seem to be a straight forward way to do it. I was wondering if any of you have been using this method and can help me find the right code to use.  The use case is to back up our work once per day. A very nice to have would be an additional script to use the exported data pools to automatically recreate the back-upped work.  In other words: Run script > back up written to files > run script > recreate celonis environment in case of total loss of data

Dear Experts, If there is decision to be made in terms of best practices on connectivity to S3  , could you comment on option 1 : connecting S3 bucket directly which contains source tables option 2 : transforming data from S3 using databricks converting into activity and case tables and placing it to seperate bucket and then data shared with celonis ? Thanks, Shaif 

Hey all looking for some insights whether this will work or not if I want a previous years data e.g ( I have 2023-2025 data in CDHDR but I also want 2022 ) can I change the UDATE filter from ( 2023 to NOW ) to ( 2022 to NOW) and disable the change date check  and do a DELTA load to only extract fresh records that are not present in celonis ( even if it’s from the previous year of already ingested data )

Hi,I'm trying to create an Action Flow that allows me to convert a CSV file to XLSX and then send it via email.Current Flow:The user enters their email in the application view and clicks the "Send Email" button.A webhook retrieves the data. We then use Celonis Get Rows to fetch data for a specific Account ID (one Account ID may have more than one document).   The data from Get Rows is added to a CSV file.Expected Solution:Instead of saving the data to CSV, I would like to:Aggregate the data differently, e.g., fetch it from another HTTP output or webhook. Use ML Workbench in Celonis to convert the JSON format to XLSX. Then send the XLSX file back to Action Flow to attach it to the email.Question:Is it possible to implement such a solution? Or is there a simpler way to generate an XLSX file as an output?

Can anyone has this mapping information?I would like to see the list that shows the mapping of Celonis activities (eg: AP process) to SAP tables.Like while calculating the clear invoice steps which tables, timestamps are used in SAP. Similarly for other activities. This will be a good reference document to begin with and help to customize the activities in Celonis if we need to do so.Appreciate your inputs here.

Dear all , Is there a handy reference for Oracle EBS tables related to procurement , order management and inventory for process mining ? 

Hi All,I wanted to know if there is any way to implement looping concept in vertica SQL in celonis. I have a use case where I have to compare two tables on the basis of a certain date type column and this comparison must be done for every month in the data type column. So, in this scenario I wanted to use the looping concept as in other coding languages like python.Thank you in advance,Akshay

Hello fellow Celonis users, I am trying to create a snooze function for my clients so that certain data will not be taken into account for analysis for the snooze set amount of time. I was wondering if it was possible to set a timer on an Action flow so that could be made possible or if you would suggest taking a different route? thank you in advance

Good morning, can someone please help with the PQL to return the case with the minimum cycle time? Thank you. Denise

SKILL:OUTPUT ATTACHMENT:Can someone explain why this is happening that encoding is not perform - what do I missed?

Hi all,I am lately thinking of how to make my life as Data Engineer easier and get a up-to-date documentation of the current backend relationships. The goal is to see: which tables from source are transformed into which VIEWS / TABLES downstream. I want to have a quick and accurate answer to:Which tables are sources for a given table in the datamodel and how are they connected? What is the potential impact on downstream tables if I change something on a given extraction. Which tables are not in use? Potentially even better would be a granularity of column and not only View/TableI believe those questions belong all to the concept of Data Lineage. Which in my opinion is currently very poorly supported within Celonis.I would like to open the discussion on how to best do this and if this is a relevant problem to more people:What is you workflow when you face the questions above? (How) are you guys documenting the SQL Backend? Is it done manually?

Hi, I'm using an API to extract datafor this purpose, the scope of Celonis, I am trying to adapt the configuration parameters that are 100% functional in POSTMAN, but when building Extractor I am finding it difficult to adapt 1) The API Resource first requires a POST request to generate the Bearer Token: Request POST: https://prod.api.wehandle.com.br/loginParamsAuthorizationHeader:Body:Scripts:Result Body Response:2) The second request uses the generated Token to GET data from the API Request GET: https://prod.api.wehandle.com.br/integrations/main/employees?page=1&limit=50&changes_to=2025-01-14T23:59:59.000Z&changes_from=2025-01-01T00:00:00.000Z&employee_address=true&employee_services=true&employee_manager=true&employee_supplier_company=trueParamsAuthorizationHeadersResult Body ResponseI'm having trouble recreating this integration successfully via Extractor BuilderPerhaps my starting point would be to decide the Authentication Method betweenBearer Authentic

Hi everyone, I need help with a filter.I want to put a filter on a kpi that only allows it to show the previous months data. The filter must be dynamic so that when I select a month from the date picker that it would always calulate the previous months data.I am currently using this PQL: FILTER "CEL_ACTIVITIES_OTT"."first_eventtime" >=ADD_MONTHS(PU_MAX ( CONSTANT ( ), FILTER_TO_NULL(ROUND_MONTH("CEL_ACTIVITIES_OTT"."first_eventtime"))),-1)AND"CEL_ACTIVITIES_OTT"."first_eventtime"<PU_MAX ( CONSTANT ( ),FILTER_TO_NULL( ROUND_MONTH("CEL_ACTIVITIES_OTT"."first_eventtime")))The problem is that I have to select both months from the date picker and Ideally I would like to only select one month. 

Hi,We have a scenario where records can be created/deleted in source system multiple times.We are currently using the replication cockpit for extracting changes, but for deleted records we do not get the full information of the deleted records, we only get the primary key of the table and a change indicator = ‘D’ (Deleted) and we would like to know what the values of all the attributes for the specific records were prior to the deletion. We have located the following staging tables and views:_CELONIS_TMP_{TableName}_NEW_DATA  _CELONIS_TMP_{TableName}_TRANSFORM_DATA {TableName}_DELETED_DATA Unfortunately, it seems like the _CELONIS_TMP_{TableName}_NEW_DATA table only includes records with change indicator = ‘U’ (Updates) and not change indicator = ‘I’ (Inserts). This makes it difficult for us to map what the historical values were, if we have both inserts and deletions within the same execution of the replication cockpit.Does anyone know of a staging table for inserts we can use, befor

Hi All,When using a case when statement how would I go about excluding all the values which are not true from the result.Lets say:CASE WHEN PROCESS EQUELS 'A' -> 'B'THEN 'A to B'WHEN PROCESS EQUELS 'C' -> 'D'THEN 'C to D'Now I'm only interested in plotting the values fall in the 'buckets' basically. How do I exclude the rest which are NULL (very large number of cases).

Hi all, I am using the COUNT function to get the total number of invoices, PO, and line items that fulfill the conditions in the code statements. The problem is the KPIs I am using were created using COUNT functions, which makes it impossible to get the total numbers by nesting the entire code in another COUNT function. This gives the error message:  Aggregate expressions cannot be nested inside aggregate expressions. Does anyone have an idea how I can resolve this issue? The code is added also. 

I am trying to make an action flow which gets triggered when the data load is more than a day old i.e., when the data load failed, and we are unaware of the failure. Current problem is that I am able to get the last refresh date from event log which is the MAX of the time stamp but to put a trigger to see if the MAx is more than a day old, the FILTER function in general does not allow aggregations. Is there a way out of it or any other path that can be taken to create an action flow to trigger a mail if the last refresh date is more than a day old.