Skip to main content

Data Ingestion & Infrastructure

Discuss and ask questions about data integration, task mining, admin settings, and PQL extractors to optimize your infrastructure.

  • 805 Topics
  • 2,120 Replies
805 Topics
vamsimadhavg
Level 2
vamsimadhavgLevel 2
asked in Data Ingestion & Infrastructure
Need to Trigger Action Flow for Last Refresh Date

I am trying to make an action flow which gets triggered when the data load is more than a day old i.e., when the data load failed, and we are unaware of the failure. Current problem is that I am able to get the last refresh date from event log which is the MAX of the time stamp but to put a trigger to see if the MAx is more than a day old, the FILTER function in general does not allow aggregations. Is there a way out of it or any other path that can be taken to create an action flow to trigger a mail if the last refresh date is more than a day old.

462
vamsimadhavg
Level 2
christoph.kronb
Level 2
christoph.kronbLevel 2
asked in Data Ingestion & Infrastructure
Celonis STXL converter program for SAP R/3

We need for a use case the text which is stored in the SAP table STXL.I know that Celonis provides a converter program for the table STXL.https://docs.celonis.com/en/extracting-the-table-stxl.html We currently using SAP R/3 and S4H. Form the IT Department I got the message that the Celonis program only works for S4H. My question is, if some knows if it is possible to use the program for R/3 and what for changes we must do that it works for R/3. I hope someone can help me.Regards, Christoph

170
christoph.kronb
Level 2
raman.silot11
Level 3
raman.silot11Level 3
asked in Data Ingestion & Infrastructure
Delta filter using joined table

Hi everyone, I am currently working on optimising extractions in Celonis and applying delta for tables. But tables like BSEG, RSEG and CDPOS are not having the relevant date fields nor I can apply header table filters which I joined with these tables in delta filters. Is there any possible solution to apply delta on these tables ?

431
ilker.karak11
Level 3
celopeers.musup
Level 3
celopeers.musupLevel 3
asked in Data Ingestion & Infrastructure
Knowledge Model | APC [Data Consumption) Space tequirements / impact.

Knowledge Model | APC [Data Consumption) Space requirements / impact.Does a Knowledge Model have any APC [Data Consumption) Space requirements / impact?If yes, how can this be determined?  The space size is required to be known as to discern if we’ll exceed APC space.

341
manuel.wetze
Level 5
A
Level 3
ayushi.ade12Level 3
asked in Data Ingestion & Infrastructure
Hi All,Any best practices on optimizing the APC consumption or any solutions.

Hi All, Any best practices on optimizing the APC consumption or any solutions.

714
celopeers.musup
Level 3
johann.riede
Level 3
johann.riedeLevel 3
asked in Data Ingestion & Infrastructure
Setting up a OAuth 2.0 connection to SuccessFactors

Hi community, has anyone managed to set up a OAuth 2.0 connection to SAPs Success Factors yet?We're running in some issues there and I'd be happy to know if there are users out there who were able to successfully connect the two systems. Best regards!Johann

184
K
Level 2
jonas.venne
Level 2
jonas.venneLevel 2
asked in Data Ingestion & Infrastructure
Hi Everyone,I am trying to Filter my data/table for entries, if two columns are empty. If they are empty I don't want to see them at all (Filter them from all my data).But the Filter (X!='' and Y!='') is not working.

This Screenshot show my unfilteres table:As you can see I got a lot of data where my resultsqueeze and resultvalidation is empty. I want to filter those entries. What i did is Filter the whole sheet with the following statement:FILTER ("reportdataextraction"."resultsqueeze" !='' and "reportdataextraction"."resultvalidation" !=''); But the statement is filtering every entry where either one of the columns is empty. This is my result:  What am i doing wrong? Thank you for the help! Bests,Jonas Vennemann

183
dxb.miner
Level 3
henning.gertz
Level 3
henning.gertzLevel 3
asked in Data Ingestion & Infrastructure
PQL Formula

The data model looks like this. (1) is VBAP(2) is VBEP(3) is MARA(4) is MVKE   Now I created an OLAP table with columns from different SAP table. When I now want to include the SAP field MVGR1 from table MVKE is receive following error message.  How should the formula look like by using e.g. DOMAIN_TABLE or BIND together with PU-Function so that I can display the field MVKE.MVGR1 in the OLAP-Table?

929
dxb.miner
Level 3
victor.diens
Level 2
victor.diensLevel 2
asked in Data Ingestion & Infrastructure
How to use a table twice in an extraction

Hi everybody! I’m pretty new to celonis development and never ask a question here before, but I hope you can help me! I’m extracting data from SAP for Production data and now have the issue that I want to join the JEST table to two other tables (AUFK and AFVC) on the column AUFK.OBJNR and AFVC.OBJNR. How can I do that since downloading the whole JEST table means like 200 million entries :-( I hope you can help me!  Thank you very much!

463
florian.mrkvi
Level 4
xiangxiang.chen11
Level 2
xiangxiang.chen11Level 2
asked in Data Ingestion & Infrastructure
How can I have a common table on the temporary table from GENERATE_RANGE function and the case table

Hello,  I need a line chart showing created/open/resolved cases count every day. ​For x axis, I used GENERATE_RANGE function to generate DATE values as below. GENERATE_RANGE ( '1D' , TO_DATE ('2024-01-01', FORMAT('%Y-%m-%d')), TODAY()) For y axis(KPI), I use: COUNT_TABLE ( "casetable" ) However, the problem is there's no common table between this generated temporary table with the case table or activity table, so the case counts cannot be calculated based on this temporary table.   ​Does anyone know a good way to create that analysis, ideally only in frontend and not change the data model?

422
dxb.miner
Level 3
M
Level 3
matt.slocu12Level 3
asked in Data Ingestion & Infrastructure
How can I get a single value from a PU Aggregation?

Imagine I have two tables CasesCase IDCustomerActivitiesCase IDActivityEvent Time I would like to get the earliest "Upload" activity event time across all cases for a given customer. It is easy for me to get the earliest "Upload" activity for each Case with: PU_FIRST(Cases, Activities.Eventtime, Activities.Activity = 'Upload) But this returns a row for every case. I cannot figure out how to get the first of all these results. I tried domain tables, but couldn't figure out how to set it up. Any thoughts?

303
deepu.sebas11
Level 2
irfan.suria12
Level 4
irfan.suria12Level 4
asked in Data Ingestion & Infrastructure
Jira - Extraction Filter Statements

Hello community,I am struggling to set extraction filter statements for Jira tables. Some column names has a $ character that causes the following error:Extraction failed to start. Error for table 'issue': Field 'project$id' does not exist or you do not have permission to view it.With the INFO:additionalFilter: "\"project$id\" IN ('1234', '5678')" Furthermore, operators such as LIKE do not work. Error executing extraction. Unknown operator: LIKE. Any hints or best practices to share when configuring Jira extraction filter statements? CheersIrfan

220
irfan.suria12
Level 4
K
Level 1
kong.yangLevel 1
asked in Data Ingestion & Infrastructure
Is on-premise Celonis products still supported by SAP?

SAP Note: 3345663 - Termination of reseller agreement: On-premise solutions by Celonis  I read SAP Support note 3345663 and it says "SAP Support for the on-premise Celonis products will end." Is Celonis no longer supported on the SAP platform to extract data?

380
Sverre Klein
Level 6
E
Level 2
eduardo.horstLevel 2
asked in Data Ingestion & Infrastructure
Process copilot security settings

Is there a way to centrally managed the creation of process copilots? I know I can do it with the permissions to "create and maintain an studio package" but that is for everything within the package. Isn't there something that can be used to choose the studio packages where the process copilot can be created? https://docs.celonis.com/en/process-copilot.html

80
E
Level 2
E
Level 2
eduardo.horstLevel 2
asked in Data Ingestion & Infrastructure
How to restrict the creation of new data connections inside a data pool?

Would any of the possible permissions in here can help me with that:https://docs.celonis.com/en/managing-your-data-integration-service-permissions.htmlEdit Data Pool or Manage Data Pool I want to only let some groups have that possibility (creating new data connections)

30
E
Level 2
A
Level 2
abhishek.kumar35Level 2
asked in Data Ingestion & Infrastructure
Task Mining Query

Hi team, wanted to know does task mining recorder works properly and give accurate results for Windows 11? I've windows 11, but I was able to install the recorder, however few things didn't worked, like screenshots didn't got captured and I was unable to get process flow view of the task even after uploading the configuration file from task mining documentation app.

10
A
Level 2
M
Level 1
marcel.klinkLevel 1
asked in Data Ingestion & Infrastructure
How can I sum up the days where a target date is between 2 date fields. The summation should happen per day. A more detailed description can be found in the details.

I have a “Case Table” table. This has 2 columns “Case ID” and “Target date”. I also have a 2 table “Horizon” with 3 columns “Case ID”, “Start date” and “End date”. The relationship between the tables is as follows: “Case Table” N : 1 "Horizon". The primary key is “Case ID” Target with the help of a PQL:I want to build a graph where on the X axis are the days and on the Y axis are the number of target dates.If the “target date” from the “Case Table” is between the “start date” (exclusive) and “end date” (inclusive) from the “Horizon” table, it should be counted.  Example (The case ID of both tables is the same “Case ID” = 1): Target date = 20.09.2024Start date = 18.09.2024End date = 22.09.2024 Result:X axis: 17.09.2024; Y axis: 0X axis: 18.09.2024; Y axis: 0X axis: 19.09.2024; Y axis: 1X axis: 20.09.2024; Y axis: 1X axis: 21.09.2024; Y axis: 1X axis: 22.09.2024; Y axis: 1X axis: 23.09.2024; Y axis: 0 A 1 is added for all days where the target date lies between the start date (

120
manuel.wetze
Level 5
M
Celonaut
margaret.daven12Celonaut
posted in Data Ingestion & Infrastructure
Removal of SAP on-premise extraction (OPE) client for data integration (2024-08-20)

Removal of SAP on-premise extraction (OPE) client for data integration (2024-08-20) We removed the SAP on-premise extraction client from the Celonis Platform Download Portal on 2024-08-20. While any OPE instances that are already in use are unaffected, you now won't be able to set up a new SAP extractor or upgrade any existing SAP extractors. From 2024-08-20, you should use our on-premise client (OPC) instead. For information about accessing the OPC, see Installing on-prem clients. For information about the benefits of using OPC, see On-premise clients (OPC).

70
M
Celonaut
eby.t11
Level 1
eby.t11Level 1
asked in Data Ingestion & Infrastructure
Is there a possibility to integrate data from Elasticsearch(where relevant journey data is present in an index) with Celonis. Also if new data can be pulled via a schedule again if the source is Elasticsearch

Is there a possibility to integrate data from Elasticsearch(where relevant journey data is present in an index) with Celonis. Also if new data can be pulled via a schedule again if the source is Elasticsearch

190
eby.t11
Level 1
S
Level 2
shubham.gupta20Level 2
posted in Data Ingestion & Infrastructure
Is there a way to extract data from image or PDF and relate it with respect to the document number and

Is there a way to extract data from image or PDF and relate it with respect to the document number and We need to visit external URL and provide the document number. Each document will have its attachment which can be in PDF/Image format. We need to extract data from PDF/Image and prepare a table of the details (tabular format output) This table needs to be push to backend data model. Overall, we want to extract data from image/PDF relate with the PO Number/Accounting document number and push this data to backend data model.

153
mateusz.dudek14
Level 7
R
Level 3
ramanathan.iyerLevel 3
asked in Data Ingestion & Infrastructure
HAs anyone tried moving Data ingestion and transformation out side Celonis like in Azure DB or Google Big Query?and what is the Pros and Cons in this approach

HAs anyone tried moving Data ingestion and transformation out side Celonis like in Azure DB or Google Big Query?and what is the Pros and Cons in this approach

80
denis.bu
Level 2
.
Level 3
.146001156Level 3
asked in Data Ingestion & Infrastructure
Is it possible to join 2 tables that have no common table?

Would like to add fields from the AR_KNVV_KNVP_PA002 in an OLAP table along with fields from the BSEG table. However, have the error below - will the PQL functionality BIND ( ) work here?

560
A
Level 3
emil.vakhi12
Level 3
emil.vakhi12Level 3
asked in Data Ingestion & Infrastructure
Gather EMS analyses/views view statistics

Hi,Is there any way for a team admin to extract/gather view statistics for analyses and views?Ideally - event logs with events like object (analysis/view) + timestamp when it was viewed. Also it would be great to have user as well. The goal is to get an objective picture of how often particular objects are used instead of interviewing the users.For example, users say: "We use this report every day!", but you look into the stats and see that this report is open once per week/month. Therefore, maybe smth can be done to improve user experience. Thanks in advance!

110
emil.vakhi12
Level 3
H
Level 3
huseyin.simseLevel 3
posted in Data Ingestion & Infrastructure
Hi everyone,

Hi everyone, I am trying to calculate Automation Rate at Case Level.Within my dataset, each case have more than one activities. And, any of the activity could be either manual or automated. However, in my calculation, if case does have any manual steps; automation rate should be zero. Do you know how can I calculate it as a KPI?

42
H
Level 3
A
Level 2
ahmad.rabayLevel 2
asked in Data Ingestion & Infrastructure
Concat Date (of type DATETIME) and Time (of Type String)

Hi, I have the followng case, where I need to write the same code but in PQL:some ideas needed, I'm trying to implement this in PQL: "tbl_xxx"."date" + COALESCE(CAST("tbl_xxx"."time" AS TIME),'00:00') AS Date_Time As you see, "tbl_xxx"."date" is of type DATETIME, and "tbl_xxx"."time" is of type STRING.In PQL you have to use || instead of +, which only takes STRING, so the "tbl_xxx"."date" needs to be converted to STRING, but the issue is with "tbl_xxx"."time", because it can be NULL... Any ideas. Thanks

360
janpeter.van.d
Level 8

Badge Winners

VIEW ALL
Community Code of Conduct