Skip to main content

Data Ingestion & Infrastructure

Discuss and ask questions about data integration, task mining, admin settings, and PQL extractors to optimize your infrastructure.

  • 801 Topics
  • 2,108 Replies
801 Topics
C
Level 3
chakradhar.pokalLevel 3
asked in Data Ingestion & Infrastructure
celonis data extraction from source system is pull based or push based ?

celonis data extraction from source system is pull based or push based ?

00
C
Level 3
C
Level 3
chakradhar.pokalLevel 3
asked in Data Ingestion & Infrastructure
During Step 4: Does this step generate a flat file or is all data held in memory?I think It will not generate flat file. Just data will be stored on private cloud. Please suggest me with correct answer.

During Step 4: Does this step generate a flat file or is all data held in memory?

20
C
Level 3
dennis.arrin
Level 3
dennis.arrinLevel 3
asked in Data Ingestion & Infrastructure
Filtering a date range around a specific activity per case

Hi guys! I’m trying to apply a particular filter to a dataset with many activities. The scenario is as follows. The raw event log contains activities done in a general hospital for over a year.If I want to get the process for a specific activity, I would normally just filter ‘Include only cases that contain ‘Knee Replacement Surgery’.  However, because there is so much going on in the hospital, such a filter is not enough. For example, if a person underwent a knee surgery is june 2021, the process map will also try to include other procedures that this person did in january 2021 or december 2021, even though these have nothing to do with the knee replacement surgery. For that reason, I’m trying to narrow the filter down as follows: “Provide an event log that only includes data of a maximum of 2 weeks in the past after the activity ‘Knee replacement surgery’ and a maximum of 2 weeks after this same activity.” Note that this is NOT the same as applying a general time filter on the event

70
dennis.arrin
Level 3
R
Level 4
raman.silot14Level 4
asked in Data Ingestion & Infrastructure
data limit of celonis ems

I am curious how much data can we extract and store in celonis ems for both free training use and production use?

10
A
Level 1
tushar.konde
Level 3
tushar.kondeLevel 3
posted in Data Ingestion & Infrastructure
dell.us-1.celonis.cloud

---------------------------------------------------------------------------HTTPError Traceback (most recent call last)~/.local/lib/python3.8/site-packages/pycelonis/celonis_api/base_objects/ibc.py in api_request(self, url, message, method, timeout, get_json, threaded, **extra) 273 try:--> 274 r.raise_for_status() 275 except exceptions.HTTPError as e: /opt/conda/lib/python3.8/site-packages/requests/models.py in raise_for_status(self) 952 if http_error_msg:--> 953 raise HTTPError(http_error_msg, response=self) 954  HTTPError: 404 Client Error: Not Found for url: https://dell.us-1.celonis.cloud/process-mining/analysis/v1.2/api/analysis/325c3e97-3a13-4c7a-b818-ac392eebd840/bookmarks/809dfc8f-86c4-4210-9818-a4b2734543c6 During handling of the above exception, another exception occurred: HTTPError Traceback (most recent call last)/tmp/ipykernel_16778/3654934662.py

125
G
Level 8
steven.bernd
Level 3
steven.berndLevel 3
asked in Data Ingestion & Infrastructure
KPI in component Filter

Hello, everyone, I would like to add my KPI "SUM(CASE WHEN (LEFT("COEP"."KSTAR", 1) IN ('6','9')) THEN "COEP"."WTGBTR" ELSE NULL END)" to the component filter to hide the 0 values. What do I have to pay attention to here, because FILTER SUM(CASE WHEN (LEFT("COEP"."KSTAR", 1) IN ('6','9')) THEN "COEP".."WTGBTR" ELSE NULL END) != 0 does not work. Greetings

20
G
Level 8
irfan.suria12
Level 4
irfan.suria12Level 4
posted in Data Ingestion & Infrastructure
WINDOW_AVG (or alternatives)

WINDOW_AVG (or alternatives) Hi all, I calculate the average cycle time from PO Item creation to Record Goods Receipt per material with the following query: PU_AVG ( DOMAIN_TABLE ( "EKPO"."MATNR" ) , DAYS_BETWEEN ( PU_FIRST ( "EKPO" , "_CEL_P2P_ACTIVITIES"."EVENTTIME" , "_CEL_P2P_ACTIVITIES"."ACTIVITY_EN" = 'Create Purchase Order Item' ) , PU_FIRST ( "EKPO" , "_CEL_P2P_ACTIVITIES"."EVENTTIME" , "_CEL_P2P_ACTIVITIES"."ACTIVITY_EN" = 'Record Goods Receipt' ) )) Now, I want to consider only the latest 10 PO items into the calculation based on the creation date. How can I do that? I tried the WINDOW_AVG function, but it does not work: WINDOW_AVG( <above query>, 0, 10, ORDER BY (PU_FIRST ( "EKPO" , "_CEL_P2P_ACTIVITIES"."EVENTTIME" , "_CEL_P2P_ACTIVITIES"."ACTIVITY_EN" = 'Create Purchase Order Ite

62
irfan.suria12
Level 4
V
Level 4
vivek.mLevel 4
posted in Data Ingestion & Infrastructure
Hello PQL experts,

Hello PQL experts, Am looking for SAP Order Confirmation related KPIs ?  Eg : Number of Orders Vs Durations ( Days) .Any inputs is appreciated in Advance ! Br.Vivek

14
V
Level 4
.
Level 3
.146012012Level 3
asked in Data Ingestion & Infrastructure
PQL question;I am trying to create a query that counts activities after a specific activity is captured.

PQL question; I am trying to create a query that counts activities after a specific activity is captured.

10
irfan.suria12
Level 4
S
Level 3
siddhesh.naik11Level 3
asked in Data Ingestion & Infrastructure
Enabling Primary Key

Based on understanding of Vertica SQL/Postgre SQL, following piece of script I am implementing to enable primary key. But even after successful, error free execution of these statements, I can see Celonis allows INSERT with same values. Can you help me in understanding if I am falling short on something to enable primary key? CREATE TABLE _CEL_ACTIVITY_PRIMARY_TEST(    _case   varchar(25),    activity_name varchar(100) not null,    eventime DATETIME not null,    PRIMARY KEY(_case)); SELECT * FROM _CEL_ACTIVITY_PRIMARY_TEST; INSERT INTO _CEL_ACTIVITY_PRIMARY_TEST VALUES ('1234','act1','2022-08-22T09:46:46');

00
S
Level 3
N
Level 2
niyi.ogunb12Level 2
asked in Data Ingestion & Infrastructure
File upload error

I get the following error message when I attempt to upload a flat file (csv/excel): "There was an error during file upload. Please check your file type and content". Any suggestion how to fix this?

00
D
Level 1
.
Level 2
.14600957Level 2
asked in Data Ingestion & Infrastructure
Connecting Celonis4.7.2 to GCP BigQuery using Simba JDBC Driver

We are trying to connect Celonis4.7.2 (onprem) to the GCP using the BigQuery Simba JDBC Driver supplied by Google. The .jar files have been unzipped into <installDir>\\lib but trying to connect from the frontend, we encounter the following errormessage: Has anyone successfully connected Celonis4.7 (or any other onprem version) to the GCP?  Thank you for your help! Cheers, Felix

10
V
Celonaut
M
Level 2
marcia.cristina.silvaLevel 2
asked in Data Ingestion & Infrastructure
Hi! I'm working with Celonis task mining, capturing data with filter by applications and URL, I have two different users, one can capture the URL and the other cannot. What could be the problem? Would it be firewall?

Hi! I'm working with Celonis task mining, capturing data with filter by applications and URL, I have two different users, one can capture the URL and the other cannot. What could be the problem? Would it be firewall?

00
R
Level 3
R
Level 4
raman.silot14Level 4
asked in Data Ingestion & Infrastructure
Data Model VS Joins

For example, in P2P data model we connect EKPO and EKKO using foreign key relationship, but then also we mention that in data jobs while creating EKKO. So my question is why we are doing the double work here? Also what is going to happen if I do not connect two tables that are required to be connected in the data model ?

00
T
Level 1
steven.bernd
Level 3
steven.berndLevel 3
asked in Data Ingestion & Infrastructure
Count days between activities, but with condition

Hello, I need your swarm knowledge. I need a combination of the following formulas.... TRIMMED_MEAN(CALC_THROUGHPUT(FIRST_OCCURRENCE[<%=Process_Start_1%>] TO LAST_OCCURRENCE[<%=Process_Start%>], REMAP_TIMESTAMPS("T_CEL_AT_HP5"."EVENTTIME", DAYS, MONDAY TUESDAY WEDNESDAY THURSDAY FRIDAY)), 5, 5) ...if the condition beforehand... "/RWEE/PMCP_TAUFK"."ARBPL_AN" LIKE 'S-%-S%' ...grips. Hope for help :-) thx

00
steven.bernd
Level 3
M
Level 2
menno.van.e12Level 2
asked in Data Ingestion & Infrastructure
Extract BPMN model including additional attributes

Hi,  I would like to extract a BPMN model from my Event Log but want to include additional attributes (like user role that is related to the activity, application in which the specific activity takes place, etc.) in order to generate a complete model of the process.  Is this possible, or can you only extract a simple BPMN model with the activities and connections?  Thanks in advance!

70
B
Celonaut
D
Level 2
daniel.nLevel 2
asked in Data Ingestion & Infrastructure
How do I efficiently reduce the size of my data model?

Hi there,I am looking for an efficient way to reduce the size of my data model: I already reduced the time period of the ingested data. As a second step I want to get rid of some unused table columns. To identify unused table columns I thought about doing the following:Export a .txt documentation of each analysis, then search for a specific table (ctrl+f) (i.e. LIPS) and document which columns are really in use in this analysis. I do this with all analyses and then I know which columns I can exclude in the extractions because they are not used anywhere.  It's not as much manual work as it sounds but still a lot. Do you know a better way to do this? How do you avoid such work in the future? Is there a standard procedure how to handle this problem or not run into it in the first place?  Thanks for your helpDaniel  

30
janpeter.van.d
Level 8
A
Level 3
amina.mrad12Level 3
asked in Data Ingestion & Infrastructure
Event Processing Rules

Hello community, we are trying to create a task mining use case but we struggle with the event processing rule  Could you please support with the following: Process start should be only be triggered when the URL = a specific URL Process end should be always when an email is sent When another URL than a specific URL is used then task mining should not capture that informationSee screenshots attached of how the event processing rules are being configured.  

10
L
Level 1
satoshi.omach12
Level 3
satoshi.omach12Level 3
asked in Data Ingestion & Infrastructure
Could you teach me will a two-way connection occur between the On-premise Extractor and the EMS Data Integration?

Could you teach me will a two-way connection occur between the On-premise Extractor and the EMS Data Integration?

00
L
Level 1
T
Level 3
takayoshi.ichikLevel 3
asked in Data Ingestion & Infrastructure
Activity count and Connection count in one OLAP table

Dear Community, I'd like to make a OLAP table that have Activity count and Connection count.  Is there any way to make this OLAP table? Thank you. Best regards,Takayoshi Ichikawa 

10
T
Level 3
mallik.koruk11
Level 3
mallik.koruk11Level 3
asked in Data Ingestion & Infrastructure
Is it possible to add cascade in a drop down button

On click on a drop down button it should show more drop downs like cascading. Is it possible to add like that?

10
mallik.koruk11
Level 3
.
Level 2
.14600791Level 2
asked in Data Ingestion & Infrastructure
Is it possible to show data in a table in columns vs. rows?

We are trying to create a report which shows contract information with different validities. I can see the information in rows but ideally need them shown in columns.Layout in rows means I have 8 rows for 8 different validities but in columns this would reduce the number of rows massively

10
G
Level 8
R
Level 4
raman.silot14Level 4
asked in Data Ingestion & Infrastructure
why we use reload from cache?

I am confused why we use Reload from cache, does it provide some kind of undo functionality? Also I am confusing it with Partial reload, are they same?

00
R
Level 4
A
Level 1
alejandra.galla12Level 1
asked in Data Ingestion & Infrastructure
Flat file with variable columns - Hi, can Celonis PQL process a text file with variable columns (see below) in order to achieve the table outcome attached? Any guidance on a general approach would be appreciated, or even just a Yes/No answer. Thanks!

1) Raw file (scrubbed):Role=MEMBER; LOC=IL;CLMC=C;Role=MEMBER; LOC=IL;CLMC=C;Role=MEMBER; LOC=IL;CLMC=C;LOC=IL; CLMC=C;LOC=IL; CLMC=C;LOB=LOB1; LOC=IL;CLMC=C;CLR=Y;LGLSST=ASGN;LOB=LOB1; LOC=IL;CLMC=C;CLR=Y;LGLSST=ASGN;LOB=LOB1; LOC=IL;CLMC=C;CLR=Y;LGLSST=ASGN;LOB=LOB1; LOC=IL;CLMC=C;CLR=Y;LGLSST=ASGN;ID=1234,TYPE=PROP;ID=1234,TYPE=PROP;LOC=IL;CLMC=C;LOC=IL;CLMC=C; 2) Desired outcome:

10
B
Level 1
A
Level 3
arunjit.das12Level 3
asked in Data Ingestion & Infrastructure
"Simple Ratio" query replication error

Automation Rate, Simple Ratio formula is as follows: KPI("Ratio", "_CEL_AP_ACTIVITIES"."USER_TYPE" = 'B') -- result: 64.07% We are trying to replicate the same simple ratio with a standalone query instead of visual editor to understand the mechanism of the Simple Ratio. We have tried two formulas and both of them gave us different answers. Using PU_COUNTSUM(PU_COUNT("BSEG" , "_CEL_AP_ACTIVITIES"."USER_TYPE" , "_CEL_AP_ACTIVITIES"."USER_TYPE" = 'B')) / SUM(PU_COUNT("BSEG" , "_CEL_AP_ACTIVITIES"."USER_TYPE")) -- result: 65.45%  Using CASE WHEN statementAVG(CASE WHEN "_CEL_AP_ACTIVITIES"."USER_TYPE" = 'B' THEN 1.0 ELSE 0.0 END) -- result: 65.51% As it can be seen from above, that each query gave us different results and we were not able to replicate the Simple ratio as well. What is the actual backend formula used for Simple Ratio? A

00
A
Level 3

Badge Winners

VIEW ALL
Community Code of Conduct