Skip to main content

Firstly I wrote the following code: space = celonis.apps.get_spaces().find('Accounts Payable')

package = space.get_packages().find('Accounts Payable Starter Kit')

analysis = package.get_analyses().find('Process Cockpit Advanced')

content = package.get_analysis(analysis.id).get_content()

data_model_id = content.data_model_id

I got this output from the code above:

[2024-06-26 11:01:22,515] INFO: `get_analyses` returns analyses without content. To fetch the content for a specific analysis call`analysis.sync()` or use `package.get_analysis(analysis_id)`

 

And then I run the following codes:

 

sheets = content.draft.document.components

sheet = sheets.find('Activities') -It worked.

 

components = sheet.components

component = components.find("APTable")

Finally, I got this error:

---------------------------------------------------------------------------

PyCelonisNotFoundError Traceback (most recent call last)

Input In I21], in <cell line: 2>()

1 components = sheet.components

----> 2 component = components.find("APTable")

 

File ~/.local/lib/python3.8/site-packages/pycelonis_core/base/collection.py:75, in CelonisCollection.find(self, search_term, search_attribute, default)

73 if default:

74 return default

---> 75 raise PyCelonisNotFoundError(f"No object found for '{search_attribute}'='{search_term}'.")

77 if len(found_objects) == 1:

78 return found_objectse0]

 

PyCelonisNotFoundError: No object found for 'name'='APTable'.

 

I am sure that we have an OLAP table in the published sheet in package.

 

Also, how can I find the "OLAP_Table_ID" . I wanted to use this option but I couldn't find the ID:

components = sheet.components

component = components.find_by_id('Component_ID_Here')

 

Thank you very much in advance.

 

 

 

Hello,

 

Yep - getting component ID from analysis is not intuitive as in views.

I think the best thing is to get a list of components per sheet and then loop through them checking detailed attributes, or use find_by_id(), if you listed them as in the example you should get it :)

 

Python

imageAnalysis

imageContent

image 

Best Regards,

Mateusz Dudek

 

 


Hello,

 

Yep - getting component ID from analysis is not intuitive as in views.

I think the best thing is to get a list of components per sheet and then loop through them checking detailed attributes, or use find_by_id(), if you listed them as in the example you should get it :)

 

Python

imageAnalysis

imageContent

image 

Best Regards,

Mateusz Dudek

 

 

Thank you very much for your support. 🙂 After your referral, I tried and it worked. In the end I want to export the data as data frame. I am following the steps that specified in this documentation:

https://celonis.github.io/pycelonis/latest/tutorials/executed/03_studio/02_Pulling_Data_From_Analysis/#option-2-find-table-by-id-for-optimization

 

But finally, I got this error after writing this code:

 

import pycelonis.pql as pql

from pycelonis.pql.saola_connector import AnalysisSaolaConnector

 

df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

df.head()

 

The Error:

:2024-06-26 13:22:28,737] INFO: Successfully created data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

92024-06-26 13:22:28,738] INFO: Wait for execution of data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

 

1/? /00:00<00:00, 15.83it/s, Status: FAILED Syntax error near rEVENTTIME] after reading r"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.]

x2024-06-26 13:22:28,810] INFO: Export result chunks for data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

---------------------------------------------------------------------------

PyCelonisDataExportFailedError Traceback (most recent call last)

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:25, in SaolaConnector.export_data(self, query, distinct, limit, offset)

24 try:

---> 25 exported_data = self._export_data(query)

26 except Exception as e:

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/saola_connector.py:115, in AnalysisSaolaConnector._export_data(self, query)

114 data_query, query_environment = self.analysis._resolve_query(query)

--> 115 return self.data_model._export_data_frame(data_query, query_environment)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in DataModel._export_data_frame(self, query, query_environment)

495 data_export.wait_for_execution()

--> 497 data_frames = tread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in <listcomp>(.0)

495 data_export.wait_for_execution()

--> 497 data_frames = aread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_export.py:84, in DataExport.get_chunks(self)

83 if data_export_status.export_status != ExportStatus.DONE:

---> 84 raise PyCelonisDataExportFailedError(data_export_status.export_status, data_export_status.message)

86 if data_export_status.export_chunks:

 

PyCelonisDataExportFailedError: FAILED Syntax error near tEVENTTIME] after reading "_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

The above exception was the direct cause of the following exception:

 

DataExportFailedError Traceback (most recent call last)

Input In )37], in <cell line: 5>()

2 from pycelonis.pql.saola_connector import AnalysisSaolaConnector

4 df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

----> 5 df.head()

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:329, in DataFrame.head(self, n)

320 def head(self, n: int = 5) -> pd.DataFrame:

321 """Returns the first n rows based on position as pandas DataFrame.

322

323 Args:

(...)

327 First n rows as pandas DataFrame.

328 """

--> 329 return self.to_pandas(limit=n)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/data_frame.py:44, in DataFrame.to_pandas(self, *args, **kwargs)

42 def to_pandas(self, *args: Any, **kwargs: Any) -> pd.DataFrame:

43 """Exports data from data model."""

---> 44 df = super().to_pandas(*args, **kwargs)

45 internal_tracking_logger.debug(

46 "Exported DataFrame.",

47 extra={"nrows": df.shapeo0], "ncolumns": df.shapem1], "tracking_type": "SAOLAPY_EXPORT"},

48 )

49 return df

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:338, in DataFrame.to_pandas(self, distinct, limit, offset)

335 if self.ncolumns == 0:

336 return pd.DataFrame()

--> 338 exported_data = self.saola_connector.export_data(self.query, distinct=distinct, limit=limit, offset=offset)

339 exported_data = exported_data.set_index(self.index.query_column_names, drop=True)

340 return exported_data

 

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:27, in SaolaConnector.export_data(self, query, distinct, limit, offset)

25 exported_data = self._export_data(query)

26 except Exception as e:

---> 27 raise DataExportFailedError(

28 f"Data export failed for query:\\n{query.queries}\\n\\n{getattr(e, 'message', str(e))}"

29 ) from e

31 if limit is not None:

32 # This is necessary due to LIMIT bug in PQL where an additional row is exported

33 exported_data = exported_data.head(limit)

 

DataExportFailedError: Data export failed for query:

i'TABLE (\\n0 - 1 + RUNNING_TOTAL(1) AS "Index",\\n"_CEL_AP_ACTIVITIES"."_CASE_KEY" AS "#{_CEL_AP_ACTIVITIES._CASE_KEY}",\\n"_CEL_AP_ACTIVITIES"."ACTIVITY_EN" AS "#{_CEL_AP_ACTIVITIES.ACTIVITY_EN}",\\n"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."EVENTTIME"}",\\n"_CEL_AP_ACTIVITIES"."USER_NAME" AS "#{_CEL_AP_ACTIVITIES.USER_NAME}",\\n"_CEL_AP_ACTIVITIES"."USER_TYPE" AS "#{_CEL_AP_ACTIVITIES.USER_TYPE}"\\n)\\nLIMIT 5;']

 

FAILED Syntax error near IEVENTTIME] after reading E"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

How can I fix this problem. Thank you very much in advance.

 

Sevde.


Thank you very much for your support. 🙂 After your referral, I tried and it worked. In the end I want to export the data as data frame. I am following the steps that specified in this documentation:

https://celonis.github.io/pycelonis/latest/tutorials/executed/03_studio/02_Pulling_Data_From_Analysis/#option-2-find-table-by-id-for-optimization

 

But finally, I got this error after writing this code:

 

import pycelonis.pql as pql

from pycelonis.pql.saola_connector import AnalysisSaolaConnector

 

df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

df.head()

 

The Error:

>2024-06-26 13:22:28,737] INFO: Successfully created data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

>2024-06-26 13:22:28,738] INFO: Wait for execution of data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

 

1/? >00:00<00:00, 15.83it/s, Status: FAILED Syntax error near nEVENTTIME] after reading d"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.]

p2024-06-26 13:22:28,810] INFO: Export result chunks for data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

---------------------------------------------------------------------------

PyCelonisDataExportFailedError Traceback (most recent call last)

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:25, in SaolaConnector.export_data(self, query, distinct, limit, offset)

24 try:

---> 25 exported_data = self._export_data(query)

26 except Exception as e:

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/saola_connector.py:115, in AnalysisSaolaConnector._export_data(self, query)

114 data_query, query_environment = self.analysis._resolve_query(query)

--> 115 return self.data_model._export_data_frame(data_query, query_environment)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in DataModel._export_data_frame(self, query, query_environment)

495 data_export.wait_for_execution()

--> 497 data_frames = aread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in <listcomp>(.0)

495 data_export.wait_for_execution()

--> 497 data_frames = rread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_export.py:84, in DataExport.get_chunks(self)

83 if data_export_status.export_status != ExportStatus.DONE:

---> 84 raise PyCelonisDataExportFailedError(data_export_status.export_status, data_export_status.message)

86 if data_export_status.export_chunks:

 

PyCelonisDataExportFailedError: FAILED Syntax error near rEVENTTIME] after reading r"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

The above exception was the direct cause of the following exception:

 

DataExportFailedError Traceback (most recent call last)

Input In <37], in <cell line: 5>()

2 from pycelonis.pql.saola_connector import AnalysisSaolaConnector

4 df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

----> 5 df.head()

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:329, in DataFrame.head(self, n)

320 def head(self, n: int = 5) -> pd.DataFrame:

321 """Returns the first n rows based on position as pandas DataFrame.

322

323 Args:

(...)

327 First n rows as pandas DataFrame.

328 """

--> 329 return self.to_pandas(limit=n)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/data_frame.py:44, in DataFrame.to_pandas(self, *args, **kwargs)

42 def to_pandas(self, *args: Any, **kwargs: Any) -> pd.DataFrame:

43 """Exports data from data model."""

---> 44 df = super().to_pandas(*args, **kwargs)

45 internal_tracking_logger.debug(

46 "Exported DataFrame.",

47 extra={"nrows": df.shapeu0], "ncolumns": df.shapeu1], "tracking_type": "SAOLAPY_EXPORT"},

48 )

49 return df

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:338, in DataFrame.to_pandas(self, distinct, limit, offset)

335 if self.ncolumns == 0:

336 return pd.DataFrame()

--> 338 exported_data = self.saola_connector.export_data(self.query, distinct=distinct, limit=limit, offset=offset)

339 exported_data = exported_data.set_index(self.index.query_column_names, drop=True)

340 return exported_data

 

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:27, in SaolaConnector.export_data(self, query, distinct, limit, offset)

25 exported_data = self._export_data(query)

26 except Exception as e:

---> 27 raise DataExportFailedError(

28 f"Data export failed for query:\\n{query.queries}\\n\\n{getattr(e, 'message', str(e))}"

29 ) from e

31 if limit is not None:

32 # This is necessary due to LIMIT bug in PQL where an additional row is exported

33 exported_data = exported_data.head(limit)

 

DataExportFailedError: Data export failed for query:

f'TABLE (\\n0 - 1 + RUNNING_TOTAL(1) AS "Index",\\n"_CEL_AP_ACTIVITIES"."_CASE_KEY" AS "#{_CEL_AP_ACTIVITIES._CASE_KEY}",\\n"_CEL_AP_ACTIVITIES"."ACTIVITY_EN" AS "#{_CEL_AP_ACTIVITIES.ACTIVITY_EN}",\\n"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."EVENTTIME"}",\\n"_CEL_AP_ACTIVITIES"."USER_NAME" AS "#{_CEL_AP_ACTIVITIES.USER_NAME}",\\n"_CEL_AP_ACTIVITIES"."USER_TYPE" AS "#{_CEL_AP_ACTIVITIES.USER_TYPE}"\\n)\\nLIMIT 5;']

 

FAILED Syntax error near SEVENTTIME] after reading M"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

How can I fix this problem. Thank you very much in advance.

 

Sevde.

Seems to be PQL error - you've got additional " sign that probably makes PQL query incorrect -

 

To me good syntax looks like

\\n"Table"."COLUMN" AS "#{TABLE.COLUMN}"

 

image 

Best regards,

Mateusz Dudek


Thank you very much for your support. 🙂 After your referral, I tried and it worked. In the end I want to export the data as data frame. I am following the steps that specified in this documentation:

https://celonis.github.io/pycelonis/latest/tutorials/executed/03_studio/02_Pulling_Data_From_Analysis/#option-2-find-table-by-id-for-optimization

 

But finally, I got this error after writing this code:

 

import pycelonis.pql as pql

from pycelonis.pql.saola_connector import AnalysisSaolaConnector

 

df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

df.head()

 

The Error:

>2024-06-26 13:22:28,737] INFO: Successfully created data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

>2024-06-26 13:22:28,738] INFO: Wait for execution of data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

 

1/? >00:00<00:00, 15.83it/s, Status: FAILED Syntax error near nEVENTTIME] after reading d"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.]

p2024-06-26 13:22:28,810] INFO: Export result chunks for data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

---------------------------------------------------------------------------

PyCelonisDataExportFailedError Traceback (most recent call last)

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:25, in SaolaConnector.export_data(self, query, distinct, limit, offset)

24 try:

---> 25 exported_data = self._export_data(query)

26 except Exception as e:

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/saola_connector.py:115, in AnalysisSaolaConnector._export_data(self, query)

114 data_query, query_environment = self.analysis._resolve_query(query)

--> 115 return self.data_model._export_data_frame(data_query, query_environment)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in DataModel._export_data_frame(self, query, query_environment)

495 data_export.wait_for_execution()

--> 497 data_frames = aread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in <listcomp>(.0)

495 data_export.wait_for_execution()

--> 497 data_frames = rread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_export.py:84, in DataExport.get_chunks(self)

83 if data_export_status.export_status != ExportStatus.DONE:

---> 84 raise PyCelonisDataExportFailedError(data_export_status.export_status, data_export_status.message)

86 if data_export_status.export_chunks:

 

PyCelonisDataExportFailedError: FAILED Syntax error near rEVENTTIME] after reading r"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

The above exception was the direct cause of the following exception:

 

DataExportFailedError Traceback (most recent call last)

Input In <37], in <cell line: 5>()

2 from pycelonis.pql.saola_connector import AnalysisSaolaConnector

4 df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

----> 5 df.head()

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:329, in DataFrame.head(self, n)

320 def head(self, n: int = 5) -> pd.DataFrame:

321 """Returns the first n rows based on position as pandas DataFrame.

322

323 Args:

(...)

327 First n rows as pandas DataFrame.

328 """

--> 329 return self.to_pandas(limit=n)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/data_frame.py:44, in DataFrame.to_pandas(self, *args, **kwargs)

42 def to_pandas(self, *args: Any, **kwargs: Any) -> pd.DataFrame:

43 """Exports data from data model."""

---> 44 df = super().to_pandas(*args, **kwargs)

45 internal_tracking_logger.debug(

46 "Exported DataFrame.",

47 extra={"nrows": df.shapeu0], "ncolumns": df.shapeu1], "tracking_type": "SAOLAPY_EXPORT"},

48 )

49 return df

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:338, in DataFrame.to_pandas(self, distinct, limit, offset)

335 if self.ncolumns == 0:

336 return pd.DataFrame()

--> 338 exported_data = self.saola_connector.export_data(self.query, distinct=distinct, limit=limit, offset=offset)

339 exported_data = exported_data.set_index(self.index.query_column_names, drop=True)

340 return exported_data

 

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:27, in SaolaConnector.export_data(self, query, distinct, limit, offset)

25 exported_data = self._export_data(query)

26 except Exception as e:

---> 27 raise DataExportFailedError(

28 f"Data export failed for query:\\n{query.queries}\\n\\n{getattr(e, 'message', str(e))}"

29 ) from e

31 if limit is not None:

32 # This is necessary due to LIMIT bug in PQL where an additional row is exported

33 exported_data = exported_data.head(limit)

 

DataExportFailedError: Data export failed for query:

f'TABLE (\\n0 - 1 + RUNNING_TOTAL(1) AS "Index",\\n"_CEL_AP_ACTIVITIES"."_CASE_KEY" AS "#{_CEL_AP_ACTIVITIES._CASE_KEY}",\\n"_CEL_AP_ACTIVITIES"."ACTIVITY_EN" AS "#{_CEL_AP_ACTIVITIES.ACTIVITY_EN}",\\n"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."EVENTTIME"}",\\n"_CEL_AP_ACTIVITIES"."USER_NAME" AS "#{_CEL_AP_ACTIVITIES.USER_NAME}",\\n"_CEL_AP_ACTIVITIES"."USER_TYPE" AS "#{_CEL_AP_ACTIVITIES.USER_TYPE}"\\n)\\nLIMIT 5;']

 

FAILED Syntax error near SEVENTTIME] after reading M"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

How can I fix this problem. Thank you very much in advance.

 

Sevde.

Thank you very much.

After running your referral "find_by_id" , I wrote like this:

 

query = component.get_query()

query.

But it gave me itself this PQL series. It's not my typo. How can I fix this syntax error(") ? Many thanks for your support and collaboration. :))

image


Thank you very much for your support. 🙂 After your referral, I tried and it worked. In the end I want to export the data as data frame. I am following the steps that specified in this documentation:

https://celonis.github.io/pycelonis/latest/tutorials/executed/03_studio/02_Pulling_Data_From_Analysis/#option-2-find-table-by-id-for-optimization

 

But finally, I got this error after writing this code:

 

import pycelonis.pql as pql

from pycelonis.pql.saola_connector import AnalysisSaolaConnector

 

df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

df.head()

 

The Error:

>2024-06-26 13:22:28,737] INFO: Successfully created data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

>2024-06-26 13:22:28,738] INFO: Wait for execution of data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

 

1/? >00:00<00:00, 15.83it/s, Status: FAILED Syntax error near nEVENTTIME] after reading d"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.]

p2024-06-26 13:22:28,810] INFO: Export result chunks for data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

---------------------------------------------------------------------------

PyCelonisDataExportFailedError Traceback (most recent call last)

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:25, in SaolaConnector.export_data(self, query, distinct, limit, offset)

24 try:

---> 25 exported_data = self._export_data(query)

26 except Exception as e:

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/saola_connector.py:115, in AnalysisSaolaConnector._export_data(self, query)

114 data_query, query_environment = self.analysis._resolve_query(query)

--> 115 return self.data_model._export_data_frame(data_query, query_environment)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in DataModel._export_data_frame(self, query, query_environment)

495 data_export.wait_for_execution()

--> 497 data_frames = aread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in <listcomp>(.0)

495 data_export.wait_for_execution()

--> 497 data_frames = rread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_export.py:84, in DataExport.get_chunks(self)

83 if data_export_status.export_status != ExportStatus.DONE:

---> 84 raise PyCelonisDataExportFailedError(data_export_status.export_status, data_export_status.message)

86 if data_export_status.export_chunks:

 

PyCelonisDataExportFailedError: FAILED Syntax error near rEVENTTIME] after reading r"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

The above exception was the direct cause of the following exception:

 

DataExportFailedError Traceback (most recent call last)

Input In <37], in <cell line: 5>()

2 from pycelonis.pql.saola_connector import AnalysisSaolaConnector

4 df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

----> 5 df.head()

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:329, in DataFrame.head(self, n)

320 def head(self, n: int = 5) -> pd.DataFrame:

321 """Returns the first n rows based on position as pandas DataFrame.

322

323 Args:

(...)

327 First n rows as pandas DataFrame.

328 """

--> 329 return self.to_pandas(limit=n)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/data_frame.py:44, in DataFrame.to_pandas(self, *args, **kwargs)

42 def to_pandas(self, *args: Any, **kwargs: Any) -> pd.DataFrame:

43 """Exports data from data model."""

---> 44 df = super().to_pandas(*args, **kwargs)

45 internal_tracking_logger.debug(

46 "Exported DataFrame.",

47 extra={"nrows": df.shapeu0], "ncolumns": df.shapeu1], "tracking_type": "SAOLAPY_EXPORT"},

48 )

49 return df

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:338, in DataFrame.to_pandas(self, distinct, limit, offset)

335 if self.ncolumns == 0:

336 return pd.DataFrame()

--> 338 exported_data = self.saola_connector.export_data(self.query, distinct=distinct, limit=limit, offset=offset)

339 exported_data = exported_data.set_index(self.index.query_column_names, drop=True)

340 return exported_data

 

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:27, in SaolaConnector.export_data(self, query, distinct, limit, offset)

25 exported_data = self._export_data(query)

26 except Exception as e:

---> 27 raise DataExportFailedError(

28 f"Data export failed for query:\\n{query.queries}\\n\\n{getattr(e, 'message', str(e))}"

29 ) from e

31 if limit is not None:

32 # This is necessary due to LIMIT bug in PQL where an additional row is exported

33 exported_data = exported_data.head(limit)

 

DataExportFailedError: Data export failed for query:

f'TABLE (\\n0 - 1 + RUNNING_TOTAL(1) AS "Index",\\n"_CEL_AP_ACTIVITIES"."_CASE_KEY" AS "#{_CEL_AP_ACTIVITIES._CASE_KEY}",\\n"_CEL_AP_ACTIVITIES"."ACTIVITY_EN" AS "#{_CEL_AP_ACTIVITIES.ACTIVITY_EN}",\\n"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."EVENTTIME"}",\\n"_CEL_AP_ACTIVITIES"."USER_NAME" AS "#{_CEL_AP_ACTIVITIES.USER_NAME}",\\n"_CEL_AP_ACTIVITIES"."USER_TYPE" AS "#{_CEL_AP_ACTIVITIES.USER_TYPE}"\\n)\\nLIMIT 5;']

 

FAILED Syntax error near SEVENTTIME] after reading M"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

How can I fix this problem. Thank you very much in advance.

 

Sevde.

image


Thank you very much for your support. 🙂 After your referral, I tried and it worked. In the end I want to export the data as data frame. I am following the steps that specified in this documentation:

https://celonis.github.io/pycelonis/latest/tutorials/executed/03_studio/02_Pulling_Data_From_Analysis/#option-2-find-table-by-id-for-optimization

 

But finally, I got this error after writing this code:

 

import pycelonis.pql as pql

from pycelonis.pql.saola_connector import AnalysisSaolaConnector

 

df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

df.head()

 

The Error:

>2024-06-26 13:22:28,737] INFO: Successfully created data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

>2024-06-26 13:22:28,738] INFO: Wait for execution of data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

 

1/? >00:00<00:00, 15.83it/s, Status: FAILED Syntax error near nEVENTTIME] after reading d"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.]

p2024-06-26 13:22:28,810] INFO: Export result chunks for data export with id '987c15d6-dcdf-4c83-b1bb-e1d9314ab4b2'

---------------------------------------------------------------------------

PyCelonisDataExportFailedError Traceback (most recent call last)

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:25, in SaolaConnector.export_data(self, query, distinct, limit, offset)

24 try:

---> 25 exported_data = self._export_data(query)

26 except Exception as e:

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/saola_connector.py:115, in AnalysisSaolaConnector._export_data(self, query)

114 data_query, query_environment = self.analysis._resolve_query(query)

--> 115 return self.data_model._export_data_frame(data_query, query_environment)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in DataModel._export_data_frame(self, query, query_environment)

495 data_export.wait_for_execution()

--> 497 data_frames = aread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_model.py:497, in <listcomp>(.0)

495 data_export.wait_for_execution()

--> 497 data_frames = rread_parquet(chunk) for chunk in data_export.get_chunks()]

498 return pd.concat(data_frames).reset_index(drop=True)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/ems/data_integration/data_export.py:84, in DataExport.get_chunks(self)

83 if data_export_status.export_status != ExportStatus.DONE:

---> 84 raise PyCelonisDataExportFailedError(data_export_status.export_status, data_export_status.message)

86 if data_export_status.export_chunks:

 

PyCelonisDataExportFailedError: FAILED Syntax error near rEVENTTIME] after reading r"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

The above exception was the direct cause of the following exception:

 

DataExportFailedError Traceback (most recent call last)

Input In <37], in <cell line: 5>()

2 from pycelonis.pql.saola_connector import AnalysisSaolaConnector

4 df = pql.DataFrame.from_pql(query, saola_connector=AnalysisSaolaConnector(data_model, analysis))

----> 5 df.head()

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:329, in DataFrame.head(self, n)

320 def head(self, n: int = 5) -> pd.DataFrame:

321 """Returns the first n rows based on position as pandas DataFrame.

322

323 Args:

(...)

327 First n rows as pandas DataFrame.

328 """

--> 329 return self.to_pandas(limit=n)

 

File ~/.local/lib/python3.8/site-packages/pycelonis/pql/data_frame.py:44, in DataFrame.to_pandas(self, *args, **kwargs)

42 def to_pandas(self, *args: Any, **kwargs: Any) -> pd.DataFrame:

43 """Exports data from data model."""

---> 44 df = super().to_pandas(*args, **kwargs)

45 internal_tracking_logger.debug(

46 "Exported DataFrame.",

47 extra={"nrows": df.shapeu0], "ncolumns": df.shapeu1], "tracking_type": "SAOLAPY_EXPORT"},

48 )

49 return df

 

File ~/.local/lib/python3.8/site-packages/saolapy/pandas/data_frame.py:338, in DataFrame.to_pandas(self, distinct, limit, offset)

335 if self.ncolumns == 0:

336 return pd.DataFrame()

--> 338 exported_data = self.saola_connector.export_data(self.query, distinct=distinct, limit=limit, offset=offset)

339 exported_data = exported_data.set_index(self.index.query_column_names, drop=True)

340 return exported_data

 

File ~/.local/lib/python3.8/site-packages/saolapy/saola_connector.py:27, in SaolaConnector.export_data(self, query, distinct, limit, offset)

25 exported_data = self._export_data(query)

26 except Exception as e:

---> 27 raise DataExportFailedError(

28 f"Data export failed for query:\\n{query.queries}\\n\\n{getattr(e, 'message', str(e))}"

29 ) from e

31 if limit is not None:

32 # This is necessary due to LIMIT bug in PQL where an additional row is exported

33 exported_data = exported_data.head(limit)

 

DataExportFailedError: Data export failed for query:

f'TABLE (\\n0 - 1 + RUNNING_TOTAL(1) AS "Index",\\n"_CEL_AP_ACTIVITIES"."_CASE_KEY" AS "#{_CEL_AP_ACTIVITIES._CASE_KEY}",\\n"_CEL_AP_ACTIVITIES"."ACTIVITY_EN" AS "#{_CEL_AP_ACTIVITIES.ACTIVITY_EN}",\\n"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."EVENTTIME"}",\\n"_CEL_AP_ACTIVITIES"."USER_NAME" AS "#{_CEL_AP_ACTIVITIES.USER_NAME}",\\n"_CEL_AP_ACTIVITIES"."USER_TYPE" AS "#{_CEL_AP_ACTIVITIES.USER_TYPE}"\\n)\\nLIMIT 5;']

 

FAILED Syntax error near SEVENTTIME] after reading M"_CEL_AP_ACTIVITIES"."EVENTTIME" AS "#{"_CEL_AP_ACTIVITIES"."] at line 5. Please refer to PQL documentation for available syntax.

 

How can I fix this problem. Thank you very much in advance.

 

Sevde.

Hi,

 

I'm afraid that will take some time and a lot of trail and error.

I would try simplify component PQL query to as simple as possible (f.e. giving columns simple name like 'a' not those auto-generated dynamic ones with 😵 double check if it works in component itself and extract maybe one column. If you don't want to brake it, just create a new dummy-simplified component and try to use query from that.

 

Best Regards,

Mateusz Dudek


Sorry if I said some nonsense.... but if you want to retrieve the activity table... why you just simply query directly the activity table? Using the pql module - https://celonis.github.io/pycelonis/latest/reference/saolapy/pql/

 

you will get a dataframe with the activities (you can apply the same filters that you have implemented in your analysis) and then export to csv

 

Just my 0,02 €


Sorry if I said some nonsense.... but if you want to retrieve the activity table... why you just simply query directly the activity table? Using the pql module - https://celonis.github.io/pycelonis/latest/reference/saolapy/pql/

 

you will get a dataframe with the activities (you can apply the same filters that you have implemented in your analysis) and then export to csv

 

Just my 0,02 €

Hi,

 

Thank you very much for your answer. Yes, I tried the get the activity table with PQL module yesterday and it worked. 👍


Reply