Skip to main content

Hello everyone,

 

for my current Usecase i need to upload a parquet-file (that is stored locally), to the data pool via File Uploads. When the file is uploaded i need to execute the referred datajob. The last step would be to press "Force Complete Reload" inside the Process Data Models Menu. These are all manuell steps.

 

Is there a way to automate the whole process?

 

best regards

Andreas

Hi @1460031749 ,

 

in scheduling you can schedule it at a certain time of the day, within it you can also execute it whenever you like on the top right. I don't think you can trigger it automatically whenever you're uploading the file.


You can schedule the data job execution and the data model reload, AFAIK.

 

But I couldn't find a way to automate the file upload.

 

We are having the same situation and we are using pycelonis, in particular datapool.create_table()

 

https://celonis.github.io/pycelonis/1.7.1/reference/celonis_api/event_collection/data_pool/#celonis_api.event_collection.data_pool.Pool.create_table

 

HTH


You can schedule the data job execution and the data model reload, AFAIK.

 

But I couldn't find a way to automate the file upload.

 

We are having the same situation and we are using pycelonis, in particular datapool.create_table()

 

https://celonis.github.io/pycelonis/1.7.1/reference/celonis_api/event_collection/data_pool/#celonis_api.event_collection.data_pool.Pool.create_table

 

HTH

Hi @Guillermo Gost, In your case you can schedule the ML Workbench.


Hi @1460031749,

 

As suggested above I would recommend PyCelonis to perform these actions

  1. Upload tables to Pool
    1. Use Pycelonis data_pool.create_table() or data_pool.append_table() if it already exists. This functions support parquet files as input.
  2. Execute Data Job
    1. Use Pycelonis Transformation.execute()
  3. Reload datamodel
    1. Use Pycelonis data_model.reload()

 

Since your file is located locally, you can or upload it to the Celonis Workbench environment by dragging it in the file explorer section of a Jupyter notebook, or you can create a Python environment locally and run the script from there.

 

Hopefully this helps!

 

Kind regards,

Jan-peter


Reply