Skip to main content
Question

Version control best practice

  • July 8, 2019
  • 13 replies
  • 23 views
A
  • Anonymous
Dear all, I started new DCP project from this month. I installed Process connector and Analyses from App store then modified some of Vertica SQL in transformation, and PQL in the analysis. After some changes, I found that I am not confident which code I modified. Referring to SAP ABAP or Java or some other languages, we normally can use version control tools, like Git, Subversion, that can manage who and when code is committed, which part of code was changed (before/after). In Celonis Project how to manage source code version ? Do you have your best practices ? Best regards, Kazuhiko

13 replies

B
Level 0
Dear Kazuhiko, There is currently no source code management in the IBC which offers rolling back the most recent changes in an PQL / Vertica SQL besides the CTRL + Z when working directly in the code. Best practice would be to have a stable and development environment and only rolling out (overwriting) the stable environment once the development code / analysis is tested for functionality. This way all the changes can be tracked by comparing the stable to the development code. For Vertica SQL code this is easily achieved by working with templates to copy transformations / extractions from one place to another. In the analysis environment just have two workspaces (Production & Development) and move the analyses between them by right clicking them --> move to. Best Regards, Benedict
A
  • Anonymous
  • July 9, 2019
Dear Benedict, Thanks for your advice. As long term solution, I would like to add source code management to your backlog. AS short term solution, I understand your opinion how to manage Analyses. On the other hand, regarding Transformation and Extraction in Event collection, I am not clear how to handle versions by template you mentioned. In my understanding, template is created to share same version with another connection (SAP instance etc.). Exporting sql file from Transformation is one of the workaround to manage version, but it takes a lot of effort and I cannot force my colleagues to do it.
B
Level 0
Hey Kazuhiko, Best would be to ask for it by writing and submitting an official feature request! By handling versions with templates I mean that you develop your transformations / extractions in a normal data jobs, lets say MyDatajob_v1. When you are ready to publish it, you make it a template, publish it. Then you start working on a new template e.g. MyDatajob_v2_BETA and again make it a template when its ready to be published. This way you can develop a template while having a stable version in another and just overwrite them manually. Best Regards, Benedict
A
  • Anonymous
  • July 16, 2019
Dear Benedict, I understand that data job (including extraction/transformation) is duplicated and rename it as new version. Thanks.
A
  • Anonymous
  • May 20, 2020
Just FYI, I developed version control tools by myself. The point (below picture) is to update change points (diffs) only and not replacing the all of objects discussed above. To do this, I can keep local change in target systems and just push update of original system. If you are interested in it, please contact to me. image1600900 128 KB
M
Level 5
  • masha.galit
  • Level 5
  • June 8, 2020
Hello @kaztakata may I ask you if you have submitted a feature request? We recently started working with Celonis and are also really puzzled by this important feature missing. Regards Masha
A
  • Anonymous
  • June 8, 2020
Dear @Masha, I did not submit feature request to Celonis development team, but explained general features I developed to Celonis Data science team. I developed it as Celonis partner (consulting firm) because I would like to distribute my business logics (template) to my clients, and extend logics without breaking client specific logics.
janpeter.van.d
Level 12
Forum|alt.badge.img+26
Hi Kaztakata, Nice visualization of this solution! We also use the ML-Workbench at this moment for the same goal. It is quite time-consuming to set up, but really helps us to automate some administrative work. Bests, Jan-peter
ravi.kumar23
Level 7
  • ravi.kumar23
  • Level 7
  • January 17, 2023

Hi Team,

Are there any updates from Celonis on this .... is the backlog for creating this implemented. I also dont see the picture attached above.

janpeter.van.d
Level 12
Forum|alt.badge.img+26

Hi @ravi.kumar23,

 

I asked this in a session on Celopeers, and they are still suggesting to extract Celonis Objects via the back-end in Python (with PyCelonis) and push it to GIT repositories. Here you can push changes from a development/acceptance Pool/Package and accept/merge it into a production version of the Pool/Package, which nobody has edit rights on in Celonis. Unfortunately they didn't had plans to integrate this in the front-end of Celonis as was told.

 

Last but not least, the reason that you cannot see the pictures is because this topic was migrated from the deprecated Celonis Community and same information unfortunately didn't make it into Celopeers.

 

Kind regards,

Jan-peter

ravi.kumar23
Level 7
  • ravi.kumar23
  • Level 7
  • January 17, 2023

Thanks @janpeter.van.d for your quick response!

 

Do you have more details on how this can be achieved by using PyCelonis?

 

We are trying to achieve CI/CD kind of stuff and this will be good start for us.

 

 

 

 

janpeter.van.d
Level 12
Forum|alt.badge.img+26

Hi Ravi,

 

Using the functions in the 'ems' section of the PyCelonis API (link: Index - PyCelonis), you can obtain all objects you want:

  • All Data Jobs (Extractions/Transformations) per data job
  • All Data Models (Tables, Keys)
  • All packages and their content (Analyses, Views, Knowledge Models)

 

Next step is to create a GIT repository (we use Github, which allows you to make private repositories and create company accounts). If you are working in ML Workbench, GIT is integrated. Once you have downloaded all packages/jobs that you want to include, you publish them in a GIT repository branch and create a merge request for each functionality that you created. Once approved, you should run a second notebook that removes the changed parts and build them based on the GIT Repository main branch (being your production version).

 

More examples what you could do with PyCelonis are available via Github: celonis/pycelonis-examples: Demo notebooks covering popular functionalities and use cases of PyCelonis (github.com). Please be aware that these examples are for PyCelonis version <=1.7, and not the current 2.0. Therefore, check the documentation to see how the syntax has changed.

 

I hope this helps!

ravi.kumar23
Level 7
  • ravi.kumar23
  • Level 7
  • January 18, 2023

Hi Ravi,

 

Using the functions in the 'ems' section of the PyCelonis API (link: Index - PyCelonis), you can obtain all objects you want:

  • All Data Jobs (Extractions/Transformations) per data job
  • All Data Models (Tables, Keys)
  • All packages and their content (Analyses, Views, Knowledge Models)

 

Next step is to create a GIT repository (we use Github, which allows you to make private repositories and create company accounts). If you are working in ML Workbench, GIT is integrated. Once you have downloaded all packages/jobs that you want to include, you publish them in a GIT repository branch and create a merge request for each functionality that you created. Once approved, you should run a second notebook that removes the changed parts and build them based on the GIT Repository main branch (being your production version).

 

More examples what you could do with PyCelonis are available via Github: celonis/pycelonis-examples: Demo notebooks covering popular functionalities and use cases of PyCelonis (github.com). Please be aware that these examples are for PyCelonis version <=1.7, and not the current 2.0. Therefore, check the documentation to see how the syntax has changed.

 

I hope this helps!

Thank you @janpeter.van.d This is helpful.

Community Code of ConductAccessibility statement