Our Robotframework script model

[Poundex]

I think our Robot script model of 1 script == 1 file might be introducing unnecessary and surprising inflexibility
to users' scripts.

The Robotframework documentation encourages organisation and DRY practices by introducing, early on, the concepts of
including other Robot files as resources
(usually for custom keyword definitions)
and
including local Python files as libraries.

It seems to be normal Robotframework practice (and therefore something which may be expected by our users) to be able
to use these to write stanzas similar to:

*** Settings ***
Resource    some_shared_custom_keywords.resource
Library     some_shared_custom_library.py

where this .resource file and this .py file are local to the script referencing them.

However, our current model where we expect a Robotframework script to exist in one single fully self-contained file
precludes both of these functionalities. While it is possible for users to encapsulate their Robotframework resources
or custom Python sources in a Python library that can be included in the extra-requirements, this is burdensome and
against the Robotframework conventions.

If the RPAResource were to allow an arbitrary Map of files, rather than just a single script key, then in the simple
case we could embed supporting files, removing the single file limitation:

{
  "id": "some_script",
  "name": "some_script",
  "script": "... the script ..."
}

{
  "id": "some_script",
  "name": "some_script",
  "files": {
    "main.robot": "... the script (referencing the Resource and the Library) ...",
    "some_keywords.resource": "... another script ...",
    "functions.py": "... some Python ..."
  }
}

We can then use the existing LinkedResource behaviour to enable and promote resource re-use.
For instance, if we had two scripts that did different, unrelated jobs, but the target of both of those jobs was the
same application, then we could allow the users to name these sets of resources as new LinkedResources.

The shared resource set:

{
  "id": "legacy_internal_application_resources",
  "name": "legacy_internal_application_resources",
  "files": {
    "legacy_internal_application_keywords.resource": "...",
    "legacy_internal_application_functions.py": "..."
  }
}

Re-using across different jobs:

<zeebe:linkedResource 
        resourceId="legacy_internal_application_resources" 
        resourceType="RPA" 
        linkName="RPAScript::support" />
<zeebe:linkedResource 
        resourceId="give_all_employees_a_pay_rise_in_the_legacy_internal_application"
        resourceType="RPA" 
        linkName="RPAScript" />

<zeebe:linkedResource 
        resourceId="legacy_internal_application_resources" 
        resourceType="RPA" 
        linkName="RPAScript::support"/>
<zeebe:linkedResource 
        resourceId="order_more_pens_in_the_legacy_internal_application" 
        resourceType="RPA" 
        linkName="RPAScript"/>

[marstamm]

Your proposal looks solid and works with all the technology we already have. Great work, I like it!

I have some thoughts on the user experience when actually working on bigger projects. Just adding some Ideas and looking for feedback :)

Thinking from the user experience, if they work with bigger scripts, using the Camunda Modeler might not be Ideal. Having multiple files be present in a single .rpa would require you to open files to see which files are included there. I would like to keep a directory structure as long as possible when working with big files.

An easy way to do this would be expanding Process applications to allow generic resources and being able to reference them by path. This was possible with the Camunda 7 deployment model.

How this could work:

• Have an "include additional files" dialog in the modeler
  
• Add your additional .py and .resource in deployment step
• Resources are resolved from deployment, no need to reference additional files manually

Cons:

• Needs additional work on Zeebe
• binding to Deployment only

---

We could also use the same UI, but internally create and deploy the "files": { structure from the original proposal right before deployment

[Poundex]

I think we can keep the directory structure, and it should be preferred. It won't be a "real" directory and as you say users would need a way of viewing the contents of the RPA files but I think this can be achieved with a new, quite simple UI component to explore the contents of the RPA files.

I don't think I follow the deployment process described. It looks to me as if users would have to manually attach any additional files. I can see how this gets them into Zeebe, but don't understand how they can be referenced by scripts (or how the Worker would know which files are to be pulled out of Zeebe and written where with what path.

I have half an idea around our RPA file format becoming a project container file, representing a virtual file system that the Modeler (and Worker) knows how to use.

[marstamm]

users would need a way of viewing the contents of the RPA files

The view inside of the RPA file is clear, it is basically a "project file" that contains all the relevant RPA files. What I meant is working with multiple projects that reference each other, it is harder if we bake everything into our .rpa format.
With bigger RPA projects, it is more likely that these will be versioned in Git and require reviews, which is harder with our format.

but don't understand how they can be referenced by scripts (or how the Worker would know which files are to be pulled out of Zeebe and written where with what path.

I have no complete solution for this yet, just wanted to bring up the solutions we used in C7. Basically, you could deploy complete directories as projects. Not necessarily from the modeler, but also directly via CMD.
This allows you to deploy your complete project, with bpmn, dmn + scripts versioned together. You could then reference any resource by its file path + binding deployment.
This would clash currently, as we do not use .robot files but .rpa, but the concept of "project deployment" might be reusable

[Poundex]

you could deploy complete directories as projects

I think this is the thing I was missing. The "Include Additional Files" control in the screenshot made me think that users would have to manually name every single file they wanted to deploy. But if you can deploy it as a directory, and access it as a directory then I think that would work.

A container file format like this does not normally play well with ordinary version control, no.
If Zeebe is the script source, and Zeebe is doing the version management, then we might be able to abstract this away with our logical view of the files that are in it.
But if users are storing their files in Git (especially for larger, multi-file setups like the ones we're discussing) then why not just allow Git to be an additional script source, let it do the change/version control, not bring the RPA format into play, and have the Worker pull things directly from the source of truth?

[marstamm]

Obviously this would require much more work on the Zeebe side, and might even be worse than doing the project file approach, just wanted to throw in alternatives we could consider for handling multiple files :)

[mattli2024]

@PhilippaC22
See attached for LFWF designs