Support for user-defined syntax entities in structurizr dsl

[Nifacy]

Description

Summary

Feature will allow users to define custom syntax entities within Structurizr DSL,
enhancing it's extensibility and adaptability to diverse modeling requirements.

Background

Currently, structurizr dsl offers a fixed set of syntax entities for modeling software
architectures. While this sufficies for many scenarios, certain use cases necessitate
the definition of custom entities to represent domain-specific concepts more accurately.

Relevance

In large organizations like ours, recurring architectural patterns have emerged across
various projects. To effectively document and communicate these patterns within
our architecture descriptions, we require the ability to define them as distinct entities
in structurizr dsl.

While initial attempts using plugins and scripts provided some flexibility, a more
integrated and flexible approach is now necessary.

Priority

Low (I'm willing to make a pull request - please confirm approach first)

More information

Proposal

Implement functionality enabling users to create their own syntax entities
using a defined structure and parsing mechanism.

Details

Custom syntax entity format

Users can define custom entities using the following template:

$<entity-keyword> <param> ... {
    ...
}

Where:

• <entity-keyword> - a unique keyword identifying the custom entity type.
  It must be alphanumeric and contain no spaces.

• <param> ... - a set of parameters, each representing a distinct token.

• { ... } - a context block containing additional information about the entity,
  similar to existing entities like container or group.

Parsing mechanism

Users will implement parsing logic for their custom entities,
adhering to a specified interface. This approach ensures that
the internal components of structurizr dsl remain encapsulated,
allowing users to extend functionality without having to
disclose the internal components form structurizr project.

The proposed interfaces are as follows:

Example

Here is an example of custom entity example:

import com.structurizr.dsl.CustomSyntaxEntity
import com.structurizr.dsl.CustomSyntaxEntityParser
import com.structurizr.dsl.CustomSyntaxEntityParserContext


class ExampleSyntaxEntityParser implements CustomSyntaxEntityParser {

    private String name;
    private String fieldA;
    private String fieldB;

    ExampleSyntaxEntityParser() {
        this.name = null;
        this.fieldA = null;
        this.fieldB = null;
    }

    public void parseHeader(String[] tokens) {
        if (tokens.length != 1) {
            throw new RuntimeException("Expected: $example <name>")
        }

        this.name = tokens[0];
    }

    public void parseContextBlockLine(String[] tokens) {
        if (tokens.length != 2) {
            throw new RuntimeException("Expected: <name> <value>");
        }

        String fieldName = tokens[0];
        String value = tokens[1];

        if (fieldName == "a") {
            if (this.fieldA != null) {
                throw new RuntimeException("Field 'a' already defined");
            }
            this.fieldA = value;
        }

        else if (fieldName == "b") {
            if (this.fieldB != null) {
                throw new RuntimeException("Field 'b' already defined");
            }
            this.fieldB = value;
        }

        else {
            throw new RuntimeException("Unknown field: " + key);
        }
    }

    public void onEnd() {
        System.out.println("Syntax entity info:");
        System.out.println("- name: " + this.name);
        System.out.println("- field a: " + this.fieldA);
        System.out.println("- field b: " + this.fieldB);
    }
}


class ExampleSyntaxEntity implements CustomSyntaxEntity {
    public String getKeyword() {
        return "example";
    }

    public CustomSyntaxEntityParser getParser(CustomSyntaxEntityParserContext context) {
        System.out.println("Parser context:");
        System.out.println("- workspace: " + context.getWorkspace());
        System.out.println("- file: " + context.getDslFile());
        System.out.println("- parser: " + context.getDslParser());

        return new ExampleSyntaxEntityParser();
    }
}

Conclusion

Benefits

• Users can tailor the DSL to their specific needs without modifying the core library.
• Facilitates the representation of domain-specific concepts, enhancing the
  expressiveness of models.
• Maintains the integrity of core components by exposing only necessary interfaces
  for extension.

[Nifacy]

Some sketches regarding this feature are already ready. You can view them here: link.

[DVDemon]

Good Idea!
Such mechanism could be used to separate logical description of architectural model from description of patterns realizing qualitative characteristics.

[szuev]

Sounds good!

[FightingFox]

I need this mechanism too! Very good.

[neopaf]

Some sketches regarding this feature are already ready. You can view them here: link.

Any pictures?

[Nifacy]

I’d be happy to provide graphical illustrations for my sketches! But I’m afraid they wouldn’t be very useful, as the sketches are purely code-based. Taking a screenshot of the code doesn’t seem like the best idea :(((

[neopaf]

I’m trying to understand the idea from the end result perspective.

Diagrams don’t change, but the code to express relations end entities may get simpler? More elegant?

maybe give a small example of a diagram and code to express it before/after proposed change?

[Nifacy]

the code to express relations end entities may get simpler? More elegant?

Yep, indeed, separate syntax for some entities may simplify their description and make it more natural!

maybe give a small example of a diagram and code to express it before/after proposed change?

Yeah, i think it will be useful for undersanding problem. Added one of usage examples in this comment!

[archdevmikhail]

A much needed improvement! I support it! =)

[voriv]

This feature would be useful for me!

[vasilenkoalexey]

Looks interesting. As far as I understand, this extension will allow to describe custom architecture details in a more correct way (instead of using properties). I support it.

[Nifacy]

Yep! Moreover, this will provide significantly more flexibility in describing, as the format of the data defined within context block can be completely arbitrary!

[Nifacy]

@simonbrowndotje, hello! I have a question about current discussion. Does it make sense for me to proceed with refining my sketches and submit a PR for review?

[simonbrowndotje]

Does it make sense for me to proceed with refining my sketches and submit a PR for review?

No ... I don't understand what the problem is you're trying to solve. Some usage examples would be useful too.

[Nifacy]

Describe one of usage examples in follow comment (link)

[Nifacy]

@simonbrowndotje, was the example informative, or are there still questions about problem being solved?

[Nifacy]

Here’s another example where extending the syntax would make a big difference.

In my project (see the "Relevance" section), we currently implement architectural patterns as plugins for Structurizr DSL. But this approach has some serious drawbacks:

1. We are facing rigid syntax limitations. Pattern parameters can only be defined as <field> <value>, where the value is just a string or a number;
2. Conceptual mismatch. Patterns aren’t really plugins. Using plugins to define them feels like a workaround and imposes unnecessary constraints.

Here’s how "Reverse Proxy" and "Saga" patterns are currently represented:

workspace "Reverse Proxy Example" {
    model {
        customer = person "Customer"

        s = softwareSystem OnlineShop {
            app2 = container "Some other container"

            webApp = container "Web App" {
                technology "Java, Spring Boot"
            }

            app3 = container "Container 3"

            db = container "Database" {
                technology "PostgreSQL"
            }

            db2 = container "Database 2" {
                technology "Mongo"
            }
        }

        app2 -> webApp "Some relationship"
        customer -> webApp "Sends Requests"
        webApp -> db "Requests data"
        webApp -> db2 "Requests users"
        app3 -> db2 "Another relationship"

       !plugin com.patterns.ReverseProxy {
           target webApp
       }
    }
}

workspace "Saga example" {
    model {
        user = person "User"

        orderSystem = softwareSystem "Order System" {
            api = container "API"
            orchestrator = container "Saga Orchestrator"
            orderService = container "Order Service"
            balanceService = container "Balance Service"

            user -> api "Creates Order"
            api -> orchestrator "Sends request to create order transaction"

            !plugin com.patterns.Saga {
                orchestrator    orchestrator

                item.0.service  orderService
                item.0.command  "Set order status PENDING"
                item.0.onError  "Set order status CANCELED"

                item.1.service  balanceService
                item.1.command  "Reserve funds"
                item.1.onError  "Refund"

                item.2.service  orderService
                item.2.command  "Set order status APPROVE"
                item.2.onError  "Set order status CANCELED"
            }
        }
    }
}

Extending the DSL syntax solves both of these issues by allowing patterns to be defined as first-class entities. This brings:

• More flexibility. Pattern parameters can have a more natural, structured format that fits each specific pattern;
• Better design. Patterns are treated as distinct concepts, making the representation more intuitive and scalable.

With the new syntax, these same patterns could be written like this:

workspace "Reverse Proxy Example" {
    !plugin com.patterns.PatterSyntaxPlugin
    // run plugin which will add new syntax entity
    // "$pattern"

    model {
        customer = person "Customer"

        s = softwareSystem OnlineShop {
            app2 = container "Some other container"

            webApp = container "Web App" {
                technology "Java, Spring Boot"
            }

            app3 = container "Container 3"

            db = container "Database" {
                technology "PostgreSQL"
            }

            db2 = container "Database 2" {
                technology "Mongo"
            }
        }

        app2 -> webApp "Some relationship"
        customer -> webApp "Sends Requests"
        webApp -> db "Requests data"
        webApp -> db2 "Requests users"
        app3 -> db2 "ABOBA"

       $pattern ReverseProxy {
           target webApp
       }
    }
}

workspace {
    !plugin com.patterns.PatterSyntaxPlugin
    // run plugin which will add new syntax entity
    // "$pattern"

    model {
        user = person "User"

        orderSystem = softwareSystem "Order System" {
            api = container "API"
            orchestrator = container "Saga Orchestrator"
            orderService = container "Order Service"
            balanceService = container "Balance Service"

            user -> api "Creates Order"
            api -> orchestrator "Sends request to create order transaction"

            $pattern Saga {
                orchestrator     orchestrator
                items [
                    (
                        service  orderService
                        command  "Set order status PENDING"
                        onError  "Set order status CANCELED"
                    )

                    (
                        service  balanceService
                        command  "Reserve funds"
                        onError  "Refund"
                    )

                    (
                        service  orderService
                        command  "Set order status APPROVE"
                        onError  "Set order status CANCELED"
                    )
                ]
            }
        }
    }
}

This makes architectural descriptions much clearer and more expressive while eliminating the limitations of plugins. Instead of just improving the current approach, this syntax extension introduces a more scalable and conceptually correct way to work with patterns.