Skip to content

feat: add Go plugin support for custom lint rules #1487

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

julioz
Copy link

@julioz julioz commented Apr 19, 2025

Add Go Plugin Support for Custom Lint Rules

This PR implements support for custom lint rules via Go plugins, addressing the need discussed in issue #1485.

Background

Organizations adopting AIPs often need to extend the api-linter with custom rules specific to their requirements. Previously, this required forking the repository and maintaining a custom version, which introduces operational overhead.

Implementation

This PR adds a --rule-plugin flag to the api-linter CLI, allowing users to load custom rules from Go plugin (.so) files at runtime. The implementation:

  1. Uses Go's native plugin package to dynamically load custom rules
  2. Defines a simple plugin API: plugins must export an AddRules function with the signature:
    func AddRules(r lint.RuleRegistry) error
  3. Provides sample code and build scripts for plugin authors

This approach follows the "simple implementation" described in issue comment #1485, focusing on a straightforward solution that can be implemented quickly with minimal changes.

Reviewing the Changes

I recommend reviewing this PR commit by commit to understand the focused nature of each changeset:

  1. First commit introduces the core plugin loading functionality
  2. Second commit adds the build-all.sh script for compatible builds
  3. Third commit improves the plugin build script with version detection
  4. Final commit adds the test proto file for demonstration

Each commit solves a specific problem related to the plugin system implementation.

Challenges & Solutions

Go's plugin system has strict version compatibility requirements - plugins must be built with:

  • The exact same Go version as the api-linter binary
  • The exact same dependency versions

To address this, we provide:

  1. A build-all.sh script that builds both the api-linter and its plugin in the same environment
  2. A julioz-api-rules/build.sh script with version detection to ensure compatibility

Testing

To run the tests for the plugin functionality:

# Run all tests in the cmd/api-linter package
go test ./cmd/api-linter/...

# Run specifically the plugin loading tests
go test ./cmd/api-linter -run TestLoadCustomRulePlugins

The build-all.sh script also includes a test phase that validates the plugin works with a sample proto file:

./build-all.sh

Usage Example

Building

# Build both api-linter and the plugin
./build-all.sh

# Output:
# Building with go version go1.23.3 darwin/arm64
# Building api-linter...
# ...
# Built plugin with:
# julioz-api-rules.so: go1.23.3
# ...
# ✓ Plugin test successful

Sample Plugin

The PR includes a sample plugin with a custom rule requiring message names to start with "Julio":

// This message should fail our rule since it doesn't start with "Julio"
message TestMessage {
  // The name of the test message
  string name = 1;
}

// This message should pass our rule
message JulioTestMessage {
  // The ID of the test message
  int32 id = 1;
}

Running the Linter

./api-linter-with-plugins --rule-plugin=julioz-api-rules/julioz-api-rules.so --proto-path=. julioz-api-rules/test.proto

Output

- file_path: julioz-api-rules/test.proto
  problems:
    - message: Message names must start with 'Julio'
      suggestion: JulioTestMessage
      location:
        start_position:
            line_number: 13
            column_number: 1
        end_position:
            line_number: 16
            column_number: 1
        path: julioz-api-rules/test.proto
      rule_id: internal::9001::julio-prefix
      rule_doc_uri: ""

Future Considerations

Given the plans for a major version bump in the near future, there are opportunities to improve the plugin system:

  1. Replacing this implementation with an RPC-based plugin system like hashicorp/go-plugin, which would be more resilient to Go version incompatibilities
  2. Publishing guidelines for plugin authors and providing better documentation

julioz added 9 commits April 19, 2025 13:22
Add the internal group function to support AIP numbers greater than 9000, which is reserved for internal/custom use as per https://google.aip.dev/adopting
This implementation demonstrates a 'Julio prefix' rule (AIP 9001) as a standalone Go plugin.

Key components:

1. Plugin contract: Exports 'AddCustomRules(registry lint.RuleRegistry) error'
2. Plugin structure:
   - main.go - Entry point exporting AddCustomRules
   - rules.go - Rule definition
   - rule_test.go - Unit tests
   - build.sh - Build script that handles versioning

3. AIP usage: Uses AIP 9001 from the 9000+ block reserved for custom/internal rules
   as per https://google.aip.dev/adopting

This plugin can be loaded dynamically by the api-linter once CLI support
is implemented. The plugin must be built with exactly the same Go version
and dependency versions as the api-linter binary.

References:
- Issue discussion: googleapis#1485
- Go plugin docs: https://pkg.go.dev/plugin
Add a new flag to the api-linter CLI that allows specifying custom rule plugins.
This is the first step toward supporting a plugin ecosystem for custom rules.

Changes:
- Added RulePluginPaths field to the cli struct to store plugin file paths
- Added --rule-plugin flag that can be used multiple times for multiple plugins
- Updated tests to verify the flag works correctly

The flag follows the pattern of other path-related flags in the CLI:
- Uses StringArrayVar to support multiple values
- Consistent naming with other flags (kebab-case)
- Clear documentation in the help text

This is part of the implementation for issue googleapis#1485, enabling organization-specific
custom rules through plugins rather than requiring users to fork the repository.

References:
- Issue discussion: googleapis#1485
- Go plugin docs: https://pkg.go.dev/plugin
Add implementation for loading custom rule plugins from .so files.
This provides the core functionality needed to dynamically load
and register user-provided rule plugins at runtime.

Changes:
- Created plugin.go with loadCustomRulePlugin and loadCustomRulePlugins functions
- Defined the plugin contract with a fixed function name "AddCustomRules"
- Added comprehensive error handling for plugin loading issues
- Created basic unit tests for the plugin loading functions

The implementation:
- Uses Go's native plugin package for loading shared libraries
- Follows a simple and well-defined contract for plugin authors
- Provides detailed error messages to aid in troubleshooting
- Keeps plugin loading separate from the main CLI flow for better modularity

This is part 2/3 of the plugin system implementation for issue googleapis#1485.
With this change, the api-linter has the core capability to load
plugins, but it still needs to be integrated into the CLI flow.
Complete the plugin system implementation by integrating the plugin loader
into the CLI workflow. This connects the previously added components to create
a fully functional plugin system.

Changes:
- Updated the lint method to load plugins before running the linter
- Added informative message showing how many plugins were loaded
- Ensured proper error handling and propagation

With this change, the plugin system is now complete and operational.
Users can now extend the api-linter with custom rules by:
1. Creating plugins with the AddCustomRules function
2. Building them as shared libraries (.so files)
3. Specifying them with the --rule-plugin flag when running api-linter

This completes the 3-part implementation for issue googleapis#1485, providing
a way for organizations to extend the linter with custom rules without
forking the entire repository.
This change removes the "Loaded X custom rule plugin(s)" message from the api-linter
output to maintain clean YAML/JSON output for tools that parse the linter results.

The message was informational only and could break automated tools that expect
valid YAML or JSON output.
This updates the test proto file to:
- Add proper package and Java package options
- Include field comments following best practices
- Disable non-essential linter checks with inline comments
- Ensure the file demonstrates the custom rule properly

These changes make the file suitable for clean demos of the custom rule
functionality without distracting linter warnings.
Updates the plugin build script to:
- Detect the installed api-linter version automatically
- Use the detected version as the build target
- Provide clear compatibility instructions in output
- Properly initialize and maintain Go module dependencies

These improvements help ensure version compatibility between
the plugin and the api-linter binary it will be used with.
This script builds both the api-linter and its plugin together
in a compatible environment, ensuring they use identical dependencies
and compiler versions.

The script:
- Builds the api-linter binary with plugin support
- Creates a properly configured plugin build environment
- Validates that both components are built with the same versions
- Tests the plugin against a sample proto file

This addresses the Go plugin system's strict version compatibility
requirements for a more reliable development workflow.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant