Skip to content

Add Prost and Tonic rules. #2033

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 31 commits into from
Jun 30, 2023
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
6cb92b2
Setup Prost and Tonic rules.
freeformstu Jun 25, 2023
2c6d472
Regenerate documentation
freeformstu Jun 25, 2023
fced3e7
Add more tests.
freeformstu Jun 27, 2023
09d8142
Add more tests and address feedback.
freeformstu Jun 27, 2023
c1f3d99
Regenerate documentation
freeformstu Jun 27, 2023
a603dc2
Add to proto docs page.
freeformstu Jun 27, 2023
cb0ab09
Bump min supported bazel version.
freeformstu Jun 27, 2023
a1714e1
buildifier
freeformstu Jun 27, 2023
f3ccc01
Always enable backtracing.
freeformstu Jun 27, 2023
a58749c
Add more info to failing rename.
freeformstu Jun 27, 2023
d7e7312
Set min rust version to 1.62.0
freeformstu Jun 27, 2023
d67c839
Handle rust keywords as package names.
freeformstu Jun 27, 2023
e03d107
exclude windows from prost toolchain support.
freeformstu Jun 27, 2023
e901991
buildifier
freeformstu Jun 27, 2023
c4fe6aa
redundant
freeformstu Jun 27, 2023
0f80615
Use prost-types to parse the file descriptor set.
freeformstu Jun 28, 2023
2870e43
Cleanup and more tests.
freeformstu Jun 28, 2023
6554778
Move prost-types to toolchain definition.
freeformstu Jun 28, 2023
7e82121
fix rustfmt
freeformstu Jun 28, 2023
00682b3
Add example of building protos with complex imports
illicitonion Jun 28, 2023
6ceafbb
impl Display
freeformstu Jun 28, 2023
e848353
Fix all tests
freeformstu Jun 28, 2023
3c91f2c
Add rust checks for the complex import protos.
freeformstu Jun 28, 2023
e08d9ad
Address feedback
freeformstu Jun 29, 2023
d3ea3d7
Fix buildifier
freeformstu Jun 29, 2023
74e5b08
Depend on remote-apis repo.
freeformstu Jun 30, 2023
9bd48e1
Remove bazel remote apis due to file length and transitive dependency…
freeformstu Jun 30, 2023
b9b2872
Update patch and docs.
freeformstu Jun 30, 2023
5d64e3c
Regenerate documentation
freeformstu Jun 30, 2023
38b9989
Regenerate documentation
freeformstu Jun 30, 2023
7ac0bd1
Update docs.
freeformstu Jun 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .bazelci/presubmit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ aspects_flags: &aspects_flags
- "--config=rustfmt"
- "--config=clippy"
min_rust_version_shell_commands: &min_rust_version_shell_commands
- sed -i 's|^rust_register_toolchains(|rust_register_toolchains(versions = ["1.59.0"],\n|' WORKSPACE.bazel
- sed -i 's|^rust_register_toolchains(|rust_register_toolchains(versions = ["1.62.0"],\n|' WORKSPACE.bazel
nightly_flags: &nightly_flags
- "--//rust/toolchain/channel=nightly"
nightly_aspects_flags: &nightly_aspects_flags
Expand Down Expand Up @@ -205,15 +205,15 @@ tasks:
ubuntu1804:
name: "Min Bazel Version"
# If updating the minimum bazel version, please also update /docs/index.md
bazel: "5.2.0"
bazel: "6.0.0"
platform: ubuntu1804
build_targets: *default_linux_targets
test_targets: *default_linux_targets
coverage_targets: *default_linux_targets
post_shell_commands: *coverage_validation_post_shell_commands
ubuntu1804_with_aspects:
name: "Min Bazel Version With Aspects"
bazel: "5.2.0"
bazel: "6.0.0"
platform: ubuntu1804
build_targets: *default_linux_targets
test_targets: *default_linux_targets
Expand Down
6 changes: 4 additions & 2 deletions WORKSPACE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ load("@rules_rust//crate_universe:repositories.bzl", "crate_universe_dependencie

crate_universe_dependencies(bootstrap = True)

load("@rules_rust//proto:repositories.bzl", "rust_proto_repositories")
load("@rules_rust//proto:repositories.bzl", "rust_proto_dependencies", "rust_proto_register_toolchains")

rust_proto_repositories()
rust_proto_dependencies()

rust_proto_register_toolchains()

load("@rules_rust//proto:transitive_repositories.bzl", "rust_proto_transitive_repositories")

Expand Down
2 changes: 1 addition & 1 deletion crate_universe/private/crates_vendor.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ bazel run //3rdparty:crates_vendor -- --repin
```

Under the hood, `--repin` will trigger a [cargo update](https://doc.rust-lang.org/cargo/commands/cargo-update.html)
call against the generated workspace. The following table describes how to controll particular values passed to the
call against the generated workspace. The following table describes how to control particular values passed to the
`cargo update` command.

| Value | Cargo command |
Expand Down
2 changes: 2 additions & 0 deletions docs/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -121,6 +121,8 @@ PAGES = dict([
"rust_proto_repositories",
"rust_proto_transitive_repositories",
"rust_proto_toolchain",
"rust_prost_library",
"rust_tonic_library",
],
),
page(
Expand Down
2 changes: 1 addition & 1 deletion docs/crate_universe.md
Original file line number Diff line number Diff line change
Expand Up @@ -462,7 +462,7 @@ bazel run //3rdparty:crates_vendor -- --repin
```

Under the hood, `--repin` will trigger a [cargo update](https://doc.rust-lang.org/cargo/commands/cargo-update.html)
call against the generated workspace. The following table describes how to controll particular values passed to the
call against the generated workspace. The following table describes how to control particular values passed to the
`cargo update` command.

| Value | Cargo command |
Expand Down
40 changes: 40 additions & 0 deletions docs/flatten.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
* [rust_library](#rust_library)
* [rust_library_group](#rust_library_group)
* [rust_proc_macro](#rust_proc_macro)
* [rust_prost_library](#rust_prost_library)
* [rust_proto_library](#rust_proto_library)
* [rust_proto_repositories](#rust_proto_repositories)
* [rust_proto_toolchain](#rust_proto_toolchain)
Expand All @@ -43,6 +44,7 @@
* [rust_stdlib_filegroup](#rust_stdlib_filegroup)
* [rust_test](#rust_test)
* [rust_test_suite](#rust_test_suite)
* [rust_tonic_library](#rust_tonic_library)
* [rust_toolchain](#rust_toolchain)
* [rust_toolchain_repository](#rust_toolchain_repository)
* [rust_toolchain_repository_proxy](#rust_toolchain_repository_proxy)
Expand Down Expand Up @@ -1792,6 +1794,25 @@ Registers the default toolchains for the `rules_rust` [bindgen][bg] rules.
| <a id="rust_bindgen_register_toolchains-register_toolchains"></a>register_toolchains | Whether or not to register toolchains. | `True` |


<a id="rust_prost_library"></a>

## rust_prost_library

<pre>
rust_prost_library(<a href="#rust_prost_library-name">name</a>, <a href="#rust_prost_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_prost_library-name"></a>name | The name of the target. | none |
| <a id="rust_prost_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_prost_library</code> rule. | none |


<a id="rust_proto_repositories"></a>

## rust_proto_repositories
Expand Down Expand Up @@ -1996,6 +2017,25 @@ rust_test_suite(
| <a id="rust_test_suite-kwargs"></a>kwargs | Additional keyword arguments for the underyling [rust_test](#rust_test) targets. The <code>tags</code> argument is also passed to the generated <code>test_suite</code> target. | none |


<a id="rust_tonic_library"></a>

## rust_tonic_library

<pre>
rust_tonic_library(<a href="#rust_tonic_library-name">name</a>, <a href="#rust_tonic_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost and Tonic.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_tonic_library-name"></a>name | The name of the target. | none |
| <a id="rust_tonic_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_tonic_library</code> rule. | none |


<a id="rust_toolchain_repository"></a>

## rust_toolchain_repository
Expand Down
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ Failure to do so will result in rules attempting to match a `stable` toolchain w

## Supported bazel versions

The oldest version of Bazel the `main` branch is tested against is `5.2.0`. Previous versions may still be functional in certain environments, but this is the minimum version we strive to fully support.
The oldest version of Bazel the `main` branch is tested against is `6.0.0`. Previous versions may still be functional in certain environments, but this is the minimum version we strive to fully support.

We test these rules against the latest rolling releases of Bazel, and aim for compatibility with them, but prioritise stable releases over rolling releases where necessary.

Expand Down
40 changes: 40 additions & 0 deletions docs/rust_proto.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@
* [rust_proto_repositories](#rust_proto_repositories)
* [rust_proto_transitive_repositories](#rust_proto_transitive_repositories)
* [rust_proto_toolchain](#rust_proto_toolchain)
* [rust_prost_library](#rust_prost_library)
* [rust_tonic_library](#rust_tonic_library)


## Overview
Expand Down Expand Up @@ -260,6 +262,25 @@ See @rules_rust//proto:BUILD for examples of defining the toolchain.
| <a id="rust_proto_toolchain-protoc"></a>protoc | The location of the <code>protoc</code> binary. It should be an executable target. | <a href="https://bazel.build/concepts/labels">Label</a> | optional | <code>@com_google_protobuf//:protoc</code> |


<a id="rust_prost_library"></a>

## rust_prost_library

<pre>
rust_prost_library(<a href="#rust_prost_library-name">name</a>, <a href="#rust_prost_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_prost_library-name"></a>name | The name of the target. | none |
| <a id="rust_prost_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_prost_library</code> rule. | none |


<a id="rust_proto_repositories"></a>

## rust_proto_repositories
Expand Down Expand Up @@ -292,3 +313,22 @@ This macro should be called immediately after the `rust_proto_repositories` macr



<a id="rust_tonic_library"></a>

## rust_tonic_library

<pre>
rust_tonic_library(<a href="#rust_tonic_library-name">name</a>, <a href="#rust_tonic_library-kwargs">kwargs</a>)
</pre>

A rule for generating a Rust library using Prost and Tonic.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_tonic_library-name"></a>name | The name of the target. | none |
| <a id="rust_tonic_library-kwargs"></a>kwargs | Additional keyword arguments for the underlying <code>rust_tonic_library</code> rule. | none |


6 changes: 5 additions & 1 deletion docs/symbols.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,11 @@ load(
_crates_vendor = "crates_vendor",
)
load(
"@rules_rust//proto:proto.bzl",
"@rules_rust//proto:defs.bzl",
_rust_grpc_library = "rust_grpc_library",
_rust_prost_library = "rust_prost_library",
_rust_proto_library = "rust_proto_library",
_rust_tonic_library = "rust_tonic_library",
)
load(
"@rules_rust//proto:repositories.bzl",
Expand Down Expand Up @@ -124,6 +126,8 @@ rust_doc_test = _rust_doc_test

rust_proto_library = _rust_proto_library
rust_grpc_library = _rust_grpc_library
rust_prost_library = _rust_prost_library
rust_tonic_library = _rust_tonic_library

rust_bindgen = _rust_bindgen
rust_bindgen_dependencies = _rust_bindgen_dependencies
Expand Down
1 change: 1 addition & 0 deletions proto/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -48,5 +48,6 @@ bzl_library(
srcs = glob(["**/*.bzl"]),
deps = [
"//proto/3rdparty:bzl_lib",
"//proto/prost:bzl_lib",
],
)
18 changes: 18 additions & 0 deletions proto/defs.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
"""Rust proto rules."""

load(
"//proto/prost:defs.bzl",
_rust_prost_library = "rust_prost_library",
_rust_tonic_library = "rust_tonic_library",
)
load(
":proto.bzl",
_rust_grpc_library = "rust_grpc_library",
_rust_proto_library = "rust_proto_library",
)

rust_proto_library = _rust_proto_library
rust_grpc_library = _rust_grpc_library

rust_prost_library = _rust_prost_library
rust_tonic_library = _rust_tonic_library
62 changes: 62 additions & 0 deletions proto/prost/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
load("@bazel_skylib//:bzl_library.bzl", "bzl_library")
load("//rust:defs.bzl", "rust_library_group")
load(":defs.bzl", "rust_prost_toolchain")

package(default_visibility = ["//visibility:public"])

toolchain_type(
name = "toolchain_type",
)

rust_library_group(
name = "prost_runtime",
deps = [
"//proto/prost/private/3rdparty/crates:prost",
],
)

rust_library_group(
name = "tonic_runtime",
deps = [
":prost_runtime",
"//proto/prost/private/3rdparty/crates:tonic",
],
)

rust_prost_toolchain(
name = "default_prost_toolchain_impl",
prost_plugin = "//proto/prost/private/3rdparty/crates:protoc-gen-prost__protoc-gen-prost",
prost_plugin_flag = "--plugin=protoc-gen-prost=%s",
prost_runtime = ":prost_runtime",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure about providing a default toolchain with a rules_rust-private runtime... When trying this out in some of my private repos, I spent quite a while chasing "X doesn't implement trait Y" issues because of my copy of tonic and prost, despite being the same version numbers, being distinct libraries from the bundled ones, before eventually realising I needed to make and register my own toolchain.

I suspect most of our users will need to specify their own toolchains for this reason, and given how inscrutible the errors are, I'd lean towards documenting this as a required step, and not supplying a default at all...

WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, users should just define their own toolchain to avoid any of this confusion. I'll get this deleted.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I moved the toolchain into private/BUILD.bazel and stopped automatically registering it in the public proto dependencies macros.

proto_compiler = "@com_google_protobuf//:protoc",
tonic_plugin = "//proto/prost/private/3rdparty/crates:protoc-gen-tonic__protoc-gen-tonic",
tonic_plugin_flag = "--plugin=protoc-gen-tonic=%s",
tonic_runtime = ":tonic_runtime",
)

toolchain(
name = "default_prost_toolchain_linux",
target_compatible_with = [
"@platforms//os:linux",
],
toolchain = "default_prost_toolchain_impl",
toolchain_type = "//proto/prost:toolchain_type",
)

toolchain(
name = "default_prost_toolchain_osx",
target_compatible_with = [
"@platforms//os:osx",
],
toolchain = "default_prost_toolchain_impl",
toolchain_type = "//proto/prost:toolchain_type",
)

bzl_library(
name = "bzl_lib",
srcs = glob(["**/*.bzl"]),
deps = [
"//proto/prost/private:bzl_lib",
"//rust:bzl_lib",
],
)
56 changes: 56 additions & 0 deletions proto/prost/defs.bzl
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
"""Rules for building proto libraries in Rust."""

load(
"//proto/prost/private:prost.bzl",
_rust_prost_library = "rust_prost_library",
_rust_prost_toolchain = "rust_prost_toolchain",
_rust_tonic_library = "rust_tonic_library",
)

def rust_prost_library(name, **kwargs):
"""A rule for generating a Rust library using Prost.

Args:
name (str): The name of the target.
**kwargs (dict): Additional keyword arguments for the underlying
`rust_prost_library` rule.
"""

# Clippy and Rustfmt will attempt to run on these targets.
# This is not correct and likely a bug in target detection.
tags = kwargs.pop("tags", [])
if "no-clippy" not in tags:
tags.append("no-clippy")
if "no-rustfmt" not in tags:
tags.append("no-rustfmt")

_rust_prost_library(
name = name,
tags = tags,
**kwargs
)

def rust_tonic_library(name, **kwargs):
"""A rule for generating a Rust library using Prost and Tonic.

Args:
name (str): The name of the target.
**kwargs (dict): Additional keyword arguments for the underlying
`rust_tonic_library` rule.
"""

# Clippy and Rustfmt will attempt to run on these targets.
# This is not correct and likely a bug in target detection.
tags = kwargs.pop("tags", [])
if "no-clippy" not in tags:
tags.append("no-clippy")
if "no-rustfmt" not in tags:
tags.append("no-rustfmt")

_rust_tonic_library(
name = name,
tags = tags,
**kwargs
)

rust_prost_toolchain = _rust_prost_toolchain
Loading