Declarative macro_rules! derive macros

[joshtriplett]

Many crates support deriving their traits with derive(Trait). Today, this
requires defining proc macros, in a separate crate, typically with several
additional dependencies adding substantial compilation time, and typically
guarded by a feature that users need to remember to enable.

However, many common cases of derives don't require any more power than an
ordinary macro_rules! macro. Supporting these common cases would allow many
crates to avoid defining proc macros, reduce dependencies and compilation time,
and provide these macros unconditionally without requiring the user to enable a
feature.

I've reviewed several existing proc-macro-based derives in the ecosystem, and
it appears that many would be able to use this feature to avoid needing proc
macros at all.

Rendered

[joshtriplett]

Nominated as a follow-up to recent lang discussions about this.

[liigo]

Since its call-side syntax is different from normal macro_rules! (i.e. macroname!()), the definition-side syntax should also variants e.g. macro_rules_derive! (v.s. #[proc_macro_derive(TraitName)]).

The same to #3697 where we could name it macro_rules_attribute! (v.s. #[proc_macro_attribute]).

[coolreader18]

One thing that would be really nice (i.e. greatly increase usability of decl macros) (though would not at all decrease the usefulness of this feature) is proper, full-featured parsing of meta fragments of decl macros. Right now, if you want to parse arguments to a decl macro in a MetaList form, you have to either enforce that the named arguments come in a specific order, or use a tt-muncher (and even then, it's not clear at first thought how you'd do it!). Obviously this is something that would introduce a whole new axis of complexity to decl macro declarations and evaluation, but being able to specify something like $[$(foo = $foo:expr)? $(bar($bar:expr))*],* and accept 0 or 1 foos and 0 or more bars in any order would be a game changer, and I can't see using attr or derive decl macros being very ergonomic for anything but the simplest cases without something like that. This RFC lets you define helper attributes for decl derive macros, but without any easy way of extracting them from out of the set of other attributes on the item, I don't see that being very useful for most people. An accessible and intuitive macro definition format tt-munchers are not.

(Shoulda brought this up at the hopes and dreams for the language session at Unconf 🙃)

[joshtriplett]

@coolreader18 I would love to see many parsing improvements for macros, including something to address this kind of parsing difficulty. I don't think that's specific to derive or attribute macros, though it certainly makes them more useful.

[matthieu-m]

To define helper attributes, put an attributes key in the macro_derive attribute, with a comma-separated list of identifiers for helper attributes: #[macro_derive(attributes(helper))]. The derive macro can process the #[helper] attribute, along with any arguments to it, as part of the item the derive macro was applied to.

In order to avoid name collisions in helper attributes between different derives, I think it would be worth it to take a page out of serde here and require namespacing of those attributes.

For example, simple serde code:

#[derive(Debug, Deserialize)]
struct Message {
    #[serde(rename = "type")]
    type_: String,
    #[serde(default = "Message::default_payload")]
    payload: String,
}

Some names (like default) are likely to be particularly sought after, and mixing dependencies which clash on those helper names will be nightmarish.

Convention for the namespace could dictate that it be either a parent module/crate name, or the name of the trait, snake-cased.

[jplatte]

I don't think that's a good idea. It's already the case that other libraries than serde look at serdes attributes, sometimes even in a context where the input type does not use serdes derives at the same time (but a proc-macro copies the serde attributes to some types it generates). As one example, the derive macro schemars::JsonSchema currently "registers" serde, schemars and validate, likely for good reason.

[ssokolow]

I don't think that's a good idea. It's already the case that other libraries than serde look at serdes attributes, sometimes even in a context where the input type does not use serdes derives at the same time (but a proc-macro copies the serde attributes to some types it generates). As one example, the derive macro schemars::JsonSchema currently "registers" serde, schemars and validate, likely for good reason.

Perhaps some kind of pub annotation then? Yes, that would have made achieving the current state of Serde attribute cross-compatibility difficult, but it has been the Rust way to default to pushing back against Hyrum's law.

[epage]

There has also been talk of "common" attributes, like skip.

For clap, namespacing by crate or derive name was insufficient and it now processes 4 different namespaces.

If we did encourage something by default, i think it should be derive name so there is a clear relationship.

On a simlar note of constraining users, imo derives should only produce a trait impl for the derive and considered proposing that be enforced but figured that deviating for what proc-macros provide would also be a downside. Also, I've seen with clap how it can be useful to include mostly-internal trait impls with the requested one.

[kpreid]

On a simlar note of constraining users, imo derives should only produce a trait impl for the derive

Note that imposing such restriction would make it difficult to write derives that implement traits like IntoIterator, where each implementation typically introduces a new type that must be nameable by the impl, until associated type position impl Trait is available.

I was recently exploring this space while working on exhaust 0.2, which needs to generate two types and several impls along with the nominally derived impl, and my current strategy for being nice to the macro’s users is that the derive macro’s generated items are all inside a const _: () = { ... } block; this ensures that the macro’s expansion has no effect on the caller’s namespace, even though more items than just a trait impl are generated. One could imagine inserting that block wrapper automatically as part of the semantics of derive macros (whether declarative or procedural), but that should probably be its own design discussion separate from introducing a new kind of macro implementation.

[matthieu-m]

I don't think that's a good idea.

For clap, namespacing by crate or derive name was insufficient and it now processes 4 different namespaces.

I would note that I specifically mentioned "as a convention".

If anything, the fact that schemars also looks at serde namespace is a validation of the need for namespacing to me. Otherwise if there's a #[default = ...] it wouldn't even know whether that's for serialization/deserialization or some unrelated purpose.

I think enforcing namespacing and having a simple convention for simple usecases -- so folks can just follow along -- would work well. Perhaps a clippy lint which has to be explicitly #[expect] for the handful of usecases requiring more schemas.

There has also been talk of "common" attributes, like skip.

Common attributes are fine: a standardized meaning should not cause "mishaps".

[joshtriplett]

We discussed this briefly in today's planning meeting.

Based on that discussion, I updated the Motivation section to include the concrete examples motivating this.

I'm also going to restart the proposed FCP, since team membership has changed since it was previously started.

@rfcbot cancel

[rfcbot]

@joshtriplett proposal cancelled.

[joshtriplett]

@rfcbot merge

[rfcbot]

Team member @joshtriplett has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @scottmcm
• @tmandry
• @traviscross

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[joshtriplett]

Nominated for consideration in a meeting to address any questions that arise.

[tmandry]

That crate likewise offers a derive_alias mechanism, which could similarly be implemented using exclusively declarative macros given the feature proposed in this RFC.

It's not clear to me how this would work exactly. That mechanism works by replacing the built-in derive attribute with a custom one that does some hacky stuff. If the claim is that this feature can be used to implement derive aliases, an example of how it's done would help.

The derive feature of the crate has various uses in the ecosystem.

The ones I see are crate-internal uses. That is a completely valid use case, but it raises a question for me. Do you think this is due to parsing limitations of declarative macros, the inconvenience of requiring users to use macro_rules_attribute::derive, or both? I would guess it's more the latter than the former. Writing out the grammar for any possible struct invocation might be a bit painful, but a person who is motivated could do it. In which case I think this feature stands on its own without #3714, though that would make macros more resilient and easier to write in general.

The motivation section might additionally benefit from reiterating some of the benefits of derives over attribute macros in general (since this is being considered alongside #3697). In particular, they work better with tools like rust-analyzer because the tool always knows the item that follows is a normal, valid item, without having to expand the macro.

[joshtriplett]

That crate likewise offers a derive_alias mechanism, which could similarly be implemented using exclusively declarative macros given the feature proposed in this RFC.

It's not clear to me how this would work exactly. That mechanism works by replacing the built-in derive attribute with a custom one that does some hacky stuff. If the claim is that this feature can be used to implement derive aliases, an example of how it's done would help.

Hmmm, good point. I've moved this mention to future work, alongside the mention of having a way for derive macros to invoke other derive macros.

The derive feature of the crate has various uses in the ecosystem.

The ones I see are crate-internal uses. That is a completely valid use case, but it raises a question for me. Do you think this is due to parsing limitations of declarative macros, the inconvenience of requiring users to use macro_rules_attribute::derive, or both? I would guess it's more the latter than the former. Writing out the grammar for any possible struct invocation might be a bit painful, but a person who is motivated could do it. In which case I think this feature stands on its own without #3714, though that would make macros more resilient and easier to write in general.

I would guess that in part it's the inconvenience and oddity of requiring use macro_rules_attribute::derive, and in part it might be hesitance to add an extra library dependency.

And I agree. I don't think there are any dependencies between these RFCs; they just benefit from each other.

The motivation section might additionally benefit from reiterating some of the benefits of derives over attribute macros in general (since this is being considered alongside #3697). In particular, they work better with tools like rust-analyzer because the tool always knows the item that follows is a normal, valid item, without having to expand the macro.

Done.

[tmandry]

Thanks @joshtriplett. The uses of macro_rules_attribute::derive in the ecosystem demonstrate to me that this is a useful feature in practice. I think it would be more widespread if using it didn't require adding a new library dependency. #3714 should make this a more useful feature, but I'm persuaded that the feature stands on its own.

@rfcbot reviewed

[nikomatsakis]

@rfcbot reviewed

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[traviscross]

@rfcbot reviewed

Thanks @joshtriplett for putting this forward. Nicely done. Everything I said in #3697 (comment) applies here as well.

For our future selves, note that this RFC includes things that anticipate RFC #3715. That's OK, I think. I'd estimate that it's more likely than not that we do end up moving that one forward as well.

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[bew]

nit: There seems to be a missed opportunity to have Universe::answer by a thing here 🤔

Suggested change

	#[derive(Answer)]
	struct Struct;
	fn main() {
	let s = Struct;
	assert_eq!(42, s.answer());
	}
	#[derive(Answer)]
	struct Universe;
	fn main() {
	let u = Universe;
	assert_eq!(42, u.answer());
	}