bug(`cheatcodes`): vm.expertRevert(bytes) reports misleading failure diagnostic

[norswap]

Component

Forge

Have you ensured that all of these are up to date?

• Foundry
• Foundryup

What version of Foundry are you on?

forge Version: 1.0.0-stable Commit SHA: e144b82 Build Timestamp: 2025-02-13T20:02:34.979686000Z (1739476954) Build Profile: maxperf

What version of Foundryup are you on?

foundryup: 1.0.1

What command(s) is the bug in?

forge test

Operating System

macOS (Apple Silicon)

Describe the bug

If I ran the following snippet in my tests:

vm.expectRevert(abi.encodeWithSelector(ValidationFailed.selector, InvalidNonce.selector));
happyEntryPoint.submit(encodedHappyTx);

I receive the following test failure:

[FAIL: Error != expected error: ValidationFailed(0x756688fe) != ValidationFailed(0x)]

The mistake was that it should have been:

vm.expectRevert(abi.encodeWithSelector(ValidationFailed.selector, abi.encodeWithSelector(InvalidNonce.selector)));

But the ValidationFailed(0x) really threw me for a loop. I guess forge wasn't able to ABI parse the incorrect revertData, leading to display Validation(0x) instead, a half-parsed version.

I would suggest it's more useful to display the entire revertData in this case, if it can't be parsed in its entirety — as foundry already does in some other places (I think when it can't identify the initial selector). Alternatively ValidationFailed(<failed to parse>) would be useful as well.

[grandizzy]

@norswap can you pls provide more info on the test you perform? I made following

error ValidationFailed(bytes4);
error InvalidNonce();
error InvalidNonce1();

contract Counter {
    function revert10162() public {
        revert ValidationFailed(InvalidNonce.selector);
    }
}

and a test like

    function testRevert10162() public {
        vm.expectRevert(abi.encodeWithSelector(ValidationFailed.selector, InvalidNonce.selector));
        counter.revert10162();
    }

is passing OK, while

    function testRevert10162() public {
        vm.expectRevert(abi.encodeWithSelector(ValidationFailed.selector, InvalidNonce1.selector));
        counter.revert10162();
    }

fails with

[FAIL: Error != expected error: ValidationFailed(0x756688fe) != ValidationFailed(0x4fbd93a6)] testRevert10162() (gas: 9154)

Thank you

[norswap]

Yes, the main thing that causes the behaviour is that ValidationFailed is defined like this: error ValidationFailed(bytes memory);

So when expectReverts gets passed abi.encodeWithSelector(InvalidNonce.selector) the logs works fine, but when passed InvalidNonce.selector it causes it to log ValidationFailed(0x).

[tushar994]

so the issue here is the error log that is emitted in this particular case? What should the expected error log be?

[norswap]

so the issue here is the error log that is emitted in this particular case? What should the expected error log be?

Instead of showing:

[FAIL: Error != expected error: ValidationFailed(0x756688fe) != ValidationFailed(0x)]

It should show:

[FAIL: Error != expected error: ValidationFailed(0x756688fe) != 0xabcdef...]

(where 0xabcdef... is the raw revert data)

This is what I mean here:

I would suggest it's more useful to display the entire revert data in this case, if it can't be parsed in its entirety

The issue is when showing ValidationFailed(0x) it seems to say that it successfully parsed the error, with an empty bytes string as the argument. In reality, it failed to parse the error, but because it picked up the selector, it shows this, ignoring the part of the revert data that it couldn't parse.

[grandizzy]

@norswap I could be missing something, but the error ValidationFailed(bytes); error is actually matched and decoded when you abi.encodeWithSelector(ValidationFailed.selector, InvalidNonce.selector) so don't think we can restrict such, @DaniPopes could you please chime in? Thank you!

[DaniPopes]

This happens because the incorrect encoding (selector, selector, 0xe17594de756688fe00000000000000000000000000000000000000000000000000000000) gets decoded successfully with empty bytes; the only way we can improve this is enable validation which would fail decoding and fallback to custom error 0xe17594de: 756688fe00000000000000000000000000000000000000000000000000000000