Skip to content

Conversation

@apoelstra
Copy link
Collaborator

  • changes var_len_buf_from_slice to interpret its n such that it's reading the type (2^8)^<2^n rather than (2^8)^<2^(n+1); subtract 1 from all instances of n except the loop counter. (Previously we were looping the wrong number of times, which this fixes.)
  • calls var_len_buf_from_slice with n = 6, which matches the type signature for Ctx8. Before we were calling with n = 8 which was just wrong
  • Changes SimplicityCtx8::buffer to have type [u8; 64] instead of the wrong [u8; 512]
  • Changes the nonsense "v.len() >= 1 << n" comparison to one that checks the bits of v.len()
  • Corrects various off-by-one errors

See #318 for more information.

Fixes #318

Copy link
Collaborator Author

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On 0d4c323 successfully ran local tests

@roconnor-blockstream
Copy link
Contributor

Why the reduction on the limit of n down from 16 to 8?

@apoelstra
Copy link
Collaborator Author

@roconnor-blockstream because I replaced n+1 with n.

@apoelstra
Copy link
Collaborator Author

If the 8 is wrong now, the 16 was wrong before. (Which now that I think about it, is likely to be the case.)

@roconnor-blockstream
Copy link
Contributor

You can verify that if 0 <= n < 16 was the restriction for when the rule was Read bytes from a Simplicity buffer of type (TWO^8)^<2^(n+1) as Value., then performing then the substitution replacing n+1 with n’ yields the following math

0 <= n < 16

0 <= (n + 1) - 1< 16

0 <= n’ - 1< 16

1 <= n’ < 17

1 <= n’ <= 16.

Indeed n’ <= 16 is the assertion I think you should be using.

Also the comment

Cannot represent >= 2**16 bytes […] as simplicity consensus rule.

is incorrect; this is not a consensus rule. This comes from

https://github.com/BlockstreamResearch/simplicity/blob/ba440a0999da8b06f2be4a0f337f52a3c7f355ba/C/frame.c#L92-L116

where simplicity_read_buffer8 only handles up to 16. However this isn't a consensus rule. It is simply what I've implemented since that more than enough to implement our set of jets.

@apoelstra apoelstra force-pushed the 2025-11/benchmark-ctx8 branch from 0d4c323 to 5021eca Compare November 20, 2025 19:49
@apoelstra
Copy link
Collaborator Author

Updated the comment and the assertion.

/// Cannot represent >= 2**16 bytes 0 <= n < 16 as simplicity consensus rule.
/// Cannot represent >= 2**16 bytes 0 <= n < 16; higher values are unused
/// by the current set of jets and would cause trouble on 16-bit machines
/// that we'd rather not think about.
Copy link
Contributor

@roconnor-blockstream roconnor-blockstream Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's just remove this comment entirely or say "n < 16" is a precondition.

P.S. I recommend you adjust the code so that "n <= 16" is the precondition.
Really the only reason not to go to as far as "n <= 32" is that I don't know if 1 << 32 fits in a usize.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will on a 64-bit machine but not on a 32-bit one. (In theory usize could even be 16 bits but probably our code is broken in many cases on such machines.)

@apoelstra apoelstra force-pushed the 2025-11/benchmark-ctx8 branch from 5021eca to 5a46145 Compare November 20, 2025 21:40
@apoelstra
Copy link
Collaborator Author

How about this?

/// # Panics:
///
/// Panics if the length of the slice is >= 2^(n + 1) bytes
/// Panics if the length of the slice is >= 2^n bytes, or if n >= 16.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

panics if n > 16.

n -= 1;
}
assert_eq!(iter.next(), None);
Ok(res.unwrap_or(Value::unit()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I'm going to actually recommend that you have precondition that 1 <= n, because 𝟙 is not actually a buffer type, and if people are passing 0 here it is probably a big mistake.

If you strongly disagree then you can keep the code as it is.

Copy link
Contributor

@roconnor-blockstream roconnor-blockstream Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually I take that back. Arguably 𝟙 is a buffer type for a degenerate empty buffer type. So you can leave the code as is.

// Simplicity consensus rule for n < 16 while reading buffers.
assert!(n < 16);
assert!(v.len() < (1 << (n + 1)));
// This precondition can probably be removed or relaxed, though
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wasn't immediately obvious what this code was doing, so I had to read the doc comment, write it myself, then compare my code against this. It might've been clearer if while n > 0 was replaced with for i in (0..n).rev() (since the loop is just counting down to zero), plus that means you can get rid of (n - 1) (subtraction on usize always makes me do a double-take) and it changes two_two_n(n + 2) to two_two_n(i + 3) (where + 3 is obviously just a bytes-to-bits conversion).

That's all nit-picking though. Otherwise LGTM.

@apoelstra apoelstra force-pushed the 2025-11/benchmark-ctx8 branch 2 times, most recently from 6fa6bcd to 690d5f2 Compare November 21, 2025 20:49
@apoelstra
Copy link
Collaborator Author

Took @canndrew's suggestions.

///
/// Panics if the length of the slice is >= 2^(n + 1) bytes
/// Panics if the length of the slice is >= 2^n bytes or if n > 16.
pub fn var_len_buf_from_slice(v: &[u8], mut n: usize) -> Result<Value, EarlyEndOfStreamError> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

n is no longer mutable?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, oops, I test-compiled the wrong crate. You're right (and the compiler warns about this).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whew. I was confused why this wasn't noticed by the compiler.

* changes `var_len_buf_from_slice` to interpret its `n` such that it's
  reading the type (2^8)^<2^n rather than (2^8)^<2^(n+1); subtract 1
  from all instances of `n` except the loop counter. (Previously we
  were looping the wrong number of times, which this fixes.)
* calls `var_len_buf_from_slice` with n = 6, which matches the type
  signature for Ctx8. Before we were calling with n = 8 which was just
  wrong
* Changes `SimplicityCtx8::buffer` to have type `[u8; 64]` instead of
  the wrong `[u8; 512]`
* Changes the nonsense "v.len() >= 1 << n" comparison to one that checks
  the bits of v.len()
* Corrects various off-by-one errors

See BlockstreamResearch#318 for more information.
@apoelstra apoelstra force-pushed the 2025-11/benchmark-ctx8 branch from 690d5f2 to 934a787 Compare November 21, 2025 21:19
Copy link
Contributor

@roconnor-blockstream roconnor-blockstream left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Collaborator Author

@apoelstra apoelstra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On 934a787 successfully ran local tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

var_len_buf_from_slice in benchmarks is wrong

3 participants