Require FixedLengthReadable for structs that always drain reader
#3640
Conversation
|
👋 Thanks for assigning @TheBlueMatt as a reviewer! |
In general, the codebase currently tends to use the Readable trait with two different semantics -- some objects know when to stop reading because they have a length prefix and some automatically read to the end of the reader. This differing usage can lead to bugs. For objects that read-to-end, it would be much safer if the compiler enforced that they could only be read from a length-limiting reader. We already have a LengthReadable trait that requires the underlying reader to provide the length, which is similar to what we want. So rather than adding an additional trait, here we refactor LengthReadable to require a fixed length reader. Upcoming commits will switch read-to-end structs that are currently read with Readable to use LengthReadable instead.
valentinewallace
force-pushed
the
2025-03-fixed-len-readable
branch
from
6f66ebb to
47916c1
Compare
|
Looking for concept ACKs before fixing fuzz! |
|
Also do we also want to do the same for |
valentinewallace
force-pushed
the
2025-03-fixed-len-readable
branch
from
47916c1 to
792fcf0
Compare
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3640 +/- ##
==========================================
+ Coverage 89.20% 90.02% +0.82%
==========================================
Files 152 155 +3
Lines 118661 126135 +7474
Branches 118661 126135 +7474
==========================================
+ Hits 105849 113559 +7710
+ Misses 10227 10101 -126
+ Partials 2585 2475 -110 ☔ View full report in Codecov by Sentry. |
|
👋 The first review has been submitted! Do you think this PR is ready for a second reviewer? If so, click here to assign a second reviewer. |
|
nit: might point out in the last commit message that the remaining structs are the ones not covered by impl_writeable_msg |
valentinewallace
force-pushed
the
2025-03-fixed-len-readable
branch
from
792fcf0 to
36698bb
Compare
See previous commit. When deserializing objects via this macro, there is no length prefix so the deser code will read the provided reader until it runs out of bytes. Readable is not an appropriate trait for this situation because it should only be used for structs that are prefixed with a length and know when to stop reading. LengthReadable instead requires that the caller supply only the bytes that are reserved for this struct.
Easier to review the previous commit this way.
When deserializing these structs, they will consume the reader until it is out of bytes. Therefore, it's safer if they are only provided with readers that limit how much of the byte stream will be read. These remaining structs are largely the ones not covered when we transitioned impl_writeable_msg to use LengthReadable in the previous commit.
valentinewallace
force-pushed
the
2025-03-fixed-len-readable
branch
from
36698bb to
b95f2ce
Compare
|
nit: squash the formatting commit in between getting the second ACK and merging. |
|
🔔 1st Reminder Hey @TheBlueMatt! This PR has been waiting for your review. |
| /// Reads a `Self` in from the given [`LengthRead`]. | ||
| fn read_from_fixed_length_buffer<R: LengthRead>(reader: &mut R) -> Result<Self, DecodeError>; | ||
| /// Reads a `Self` in from the given [`FixedLengthReader`]. | ||
| fn read_from_fixed_length_buffer<'a, R: Read>( |
There was a problem hiding this comment.
We should either use a trait here, rather than forcing a FixedLengthReader, or we should remove LengthRead entirely (which looks ~unused now?). It'd be kinda nice to use a trait, I think, but maybe you thought it wouldn't be strict enough? I guess we could kinda just rename LengthRead as LengthLimiedRead and describe the semantics?
| @@ -1101,7 +1101,9 @@ pub fn do_test<Out: Output>(data: &[u8], underlying_out: Out, anchors: bool) { | |||
| // update_fail_htlc as we do when we reject a payment. | |||
| let mut msg_ser = update_add.encode(); | |||
| msg_ser[1000] ^= 0xff; | |||
| let new_msg = UpdateAddHTLC::read(&mut Cursor::new(&msg_ser)).unwrap(); | |||
| let mut cursor = Cursor::new(&msg_ser); | |||
There was a problem hiding this comment.
nit: we don't actually need a Cursor, we can just &mut &msg_ser[..]. Same elsewhere in this commit. Would be nice to slowly phase out Cursor as we touch it.
| @@ -1033,8 +1034,10 @@ impl OfferContents { | |||
| } | |||
| } | |||
|
|
|||
| impl Readable for Offer { | |||
| fn read<R: io::Read>(reader: &mut R) -> Result<Self, DecodeError> { | |||
| impl LengthReadable for Offer { | |||
There was a problem hiding this comment.
Don't we need to do this for all the other bolt 12 structs too?
Closes #3292