Improved readability, added pull-based and push-based operation configurability, added safety strategy

Author: ufarooqstatus

pubsub/gossipsub/gossipsub-v1.4.md

@@ -1,8 +1,8 @@
-# gossipsub v1.4: Message preamble to limit duplicate transmissions of large messages
+# gossipsub v1.4: PREAMBLE and IMRECEIVING to limit duplicate transmissions of large messages
 
 | Lifecycle Stage | Maturity                  | Status | Latest Revision |
 |-----------------|---------------------------|--------|-----------------|
-| 1A              | Working Draft             | Active | r0, 2024-12-17  |
+| 1A              | Working Draft             | Active | r1, 2025-05-01  |
 
 Authors: [@ufarooqstatus], [@kaiserd]
 
@@ -31,18 +31,19 @@ Under the current arrangement, if IHAVE announcements are received for a message
 the receiver may generate multiple IWANT requests, triggering unnecessary retransmissions of the same message. 
 Higher message reception time also increases the probability of simultaneously receiving the same message from many peers. 
 
-Prepending a preamble while sending a large message can allow receivers 
-to immediately gain insights into the IDs and lengths of the messages they are receiving.  
-This can allow receivers to defer IWANT requests for messages they are currently receiving, 
+Sending a short control message, called PREAMBLE, before every large message transmission 
+can allow receivers to immediately gain insights into the IDs and lengths of the messages they are receiving. 
+This allows receivers to defer IWANT requests for messages they are already receiving,  
 thereby reducing the number of unnecessary IWANT requests.
 
 At the same time, receivers may inform their mesh members about ongoing message reception.
-For this purpose, a new control message (IMReceiving) is introduced, 
-indicating that the peer sending IMReceiving message is currently receiving messages identified by their message IDs, 
+For this purpose, a new control message (IMRECEIVING) is introduced, 
+indicating that the peer sending IMRECEIVING is currently receiving a message identified by the announced message ID, 
 and sending duplicates might be unnecessary.
 
-On receiving an IMReceiving message, a peer should defer sending of messages identified by the message IDs in IMReceiving message.
-This can lead to a significant reduction in bandwidth utilization and message latency.  
+On receiving an IMRECEIVING message, a peer should refrain from sending the message identified by the announced message ID.
+This can lead to a significant reduction in bandwidth utilization and message latency.
+The [Safety Strategy](#safety-strategy) below safeguards against malicious behavior.  
 
 # Specification
 
@@ -57,81 +58,113 @@ prefix, by default this is `meshsub` giving the default protocol id:
 
 This section lists the configuration parameters that clients need to agree on to avoid peer penalizations.
 
-| Parameter                | Description                                                      | Reasonable Default |
-|--------------------------|------------------------------------------------------------------|--------------|
-| `peer_preamble_announcements` | The maximum number of preamble announcements for unfinished transfers per peer | 1???  |
-| `mesh_preamble_announcements` | The maximum number of preamble announcements to accept for unfinished transfers per topic per heartbeat interval | 3???  |
-| `max_iwant_requests` | The maximum number of simultaneous IWANT requests for a message | 1???  |
-| `preamble_threshold` | The smallest message size to use message preamble | 200KB???  |
+| Parameter                | Description                                                      |
+|--------------------------|------------------------------------------------------------------|
+| `peer_preamble_announcements` | The maximum number of PREAMBLE announcements for unfinished transfers per peer |
+| `max_iwant_requests` | The maximum number of simultaneous IWANT requests for a message |
+| `preamble_threshold` | The minimum message size (in bytes) required to enable the use of a message PREAMBLE |
+| `fallback_mode` | Message fetching strategy (pull or push) to use when a peer fails to deliver the message after sending a PREAMBLE. |
 
 
 
-## Message Preamble
+## Message PREAMBLE
 
 ### Basic scenario
 
 When a peer starts relaying a message that exceeds the preamble_threshold size, 
-it may include a control message (called message preamble) at the start of the message.
+it should transmit a preceding control message called PREAMBLE.
 
-The purpose of the preamble is to allow receivers to instantly learn about the incoming message. 
-The preamble must include the message ID and length, 
+PREAMBLE serves as a commitment from the sender, 
+indicating that the promised data message will follow without delay. 
+A PREAMBLE must include the message ID and length, 
 providing receivers with immediate access to critical information about the incoming message. 
-The receiver must immediately process a message preamble without waiting to download the entire message.
+The receiver must immediately process a PREAMBLE without waiting to download the entire message.
 
-On receiving a preamble that advertises a message ID not present in the seen cache, 
+On receiving a PREAMBLE that advertises a message ID not present in the seen cache, 
 a receiver should add that message ID to the ongoing_receives list.
-The ongoing_receives list is crucial in limiting IWANT requests and simultaneous reception of the same message from mesh peers.
- 
-The receiver may use message length to leniently estimate message download time.
-If the message is successfully downloaded before the estimated download time has elapsed, 
+The ongoing_receives list is crucial in limiting IWANT requests and simultaneous receptions of the same message from mesh peers.
+
+The receiver may use message length to conservatively estimate message download duration.
+If the message is successfully downloaded before the estimated download duration has elapsed, 
 the message ID is removed from the ongoing_receives list and added to the seen cache. 
 
 If the download takes longer than the estimated download time, 
-the sender may be penalized through P4 to discourage peers from intentionally delaying message transmission.
+the sender may be penalized through a behavioral penalty to discourage peers from intentionally delaying message transmission. 
+See also [Safety Strategy](#safety-strategy) below.
 
-The message preamble is considered _optional_ for both the sender and the receiver. 
-This means that the sender may choose not to prepend the preamble, 
-and the receiver may also opt to ignore it.
-Adding a message preamble may increase control overhead for small messages. 
+PREAMBLE is considered _optional_ for both the sender and the receiver. 
+This means that the sender can choose not to send the PREAMBLE, 
+and the receiver can also opt to ignore it.
+Adding a PREAMBLE may increase control overhead for small messages. 
 Therefore, it is preferable to use it only for messages that exceed the preamble_threshold.
 
 
 ### Limiting IWANT Requests
 
 When a peer receives an IHAVE announcement for a message ID not present in the seen cache, 
-the peer must also check the  ongoing_receives list before making an IWANT request. 
+the peer must also check the ongoing_receives list before making an IWANT request. 
 
 If the message ID is found in the ongoing_receives list, 
 the peer should postpone sending the IWANT request for a defer_interval. 
-The defer_interval may be based on the message download time.
+The defer_interval may be based on the estimated message download duration.
 
 If the message download completes before the defer_interval expires, 
 the IWANT request for that message will not be generated. 
 However, if the defer_interval elapses and the download has not completed, 
 the peer can proceed to make the IWANT request for the missing message. 
 
-The total number of outstanding IWANT requests for a single message should not exceed max_iwant_requests.
-Every peer must respond to the received IWANT requests. 
-Failing to do so may result in behavioural penalty through P7. 
+The total number of outstanding IWANT requests for a single message must not exceed max_iwant_requests.
+Every peer must respond to incoming IWANT requests as long as the number of responses remains within the limits 
+defined by [gossipsub v1.1](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/gossipsub-v1.1.md).
+Failing to do so should result in a behavioral penalty. 
 This will discourage peers from intentionally not replying to IWANT requests.
 
-### IMReceiving Message
+### IMRECEIVING Message
 
-The IMReceiving message serves a distinct purpose compared to the IDONTWANT message. 
+The IMRECEIVING message serves a distinct purpose compared to the IDONTWANT message. 
 An IDONTWANT can only be transmitted after receiving the entire message. 
-In contrast, an IMReceiving should be transmitted immediately after receiving a preamble indicating a large message transfer. 
-The IMReceiving message requests peers to refrain from resending a large message that is already being received.
+In contrast, an IMRECEIVING should be transmitted immediately after receiving a PREAMBLE, indicating an ongoing large message reception.
+The IMRECEIVING message requests peers in the full message mesh to refrain from resending a large message that is already being received.
 
-When a peer receives a message preamble indicating a message ID that is not present in the seen cache, 
-it should send an IMReceiving message to the nodes in its full message mesh. 
+When a peer receives a PREAMBLE indicating a message ID that is not present in the seen cache and ongoing_receives list, 
+it should send IMRECEIVING messages to the nodes in its full message mesh. 
 On receiving an IMReceiving from a peer, 
-a node should postpone sending the messages indicated by the IMReceiving to that peer. 
-The defer_interval can be leniently estimated based on the message length. 
-
-An IDONTWANT from the peer will indicate successful reception of the message. 
-If an IDONTWANT for the deferred message is not received and the defer_interval has elapsed, 
-the node may proceed to transmit the message.
-
+a node should refrain from sending the message indicated by the IMRECEIVING to that peer.  
+
+A corresponding IDONTWANT from the peer that issued IMRECEIVING will indicate successful reception of the message. 
+Otherwise, the peers in the full message may proceed with a push-based or a pull-based operation depending upon the selected fallback_mode.
+See [Safety Strategy](#safety-strategy) below.
+
+
+## Safety Strategy
+
+A malicious peer can attempt to exploit this approach by sending a PREAMBLE but 
+never completing (or deliberately delaying) the promised message transfer or 
+by misrepresenting the message size, 
+potentially hindering the message propagation across the network.
+A simple defense mechanism can remedy this problem.
+
+- A peer must accept and process a PREAMBLE only from mesh members and only once for any message ID.
+- A peer must limit the maximum number of PREAMBLE announcements for unfinished message transfers to peer_preamble_announcements.
+- A peer must extract the message ID and length from PREAMBLE to report in IMRECEIVING announcements.
+- A peer must ignore a received IMRECEIVING message if the reported length field does not correspond to the announced message ID.
+- If a peer identifies a misrepresented length field in a received PREAMBLE, it must penalize the sender for an invalid message.
+
+Upon receiving a PREAMBLE, a peer uses the length field to conservatively estimate message transfer duration, 
+ensuring sufficient time allocation for message reception. 
+If a message transfer is not complete during the estimated duration, 
+the sender is penalized through a behavioral penalty, 
+and message retrieval is performed based on a pull-based or a push-based strategy, 
+as selected by the fallback_mode.
+
+In a pull-based strategy, the peer receiving the message uses an IWANT request to fetch the message. 
+The peer can infer message availability at mesh members through received IDONTWANT announcements.
+In a push-based model, mesh members proactively send the message 
+if they do not receive a corresponding IDONTWANT announcement within the estimated message transfer duration. 
+Mesh members can derive a conservative estimate for duration based on the length field.
+
+Negative scoring helps prune non-conforming peers, 
+whereas the fallback strategy helps recover from incomplete message transfers.
 
 ## Protobuf Extension