|
1 | 1 | - contract states are commutative monoids, they can be "merged" in any order to arrive at the same result. This may reduce some potential race conditions.
|
2 | 2 |
|
3 |
| -## Transport Layer Key Management Issues (2025-01-06) |
| 3 | +## Important Testing Notes |
4 | 4 |
|
5 |
| -### Problem |
6 |
| -Integration test `test_put_contract` was failing with "Failed to decrypt packet" errors after v0.1.5 deployment to production. The same decryption failures were affecting the River app in production. |
| 5 | +### Always Use Network Mode for Testing |
| 6 | +- **NEVER use local mode for testing** - it uses very different code paths |
| 7 | +- Local mode bypasses critical networking components that need to be tested |
| 8 | +- Always test with `freenet network` to ensure realistic behavior |
7 | 9 |
|
8 |
| -### Root Cause |
9 |
| -The transport layer was incorrectly handling symmetric key establishment for gateway connections: |
| 10 | +## Quick Reference - Essential Commands |
10 | 11 |
|
11 |
| -1. **Gateway connection key misuse**: Gateway was using different keys for inbound/outbound when it should use the same client key for both directions |
12 |
| -2. **Client ACK encryption error**: Client was encrypting its final ACK response with the gateway's key instead of its own key |
13 |
| -3. **Packet routing overflow**: When existing connection channels became full, packets were misrouted to new gateway connection handlers instead of waiting |
| 12 | +### River Development |
| 13 | +```bash |
| 14 | +# Publish River (use this, not custom scripts) |
| 15 | +cd ~/code/freenet/river && RUST_MIN_STACK=16777216 cargo make publish-river-debug |
14 | 16 |
|
15 |
| -### Key Protocol Rules |
16 |
| -- **Gateway connections**: Use the same symmetric key (client's key) for both inbound and outbound communication |
17 |
| -- **Peer-to-peer connections**: Use different symmetric keys for each direction (each peer's own inbound key) |
18 |
| -- **Connection establishment**: Only initial gateway connections and explicit connect operations should create new connections |
19 |
| -- **PUT/GET/SUBSCRIBE/UPDATE operations**: Should only use existing active connections, never create new ones |
20 |
| - |
21 |
| -### Fixes Applied |
22 |
| - |
23 |
| -#### 1. Gateway Connection Key Fix (`crates/core/src/transport/connection_handler.rs:578-584`) |
24 |
| -```rust |
25 |
| -// For gateway connections, use the same key for both directions |
26 |
| -let inbound_key = outbound_key.clone(); |
27 |
| -let outbound_ack_packet = SymmetricMessage::ack_ok( |
28 |
| - &outbound_key, |
29 |
| - outbound_key_bytes.try_into().unwrap(), |
30 |
| - remote_addr, |
31 |
| -)?; |
32 |
| -``` |
33 |
| - |
34 |
| -#### 2. Client ACK Response Fix (`crates/core/src/transport/connection_handler.rs:798-811`) |
35 |
| -```rust |
36 |
| -// Use our own key to encrypt the ACK response (same key for both directions with gateway) |
37 |
| -outbound_packets |
38 |
| - .send(( |
39 |
| - remote_addr, |
40 |
| - SymmetricMessage::ack_ok( |
41 |
| - &inbound_sym_key, // Use our own key, not the gateway's |
42 |
| - inbound_sym_key_bytes, |
43 |
| - remote_addr, |
44 |
| - )? |
45 |
| - .prepared_send(), |
46 |
| - )) |
47 |
| - .await |
48 |
| - .map_err(|_| TransportError::ChannelClosed)?; |
49 |
| -``` |
50 |
| - |
51 |
| -#### 3. Packet Sending Consistency (`crates/core/src/transport/connection_handler.rs:740-747`) |
52 |
| -```rust |
53 |
| -let packet_to_send = our_inbound.prepared_send(); |
54 |
| -outbound_packets |
55 |
| - .send((remote_addr, packet_to_send.clone())) |
56 |
| - .await |
57 |
| - .map_err(|_| TransportError::ChannelClosed)?; |
58 |
| -sent_tracker |
59 |
| - .report_sent_packet(SymmetricMessage::FIRST_PACKET_ID, packet_to_send); |
| 17 | +# Verify River build time (CRITICAL - only way to confirm new version is served) |
| 18 | +curl -s http://127.0.0.1:50509/v1/contract/web/BcfxyjCH4snaknrBoCiqhYc9UFvmiJvhsp5d4L5DuvRa/ | grep -o 'Built: [^<]*' | head -1 |
60 | 19 | ```
|
61 | 20 |
|
62 |
| -### Testing |
63 |
| -- Created specialized transport tests in `crates/core/src/transport/test_gateway_handshake.rs` |
64 |
| -- `test_gateway_handshake_symmetric_key_usage()`: Verifies gateway connections use same key for both directions |
65 |
| -- `test_peer_to_peer_different_keys()`: Verifies peer-to-peer connections use different keys |
66 |
| -- Both specialized tests pass, confirming the transport layer fixes work correctly |
| 21 | +### Freenet Management |
| 22 | +```bash |
| 23 | +# Start Freenet |
| 24 | +./target/release/freenet network > freenet-debug.log 2>&1 & |
67 | 25 |
|
68 |
| -### Root Cause Analysis Complete |
| 26 | +# Check status |
| 27 | +ps aux | grep freenet | grep -v grep | grep -v tail | grep -v journalctl |
69 | 28 |
|
70 |
| -#### PUT Operation Connection Creation Issue |
71 |
| -**Location**: `crates/core/src/node/network_bridge/p2p_protoc.rs:242-291` |
72 |
| - |
73 |
| -**Problem**: PUT/GET/SUBSCRIBE/UPDATE operations create new connections when no existing connection is found, violating the protocol rule that these operations should only use existing active connections. |
74 |
| - |
75 |
| -**Behavior**: When `NetworkBridge.send()` is called and no existing connection exists: |
76 |
| -1. System logs warning: "No existing outbound connection, establishing connection first" |
77 |
| -2. Creates new connection via `NodeEvent::ConnectPeer` |
78 |
| -3. Waits up to 5 seconds for connection establishment |
79 |
| -4. Attempts to send message on newly created connection |
80 |
| - |
81 |
| -**Channel Overflow Root Cause**: Channels fill up due to throughput mismatch: |
82 |
| -- **Fast UDP ingress**: Socket receives packets quickly |
83 |
| -- **Slow application processing**: `peer_connection_listener` processes one message at a time sequentially |
84 |
| -- **Limited buffering**: 100-packet channel buffer insufficient for high-throughput scenarios |
85 |
| -- **No flow control**: System creates new connections instead of implementing proper backpressure |
86 |
| - |
87 |
| -**Cascade Effect**: Channel overflow → packet misrouting → wrong connection handlers → decryption failures → new connection creation |
88 |
| - |
89 |
| -#### Required Fix |
90 |
| -The network bridge should fail gracefully or retry with existing connections instead of creating new ones for PUT/GET/SUBSCRIBE/UPDATE operations. Only initial gateway connections and explicit CONNECT operations should establish new connections. |
| 29 | +# Monitor logs |
| 30 | +tail -f freenet-debug.log |
| 31 | +``` |
91 | 32 |
|
92 |
| -### Next Steps |
93 |
| -1. Modify PUT/GET operation handling to use only existing connections |
94 |
| -2. Implement proper backpressure handling for full channels instead of creating new connections |
95 |
| -3. Test that integration test `test_put_contract` passes after the fix |
| 33 | +## Detailed Documentation Files |
| 34 | + |
| 35 | +### Current Active Debugging |
| 36 | +- **Directory**: `freenet-invitation-bug.local/` (consolidated debugging) |
| 37 | + - `README.md` - Overview and quick commands |
| 38 | + - `river-notes/` - River-specific debugging documentation |
| 39 | + - `contract-test/` - Minimal Rust test to reproduce PUT/GET issue |
| 40 | + |
| 41 | +### River Invitation Bug (2025-01-18) |
| 42 | +- **Status**: CONFIRMED - Contract operations hang on live network, work in integration tests |
| 43 | +- **Root Cause**: Freenet node receives WebSocket requests but never responds |
| 44 | +- **Test Directory**: `freenet-invitation-bug.local/live-network-test/` |
| 45 | +- **Confirmed Findings**: |
| 46 | + - River correctly sends PUT/GET requests via WebSocket |
| 47 | + - Raw WebSocket test: Receives binary error response from server |
| 48 | + - freenet-stdlib test: GET request sent but never receives response (2min timeout) |
| 49 | + - Integration test `test_put_contract` passes when run in isolation |
| 50 | + - Issue affects both PUT and GET operations |
| 51 | +- **Current Investigation**: Systematically debugging why Freenet node doesn't respond to contract operations |
| 52 | +- **See**: `freenet-invitation-bug.local/river-notes/invitation-bug-analysis-update.md` |
| 53 | + |
| 54 | +### Historical Analysis (Reference Only) |
| 55 | +- **Transport Layer Issues**: See lines 3-145 in previous version of this file (archived) |
| 56 | +- **River Testing Procedures**: See lines 97-145 in previous version of this file (archived) |
| 57 | + |
| 58 | +### CI Tools |
| 59 | +- **GitHub CI Monitoring**: `~/code/agent.scripts/wait-for-ci.sh [PR_NUMBER]` |
| 60 | + |
| 61 | +### Testing Tools |
| 62 | +- **Puppeteer Testing Guide**: `puppeteer-testing-guide.local.md` - Essential patterns for testing Dioxus apps with MCP Puppeteer tools |
| 63 | + |
| 64 | +## Key Code Locations |
| 65 | +- **River Room Creation**: `/home/ian/code/freenet/river/ui/src/components/room_list/create_room_modal.rs` |
| 66 | +- **River Room Synchronizer**: `/home/ian/code/freenet/river/ui/src/components/app/freenet_api/room_synchronizer.rs` |
| 67 | +- **River Room Data**: `/home/ian/code/freenet/river/ui/src/room_data.rs` |
| 68 | + |
| 69 | +## Organization Rules |
| 70 | +1. **Check this file first** for command reference and active debugging directories |
| 71 | +2. **Use standard commands** instead of creating custom scripts |
| 72 | +3. **Verify River build timestamps** after publishing |
| 73 | +4. **Create timestamped .local directories** for complex debugging sessions |
| 74 | +5. **Update this index** when adding new debugging directories or tools |
0 commit comments