Auto-whittaker optimizations: Functional Decoupling

[davidchuyaya]

Implement functional decoupling, splitting a component C into C1 -> C2, where C1 is independent of C2

Protocol to test optimizations on:

• Implement Paxos in HF+
• Collect useful statistics from Paxos (throughput, latency) Telemetry: local and global #928, make benchmarking repeatable
• Debug performance issues

Detecting C2:

• Walk up from network edge up directed graph of operators
• Find cycles in HF+ graph (strong components algorithm), discuss with @MingweiSamuel
• Detect pure operators
• Identify all possible points of functional decoupling

Applying functional decoupling:

• Find all EDB relations that need to be duplicated for functional decoupling
• Find cut of graph for functional decoupling, create new component, create rewrite that changes dataflow edges to networking edges
• Test on Paxos, debug performance issues
• Apply intelligently based on performance statistics