test(replication-chaos): treat the BotoCoreError family as transient, not fatal

The S2 SIGKILL scenario flaked: get_body() only caught (ClientError,
EndpointConnectionError), but a server that is down / being SIGKILLed / mid-restart
also raises other BotoCoreError subclasses (ResponseStreamingError, ConnectionError,
ReadTimeoutError) — boto3 has retries.max_attempts=1, so a single transient stream
hiccup during the convergence poll propagated as an unhandled exception and failed
the run. Catch the BotoCoreError base instead; converged() still counts real
convergence, so this makes the poll resilient without masking an actual failure.
Not a regression from the build-once CI change (it passed on the prior commit with
the same binary); a pre-existing timing flake.

Author: Cairn Build

conformance/replication_chaos.py

@@ -18,7 +18,7 @@
 import os, signal, subprocess, sys, time
 import boto3
 from botocore.config import Config
-from botocore.exceptions import ClientError, EndpointConnectionError
+from botocore.exceptions import BotoCoreError, ClientError
 
 BIN = os.environ.get("BIN", "target/debug/cairn")
 ROOT = os.environ["DATA"]
@@ -115,7 +115,13 @@ def setup_replication(src):
 def get_body(cl, key):
     try:
         return cl.get_object(Bucket=BUCKET, Key=key)["Body"].read()
-    except (ClientError, EndpointConnectionError):
+    except ClientError:
+        return None
+    except BotoCoreError:
+        # A server that is down, being SIGKILLed, or mid-restart raises the BotoCoreError family
+        # (ConnectionError, ResponseStreamingError, ReadTimeoutError, EndpointConnectionError). In a
+        # fault-injection test that is expected: treat it as "not available yet" so the convergence
+        # poll keeps trying instead of crashing the run.
         return None
 
 def converged(tgt, expected, timeout=90):