Flaky test: TestAccCloudResourceImage_full

[lgfa29]

The TestAccCloudResourceImage_full has been observed to fail at times with the following error:

=== FAIL: internal/provider TestAccCloudResourceImage_full (181.81s)
    resource_image_test.go:94: Step 1/2 error: Error running apply: exit status 1
        
        Error: Error creating snapshot
        
          with oxide_snapshot.acc-support-773032af-aac2-4ec4-b6ba-25c03701bf9c,
          on terraform_plugin_test.tf line 29, in resource "oxide_snapshot" "acc-support-773032af-aac2-4ec4-b6ba-25c03701bf9c":
          29:  resource "oxide_snapshot" "acc-support-773032af-aac2-4ec4-b6ba-25c03701bf9c" {
        
        API error: error sending request: Post
        "http://localhost:8080/v1/snapshots?project=1befb81b-31ba-48e6-bd64-4ffdd5812afd":
        context deadline exceeded
    panic.go:615: Error running post-test destroy, there may be dangling resources: exit status 1
        
        Error: Unable to delete disk:
        
        API error: DELETE
        http://localhost:8080/v1/disks/0e834800-ee6d-4a55-85a2-77dc20f45b65
        ----------- RESPONSE -----------
        Status: 400 InvalidRequest
        Message: disk cannot be deleted in state "maintenance"
        RequestID: 60a14a52-6b60-4461-8e9f-bd05d066ae4a
        ------- RESPONSE HEADERS -------
        X-Request-Id: [60a14a52-6b60-4461-8e9f-bd05d066ae4a]
        Content-Length: [156]
        Date: [Wed, 25 Feb 2026 08:03:28 GMT]
        Content-Type: [application/json]

Sample run: https://github.com/oxidecomputer/terraform-provider-oxide/actions/runs/22387128073/job/64801391010#step:15:247
Logs: logs-terraform-64801391010.zip

The main culprit is an an error in the simulated crucible that prevents the snapshot from completing. Nexus will then retry the request with an exponential backoff, but the request will eventually timeout (context deadline exceeded).

08:00:30.198Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): sending attach request for 0e834800-ee6d-4a55-85a2-77dc20f45b65 to [::1]:36361
    saga_id = 74bc4d3c-bf12-4a6f-8253-1c60e8826438
    saga_name = snapshot-create
08:00:30.266Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Succeeded(Null)
    node_id = 13
    saga_id = 74bc4d3c-bf12-4a6f-8253-1c60e8826438
08:00:30.274Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Started
    node_id = 14
    saga_id = 74bc4d3c-bf12-4a6f-8253-1c60e8826438
08:00:30.280Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): sending snapshot request with id dc83911d-d59f-418b-aa86-b96a5796d08f for disk 0e834800-ee6d-4a55-85a2-77dc20f45b65 to pantry endpoint http://[::1]:36361
    saga_id = 74bc4d3c-bf12-4a6f-8253-1c60e8826438
    saga_name = snapshot-create
08:00:30.339Z WARN e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saw transient communication error error sending request for url (http://[::1]:36361/crucible/pantry/0/volume/0e834800-ee6d-4a55-85a2-77dc20f45b65/snapshot), retrying...
    saga_id = 74bc4d3c-bf12-4a6f-8253-1c60e8826438
    saga_name = snapshot-create
08:00:30.340Z WARN e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): failed external call (ProgenitorError(Communication Error: error sending request for url (http://[::1]:36361/crucible/pantry/0/volume/0e834800-ee6d-4a55-85a2-77dc20f45b65/snapshot))), will retry in 224.637991ms
    saga_id = 74bc4d3c-bf12-4a6f-8253-1c60e8826438
    saga_name = snapshot-create

The second error is a consequence of the fist one. The Terraform test suite will try to delete the disk, but since the saga is still active, the disk is in maintenance mode.

Disk deletion saga

08:03:28.064Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): tracking newly created saga
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.064Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): preparing saga
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    saga_name = disk-delete
08:03:28.064Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saga create
    dag = {"end_node":14,"graph":{"edge_property":"directed","edges":[[0,1,null],[1,2,null],[2,3,null],[4,5,null],[4,6,null],[6,7,null],[5,7,null],[7,8,null],[8,9,null],[9,10,null],[10,11,null],[3,4,null],[11,12,null],[13,0,null],[12,14,null]],"node_holes":[],"nodes":[{"Action":{"action_name":"disk_delete.delete_disk_record","label":"DeleteDiskRecord","name":"deleted_disk"}},{"Action":{"action_name":"disk_delete.space_account","label":"SpaceAccount","name":"no_result1"}},{"Constant":{"name":"params_for_volume_delete_subsaga","value":{"serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"272b2939-5b35-4c46-8186-88424c3180d4","silo_user_id":"e87a45a2-d57b-40cb-8403-2e371c159e9b"}},"credential_id":"5f270637-17f0-47bf-8e27-9722c401485f","device_token_expiration":null},{"mapped_fleet_roles":{"admin":["admin"]}}]}},"volume_id":"72cdb49f-7794-451f-b0c5-655e7352ea88"}}},{"SubsagaStart":{"params_node_name":"params_for_volume_delete_subsaga","saga_name":"volume-delete"}},{"Action":{"action_name":"volume_delete.decrease_crucible_resource_count","label":"DecreaseCrucibleResourceCount","name":"crucible_resources_to_delete"}},{"Action":{"action_name":"volume_delete.delete_crucible_regions","label":"DeleteCrucibleRegions","name":"no_result_1"}},{"Action":{"action_name":"volume_delete.delete_crucible_running_snapshots","label":"DeleteCrucibleRunningSnapshots","name":"no_result_2"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshots","label":"DeleteCrucibleSnapshots","name":"no_result_3"}},{"Action":{"action_name":"volume_delete.delete_crucible_snapshot_records","label":"DeleteCrucibleSnapshotRecords","name":"no_result_4"}},{"Action":{"action_name":"volume_delete.find_freed_crucible_regions","label":"FindFreedCrucibleRegions","name":"freed_crucible_regions"}},{"Action":{"action_name":"volume_delete.delete_freed_crucible_regions","label":"DeleteFreedCrucibleRegions","name":"no_result_5"}},{"Action":{"action_name":"volume_delete.hard_delete_volume_record","label":"HardDeleteVolumeRecord","name":"volume_hard_deleted"}},{"SubsagaEnd":{"name":"volume_delete_subsaga_no_result"}},{"Start":{"params":{"disk":{"Crucible":{"disk":{"block_size":"Traditional","disk_type":"Crucible","identity":{"description":"a test disk","id":"0e834800-ee6d-4a55-85a2-77dc20f45b65","name":"acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50","time_created":"2026-02-25T08:00:27.136307Z","time_deleted":null,"time_modified":"2026-02-25T08:00:27.136307Z"},"project_id":"1befb81b-31ba-48e6-bd64-4ffdd5812afd","rcgen":1,"runtime_state":{"attach_instance_id":null,"disk_state":"maintenance","gen":3,"time_updated":"2026-02-25T08:00:29.842787Z"},"size":1073741824,"slot":null},"disk_type_crucible":{"create_image_id":null,"create_snapshot_id":null,"disk_id":"0e834800-ee6d-4a55-85a2-77dc20f45b65","pantry_address":null,"read_only":false,"volume_id":"72cdb49f-7794-451f-b0c5-655e7352ea88"}}},"project_id":"1befb81b-31ba-48e6-bd64-4ffdd5812afd","serialized_authn":{"kind":{"Authenticated":[{"actor":{"SiloUser":{"silo_id":"272b2939-5b35-4c46-8186-88424c3180d4","silo_user_id":"e87a45a2-d57b-40cb-8403-2e371c159e9b"}},"credential_id":"5f270637-17f0-47bf-8e27-9722c401485f","device_token_expiration":null},{"mapped_fleet_roles":{"admin":["admin"]}}]}}}}},"End"]},"saga_name":"disk-delete","start_node":13}
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    saga_name = disk-delete
    sec_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
08:03:28.064Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): creating saga
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    saga_name = disk-delete
08:03:28.120Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): starting saga
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    saga_name = disk-delete
08:03:28.120Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saga start
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    saga_name = disk-delete
    sec_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
08:03:28.120Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Started
    node_id = 13
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.122Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Succeeded(Null)
    node_id = 13
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.125Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Started
    node_id = 0
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.130Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("disk cannot be deleted in state \\"maintenance\\""), "internal_context": String("")}}} })
    node_id = 0
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.132Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): update for saga cached state
    new_state = Unwinding
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    sec_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
08:03:28.132Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): updating state
    new_state = unwinding
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.135Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = UndoStarted
    node_id = 13
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.137Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = UndoFinished
    node_id = 13
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.139Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): update for saga cached state
    new_state = Done
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    sec_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
08:03:28.139Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): updating state
    new_state = done
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.142Z WARN e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saga finished
    action_error_node_name = "deleted_disk"
    action_error_source = ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("disk cannot be deleted in state \\"maintenance\\""), "internal_context": String("")}}} }
    result = failure
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    saga_name = disk-delete
    sec_id = e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c
    undo_result = success
08:03:28.142Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): tracked saga has finished
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
08:03:28.143Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saga finished
    saga_id = 34763a14-11f3-40ea-adf1-5277faf61031
    saga_name = disk-delete
    saga_result = SagaResult { saga_id: 34763a14-11f3-40ea-adf1-5277faf61031, saga_log: SagaLog { saga_id: 34763a14-11f3-40ea-adf1-5277faf61031, unwinding: true, events: [N013 started, N013 succeeded, N000 started, N000 failed, N013 undo_started, N013 undo_finished], node_status: {0: Failed(ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("disk cannot be deleted in state \\"maintenance\\""), "internal_context": String("")}}} }), 13: UndoFinished} }, kind: Err(SagaResultErr { error_node_name: "deleted_disk", error_source: ActionFailed { source_error: Object {"InvalidRequest": Object {"message": Object {"external_message": String("disk cannot be deleted in state \\"maintenance\\""), "internal_context": String("")}}} }, undo_failure: None }) }

This is an unrecoverable error, so even re-running the test will not make it pass because the crucible pantry is not happy.

08:05:07.411Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (dropshot_external): request completed
    error_message_external = Internal Server Error
    error_message_internal = saga ACTION error at node "pantry_address": deserialize failed: unknown variant `failed to claim pantry client from pool: Backends exist, but none are online`, expected one of `ObjectNotFound`, `ObjectAlreadyExists`, `InvalidRequest`, `Unauthenticated`, `InvalidValue`, `Forbidden`, `InternalError`, `ServiceUnavailable`, `InsufficientCapacity`, `TypeVersionMismatch`, `Conflict`, `NotFound`, `Gone`
    latency_us = 30594998
    local_addr = 0.0.0.0:12220
    method = POST
    remote_addr = 172.18.0.3:41762
    req_id = 37502dbf-9a25-46ff-adf2-1740d026b086
    response_code = 500
    uri = /v1/snapshots?project=1befb81b-31ba-48e6-bd64-4ffdd5812afd

[lgfa29]

Crucible seems to get stuck when creating regions for the snapshot. These are the last messages in the logs about the regions creation:

08:00:27.468Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): Allocated regions for volume
    datasets_and_regions = [(CrucibleDataset { identity: CrucibleDatasetIdentity { id: 719df3de-6365-4f9b-9ac3-b4c8705b8b04 (dataset), time_created: 2026-02-25T07:57:22.054208Z, time_modified: 2026-02-25T07:57:22.054208Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 8a2174f0-5323-4b40-b3c7-a1503c2d0141 (zpool), ip: ::1, port: SqlU16(36659), size_used: 6710886400, no_provision: false }, Region { identity: RegionIdentity { id: 49f5741a-d3fe-455e-8935-4c0e4e29a17e, time_created: 2026-02-25T08:00:27.407827Z, time_modified: 2026-02-25T08:00:27.407827Z }, dataset_id: 719df3de-6365-4f9b-9ac3-b4c8705b8b04 (dataset), volume_id: 72cdb49f-7794-451f-b0c5-655e7352ea88 (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive }), (CrucibleDataset { identity: CrucibleDatasetIdentity { id: 9993ade3-b396-4d44-ad60-ae802c6dbe09 (dataset), time_created: 2026-02-25T07:57:21.185166Z, time_modified: 2026-02-25T07:57:21.185166Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 95885da3-b8a8-4ed2-96d7-e8410077a3ad (zpool), ip: ::1, port: SqlU16(38747), size_used: 6710886400, no_provision: false }, Region { identity: RegionIdentity { id: eb0ed45c-9135-40ee-80fa-a3196b65fcd1, time_created: 2026-02-25T08:00:27.407827Z, time_modified: 2026-02-25T08:00:27.407827Z }, dataset_id: 9993ade3-b396-4d44-ad60-ae802c6dbe09 (dataset), volume_id: 72cdb49f-7794-451f-b0c5-655e7352ea88 (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive }), (CrucibleDataset { identity: CrucibleDatasetIdentity { id: 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset), time_created: 2026-02-25T07:57:22.445518Z, time_modified: 2026-02-25T07:57:22.445518Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 08182027-59b4-4ec8-b2db-edb26c7ae933 (zpool), ip: ::1, port: SqlU16(40241), size_used: 6710886400, no_provision: false }, Region { identity: RegionIdentity { id: 50cf5783-8493-4a26-86b5-af5c359f51fc, time_created: 2026-02-25T08:00:27.407827Z, time_modified: 2026-02-25T08:00:27.407827Z }, dataset_id: 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset), volume_id: 72cdb49f-7794-451f-b0c5-655e7352ea88 (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive })]
    maybe_snapshot_id = None
    volume_id = 72cdb49f-7794-451f-b0c5-655e7352ea88
08:00:27.468Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Succeeded(Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("49f5741a-d3fe-455e-8935-4c0e4e29a17e"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb0ed45c-9135-40ee-80fa-a3196b65fcd1"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("50cf5783-8493-4a26-86b5-af5c359f51fc"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}]])
    node_id = 4
    saga_id = 1c2ab465-924d-4e70-b0a5-8da0c420a446
08:00:27.517Z INFO omicron-dev (omicron_sled_agent::sim::Server): created region 49f5741a-d3fe-455e-8935-4c0e4e29a17e
    end_port = 1100
    kind = storage
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    start_port = 1050
08:00:27.823Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saga finished
    saga_id = 1c2ab465-924d-4e70-b0a5-8da0c420a446
    saga_name = disk-create
    saga_result = SagaResult { saga_id: 1c2ab465-924d-4e70-b0a5-8da0c420a446, saga_log: SagaLog { saga_id: 1c2ab465-924d-4e70-b0a5-8da0c420a446, unwinding: false, events: [N009 started, N009 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N010 started, N010 succeeded], node_status: {0: Succeeded(String("0e834800-ee6d-4a55-85a2-77dc20f45b65")), 1: Succeeded(String("72cdb49f-7794-451f-b0c5-655e7352ea88")), 2: Succeeded(Null), 3: Succeeded(Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}), 4: Succeeded(Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("49f5741a-d3fe-455e-8935-4c0e4e29a17e"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb0ed45c-9135-40ee-80fa-a3196b65fcd1"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("50cf5783-8493-4a26-86b5-af5c359f51fc"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}]]), 5: Succeeded(Null), 6: Succeeded(Object {"block_size": Number(512), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "read_only_parent": Null, "sub_volumes": Array [Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "extent_count": Number(16), "gen": Number(1), "opts": Object {"cert_pem": Null, "control": Null, "flush_timeout": Null, "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "key": String("4pWJZKbuUMp1BIcXHrT0GR6uWso94P2U/F4qt3GjNn0="), "key_pem": Null, "lossy": Bool(false), "read_only": Bool(false), "root_cert_pem": Null, "target": Array [String("[::1]:1056"), String("[::1]:1006"), String("[::1]:1106")]}, "type": String("region")}], "type": String("volume")}), 7: Succeeded(Null), 8: Succeeded(Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}}), 9: Succeeded(Null), 10: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}}, node_outputs: {"created_disk": Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}}, "created_volume": Null, "crucible_disk": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}, "datasets_and_regions": Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("49f5741a-d3fe-455e-8935-4c0e4e29a17e"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb0ed45c-9135-40ee-80fa-a3196b65fcd1"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("50cf5783-8493-4a26-86b5-af5c359f51fc"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}]], "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "no_result": Null, "regions_ensure": Object {"block_size": Number(512), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "read_only_parent": Null, "sub_volumes": Array [Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "extent_count": Number(16), "gen": Number(1), "opts": Object {"cert_pem": Null, "control": Null, "flush_timeout": Null, "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "key": String("4pWJZKbuUMp1BIcXHrT0GR6uWso94P2U/F4qt3GjNn0="), "key_pem": Null, "lossy": Bool(false), "read_only": Bool(false), "root_cert_pem": Null, "target": Array [String("[::1]:1056"), String("[::1]:1006"), String("[::1]:1106")]}, "type": String("region")}], "type": String("volume")}, "regions_ensure_undo": Null, "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")} }) }
08:00:30.337Z INFO omicron-dev (omicron_sled_agent::sim::SimulatedUpstairs): found region
    region_id = 49f5741a-d3fe-455e-8935-4c0e4e29a17e
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    snapshot_id = dc83911d-d59f-418b-aa86-b96a5796d08f
08:00:30.337Z INFO omicron-dev (omicron_sled_agent::sim::Server): creating region 49f5741a-d3fe-455e-8935-4c0e4e29a17e snapshot dc83911d-d59f-418b-aa86-b96a5796d08f
    end_port = 1100
    kind = storage
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    start_port = 1050

08:00:27.468Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): Allocated regions for volume
    datasets_and_regions = [(CrucibleDataset { identity: CrucibleDatasetIdentity { id: 719df3de-6365-4f9b-9ac3-b4c8705b8b04 (dataset), time_created: 2026-02-25T07:57:22.054208Z, time_modified: 2026-02-25T07:57:22.054208Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 8a2174f0-5323-4b40-b3c7-a1503c2d0141 (zpool), ip: ::1, port: SqlU16(36659), size_used: 6710886400, no_provision: false }, Region { identity: RegionIdentity { id: 49f5741a-d3fe-455e-8935-4c0e4e29a17e, time_created: 2026-02-25T08:00:27.407827Z, time_modified: 2026-02-25T08:00:27.407827Z }, dataset_id: 719df3de-6365-4f9b-9ac3-b4c8705b8b04 (dataset), volume_id: 72cdb49f-7794-451f-b0c5-655e7352ea88 (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive }), (CrucibleDataset { identity: CrucibleDatasetIdentity { id: 9993ade3-b396-4d44-ad60-ae802c6dbe09 (dataset), time_created: 2026-02-25T07:57:21.185166Z, time_modified: 2026-02-25T07:57:21.185166Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 95885da3-b8a8-4ed2-96d7-e8410077a3ad (zpool), ip: ::1, port: SqlU16(38747), size_used: 6710886400, no_provision: false }, Region { identity: RegionIdentity { id: eb0ed45c-9135-40ee-80fa-a3196b65fcd1, time_created: 2026-02-25T08:00:27.407827Z, time_modified: 2026-02-25T08:00:27.407827Z }, dataset_id: 9993ade3-b396-4d44-ad60-ae802c6dbe09 (dataset), volume_id: 72cdb49f-7794-451f-b0c5-655e7352ea88 (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive }), (CrucibleDataset { identity: CrucibleDatasetIdentity { id: 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset), time_created: 2026-02-25T07:57:22.445518Z, time_modified: 2026-02-25T07:57:22.445518Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 08182027-59b4-4ec8-b2db-edb26c7ae933 (zpool), ip: ::1, port: SqlU16(40241), size_used: 6710886400, no_provision: false }, Region { identity: RegionIdentity { id: 50cf5783-8493-4a26-86b5-af5c359f51fc, time_created: 2026-02-25T08:00:27.407827Z, time_modified: 2026-02-25T08:00:27.407827Z }, dataset_id: 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset), volume_id: 72cdb49f-7794-451f-b0c5-655e7352ea88 (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive })]
    maybe_snapshot_id = None
    volume_id = 72cdb49f-7794-451f-b0c5-655e7352ea88
08:00:27.468Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Succeeded(Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("49f5741a-d3fe-455e-8935-4c0e4e29a17e"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb0ed45c-9135-40ee-80fa-a3196b65fcd1"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("50cf5783-8493-4a26-86b5-af5c359f51fc"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}]])
    node_id = 4
    saga_id = 1c2ab465-924d-4e70-b0a5-8da0c420a446
08:00:27.517Z INFO omicron-dev (omicron_sled_agent::sim::Server): created region eb0ed45c-9135-40ee-80fa-a3196b65fcd1
    end_port = 1050
    kind = storage
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    start_port = 1000
08:00:27.823Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saga finished
    saga_id = 1c2ab465-924d-4e70-b0a5-8da0c420a446
    saga_name = disk-create
    saga_result = SagaResult { saga_id: 1c2ab465-924d-4e70-b0a5-8da0c420a446, saga_log: SagaLog { saga_id: 1c2ab465-924d-4e70-b0a5-8da0c420a446, unwinding: false, events: [N009 started, N009 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N010 started, N010 succeeded], node_status: {0: Succeeded(String("0e834800-ee6d-4a55-85a2-77dc20f45b65")), 1: Succeeded(String("72cdb49f-7794-451f-b0c5-655e7352ea88")), 2: Succeeded(Null), 3: Succeeded(Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}), 4: Succeeded(Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("49f5741a-d3fe-455e-8935-4c0e4e29a17e"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb0ed45c-9135-40ee-80fa-a3196b65fcd1"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("50cf5783-8493-4a26-86b5-af5c359f51fc"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}]]), 5: Succeeded(Null), 6: Succeeded(Object {"block_size": Number(512), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "read_only_parent": Null, "sub_volumes": Array [Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "extent_count": Number(16), "gen": Number(1), "opts": Object {"cert_pem": Null, "control": Null, "flush_timeout": Null, "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "key": String("4pWJZKbuUMp1BIcXHrT0GR6uWso94P2U/F4qt3GjNn0="), "key_pem": Null, "lossy": Bool(false), "read_only": Bool(false), "root_cert_pem": Null, "target": Array [String("[::1]:1056"), String("[::1]:1006"), String("[::1]:1106")]}, "type": String("region")}], "type": String("volume")}), 7: Succeeded(Null), 8: Succeeded(Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}}), 9: Succeeded(Null), 10: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}}, node_outputs: {"created_disk": Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}}, "created_volume": Null, "crucible_disk": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "name": String("acc-terraform-6415b431-8c9d-4a57-b5a0-fd0077df1f50"), "time_created": String("2026-02-25T08:00:27.136307Z"), "time_deleted": Null, "time_modified": String("2026-02-25T08:00:27.136307Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T08:00:27.136305175Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}}, "datasets_and_regions": Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("49f5741a-d3fe-455e-8935-4c0e4e29a17e"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("eb0ed45c-9135-40ee-80fa-a3196b65fcd1"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(6710886400), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("50cf5783-8493-4a26-86b5-af5c359f51fc"), "time_created": String("2026-02-25T08:00:27.407827Z"), "time_modified": String("2026-02-25T08:00:27.407827Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")}]], "disk_id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "no_result": Null, "regions_ensure": Object {"block_size": Number(512), "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "read_only_parent": Null, "sub_volumes": Array [Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "extent_count": Number(16), "gen": Number(1), "opts": Object {"cert_pem": Null, "control": Null, "flush_timeout": Null, "id": String("0e834800-ee6d-4a55-85a2-77dc20f45b65"), "key": String("4pWJZKbuUMp1BIcXHrT0GR6uWso94P2U/F4qt3GjNn0="), "key_pem": Null, "lossy": Bool(false), "read_only": Bool(false), "root_cert_pem": Null, "target": Array [String("[::1]:1056"), String("[::1]:1006"), String("[::1]:1106")]}, "type": String("region")}], "type": String("volume")}, "regions_ensure_undo": Null, "volume_id": String("72cdb49f-7794-451f-b0c5-655e7352ea88")} }) }
08:00:30.337Z INFO omicron-dev (omicron_sled_agent::sim::SimulatedUpstairs): found region
    region_id = eb0ed45c-9135-40ee-80fa-a3196b65fcd1
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    snapshot_id = dc83911d-d59f-418b-aa86-b96a5796d08f
08:00:30.337Z INFO omicron-dev (omicron_sled_agent::sim::Server): creating region eb0ed45c-9135-40ee-80fa-a3196b65fcd1 snapshot dc83911d-d59f-418b-aa86-b96a5796d08f
    end_port = 1050
    kind = storage
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    start_port = 1000

For reference, this is what a successful request looks like:

07:59:44.941Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): Allocated regions for volume
    datasets_and_regions = [(CrucibleDataset { identity: CrucibleDatasetIdentity { id: 719df3de-6365-4f9b-9ac3-b4c8705b8b04 (dataset), time_created: 2026-02-25T07:57:22.054208Z, time_modified: 2026-02-25T07:57:22.054208Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 8a2174f0-5323-4b40-b3c7-a1503c2d0141 (zpool), ip: ::1, port: SqlU16(36659), size_used: 9395240960, no_provision: false }, Region { identity: RegionIdentity { id: 48dffa4e-97f0-4c0d-8348-7ab5f9a5de5c, time_created: 2026-02-25T07:59:44.902183Z, time_modified: 2026-02-25T07:59:44.902183Z }, dataset_id: 719df3de-6365-4f9b-9ac3-b4c8705b8b04 (dataset), volume_id: cd2d3967-5c6b-4cba-a250-73d2d877483c (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive }), (CrucibleDataset { identity: CrucibleDatasetIdentity { id: 9993ade3-b396-4d44-ad60-ae802c6dbe09 (dataset), time_created: 2026-02-25T07:57:21.185166Z, time_modified: 2026-02-25T07:57:21.185166Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 95885da3-b8a8-4ed2-96d7-e8410077a3ad (zpool), ip: ::1, port: SqlU16(38747), size_used: 9395240960, no_provision: false }, Region { identity: RegionIdentity { id: 6ddf70f0-a4dc-4c10-b83d-ef05debda60c, time_created: 2026-02-25T07:59:44.902183Z, time_modified: 2026-02-25T07:59:44.902183Z }, dataset_id: 9993ade3-b396-4d44-ad60-ae802c6dbe09 (dataset), volume_id: cd2d3967-5c6b-4cba-a250-73d2d877483c (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive }), (CrucibleDataset { identity: CrucibleDatasetIdentity { id: 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset), time_created: 2026-02-25T07:57:22.445518Z, time_modified: 2026-02-25T07:57:22.445518Z }, time_deleted: None, rcgen: Generation(Generation(1)), pool_id: 08182027-59b4-4ec8-b2db-edb26c7ae933 (zpool), ip: ::1, port: SqlU16(40241), size_used: 9395240960, no_provision: false }, Region { identity: RegionIdentity { id: d2f8d053-1032-48a7-b727-6055aa952150, time_created: 2026-02-25T07:59:44.902183Z, time_modified: 2026-02-25T07:59:44.902183Z }, dataset_id: 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset), volume_id: cd2d3967-5c6b-4cba-a250-73d2d877483c (volume), block_size: ByteCount(ByteCount(512)), blocks_per_extent: 131072, extent_count: 16, port: None, read_only: false, deleting: false, reservation_percent: TwentyFive })]
    maybe_snapshot_id = None
    volume_id = cd2d3967-5c6b-4cba-a250-73d2d877483c
07:59:44.943Z DEBG e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): recording saga event
    event_type = Succeeded(Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("48dffa4e-97f0-4c0d-8348-7ab5f9a5de5c"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("6ddf70f0-a4dc-4c10-b83d-ef05debda60c"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("d2f8d053-1032-48a7-b727-6055aa952150"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}]])
    node_id = 4
    saga_id = 8f8b7ebf-5e7d-4322-8599-199b82be3c2a
07:59:44.986Z INFO omicron-dev (omicron_sled_agent::sim::Server): created region d2f8d053-1032-48a7-b727-6055aa952150
    end_port = 1150
    kind = storage
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    start_port = 1100
07:59:45.393Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): saga finished
    saga_id = 8f8b7ebf-5e7d-4322-8599-199b82be3c2a
    saga_name = disk-create
    saga_result = SagaResult { saga_id: 8f8b7ebf-5e7d-4322-8599-199b82be3c2a, saga_log: SagaLog { saga_id: 8f8b7ebf-5e7d-4322-8599-199b82be3c2a, unwinding: false, events: [N009 started, N009 succeeded, N000 started, N000 succeeded, N001 started, N001 succeeded, N002 started, N002 succeeded, N003 started, N003 succeeded, N004 started, N004 succeeded, N005 started, N005 succeeded, N006 started, N006 succeeded, N007 started, N007 succeeded, N008 started, N008 succeeded, N010 started, N010 succeeded], node_status: {0: Succeeded(String("46b28588-8f99-4361-bfac-bc849864da3b")), 1: Succeeded(String("cd2d3967-5c6b-4cba-a250-73d2d877483c")), 2: Succeeded(Null), 3: Succeeded(Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "name": String("acc-terraform-ba5477e8-5c58-4c3a-a189-7b02f57e1af2"), "time_created": String("2026-02-25T07:59:44.465785Z"), "time_deleted": Null, "time_modified": String("2026-02-25T07:59:44.465785Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T07:59:44.465783240Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}}), 4: Succeeded(Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("48dffa4e-97f0-4c0d-8348-7ab5f9a5de5c"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("6ddf70f0-a4dc-4c10-b83d-ef05debda60c"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("d2f8d053-1032-48a7-b727-6055aa952150"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}]]), 5: Succeeded(Null), 6: Succeeded(Object {"block_size": Number(512), "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "read_only_parent": Null, "sub_volumes": Array [Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "extent_count": Number(16), "gen": Number(1), "opts": Object {"cert_pem": Null, "control": Null, "flush_timeout": Null, "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "key": String("rnfX+g6kvkh5yvuCJLrmmA+suk9tKwO4H/KxXSn8hoU="), "key_pem": Null, "lossy": Bool(false), "read_only": Bool(false), "root_cert_pem": Null, "target": Array [String("[::1]:1058"), String("[::1]:1008"), String("[::1]:1108")]}, "type": String("region")}], "type": String("volume")}), 7: Succeeded(Null), 8: Succeeded(Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "name": String("acc-terraform-ba5477e8-5c58-4c3a-a189-7b02f57e1af2"), "time_created": String("2026-02-25T07:59:44.465785Z"), "time_deleted": Null, "time_modified": String("2026-02-25T07:59:44.465785Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T07:59:44.465783240Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}}}), 9: Succeeded(Null), 10: Succeeded(Null)} }, kind: Ok(SagaResultOk { saga_output: Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "name": String("acc-terraform-ba5477e8-5c58-4c3a-a189-7b02f57e1af2"), "time_created": String("2026-02-25T07:59:44.465785Z"), "time_deleted": Null, "time_modified": String("2026-02-25T07:59:44.465785Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T07:59:44.465783240Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}}}, node_outputs: {"created_disk": Object {"Crucible": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "name": String("acc-terraform-ba5477e8-5c58-4c3a-a189-7b02f57e1af2"), "time_created": String("2026-02-25T07:59:44.465785Z"), "time_deleted": Null, "time_modified": String("2026-02-25T07:59:44.465785Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T07:59:44.465783240Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}}}, "created_volume": Null, "crucible_disk": Object {"disk": Object {"block_size": String("Traditional"), "disk_type": String("Crucible"), "identity": Object {"description": String("a test disk"), "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "name": String("acc-terraform-ba5477e8-5c58-4c3a-a189-7b02f57e1af2"), "time_created": String("2026-02-25T07:59:44.465785Z"), "time_deleted": Null, "time_modified": String("2026-02-25T07:59:44.465785Z")}, "project_id": String("1befb81b-31ba-48e6-bd64-4ffdd5812afd"), "rcgen": Number(1), "runtime_state": Object {"attach_instance_id": Null, "disk_state": String("creating"), "gen": Number(1), "time_updated": String("2026-02-25T07:59:44.465783240Z")}, "size": Number(1073741824), "slot": Null}, "disk_type_crucible": Object {"create_image_id": Null, "create_snapshot_id": Null, "disk_id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "pantry_address": Null, "read_only": Bool(false), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}}, "datasets_and_regions": Array [Array [Object {"identity": Object {"id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "time_created": String("2026-02-25T07:57:22.054208Z"), "time_modified": String("2026-02-25T07:57:22.054208Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("8a2174f0-5323-4b40-b3c7-a1503c2d0141"), "port": Number(36659), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("719df3de-6365-4f9b-9ac3-b4c8705b8b04"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("48dffa4e-97f0-4c0d-8348-7ab5f9a5de5c"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}], Array [Object {"identity": Object {"id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "time_created": String("2026-02-25T07:57:21.185166Z"), "time_modified": String("2026-02-25T07:57:21.185166Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("95885da3-b8a8-4ed2-96d7-e8410077a3ad"), "port": Number(38747), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9993ade3-b396-4d44-ad60-ae802c6dbe09"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("6ddf70f0-a4dc-4c10-b83d-ef05debda60c"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}], Array [Object {"identity": Object {"id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "time_created": String("2026-02-25T07:57:22.445518Z"), "time_modified": String("2026-02-25T07:57:22.445518Z")}, "ip": String("::1"), "no_provision": Bool(false), "pool_id": String("08182027-59b4-4ec8-b2db-edb26c7ae933"), "port": Number(40241), "rcgen": Number(1), "size_used": Number(9395240960), "time_deleted": Null}, Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "dataset_id": String("9eb738c9-2f66-4f42-9bf6-d78c6167419d"), "deleting": Bool(false), "extent_count": Number(16), "identity": Object {"id": String("d2f8d053-1032-48a7-b727-6055aa952150"), "time_created": String("2026-02-25T07:59:44.902183Z"), "time_modified": String("2026-02-25T07:59:44.902183Z")}, "port": Null, "read_only": Bool(false), "reservation_percent": String("TwentyFive"), "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")}]], "disk_id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "no_result": Null, "regions_ensure": Object {"block_size": Number(512), "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "read_only_parent": Null, "sub_volumes": Array [Object {"block_size": Number(512), "blocks_per_extent": Number(131072), "extent_count": Number(16), "gen": Number(1), "opts": Object {"cert_pem": Null, "control": Null, "flush_timeout": Null, "id": String("46b28588-8f99-4361-bfac-bc849864da3b"), "key": String("rnfX+g6kvkh5yvuCJLrmmA+suk9tKwO4H/KxXSn8hoU="), "key_pem": Null, "lossy": Bool(false), "read_only": Bool(false), "root_cert_pem": Null, "target": Array [String("[::1]:1058"), String("[::1]:1008"), String("[::1]:1108")]}, "type": String("region")}], "type": String("volume")}, "regions_ensure_undo": Null, "volume_id": String("cd2d3967-5c6b-4cba-a250-73d2d877483c")} }) }
07:59:48.640Z INFO omicron-dev (omicron_sled_agent::sim::SimulatedUpstairs): found region
    region_id = d2f8d053-1032-48a7-b727-6055aa952150
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    snapshot_id = 38d95da6-af8a-4762-bb5c-c3e7e5df64dd
07:59:48.640Z INFO omicron-dev (omicron_sled_agent::sim::Server): creating region d2f8d053-1032-48a7-b727-6055aa952150 snapshot 38d95da6-af8a-4762-bb5c-c3e7e5df64dd
    end_port = 1150
    kind = storage
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    start_port = 1100
07:59:48.875Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): contacting crucible agent to confirm region exists
    dataset = 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset)
    region = d2f8d053-1032-48a7-b727-6055aa952150
    saga_id = 91570677-7eaa-4af4-a8bb-c92f9ffefc02
    saga_name = snapshot-create
07:59:48.886Z TRAC omicron-dev (omicron_sled_agent::sim::Server): incoming request
    kind = storage
    local_addr = [::1]:40241
    method = GET
    remote_addr = [::1]:44278
    req_id = 059df98e-1571-40bf-95a8-3cc658b25b15
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /crucible/0/regions/d2f8d053-1032-48a7-b727-6055aa952150
07:59:48.887Z INFO omicron-dev (omicron_sled_agent::sim::Server): request completed
    kind = storage
    latency_us = 301
    local_addr = [::1]:40241
    method = GET
    remote_addr = [::1]:44278
    req_id = 059df98e-1571-40bf-95a8-3cc658b25b15
    response_code = 200
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /crucible/0/regions/d2f8d053-1032-48a7-b727-6055aa952150
07:59:48.887Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): confirmed the region exists
    dataset = 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset)
    region = Region { block_size: 512, cert_pem: None, encrypted: true, extent_count: 16, extent_size: 131072, id: RegionId("d2f8d053-1032-48a7-b727-6055aa952150"), key_pem: None, port_number: 1108, read_only: false, root_pem: None, source: None, state: Created }
    saga_id = 91570677-7eaa-4af4-a8bb-c92f9ffefc02
    saga_name = snapshot-create
07:59:48.888Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): contacting crucible agent to confirm snapshot exists
    dataset = 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset)
    region = d2f8d053-1032-48a7-b727-6055aa952150
    saga_id = 91570677-7eaa-4af4-a8bb-c92f9ffefc02
    saga_name = snapshot-create
    snapshot = 38d95da6-af8a-4762-bb5c-c3e7e5df64dd
07:59:48.899Z TRAC omicron-dev (omicron_sled_agent::sim::Server): incoming request
    kind = storage
    local_addr = [::1]:40241
    method = GET
    remote_addr = [::1]:44278
    req_id = 6c2fa8ca-bbf6-443b-9012-2bdb3d435a58
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /crucible/0/regions/d2f8d053-1032-48a7-b727-6055aa952150/snapshots/38d95da6-af8a-4762-bb5c-c3e7e5df64dd
07:59:48.902Z INFO omicron-dev (omicron_sled_agent::sim::Server): request completed
    kind = storage
    latency_us = 1929
    local_addr = [::1]:40241
    method = GET
    remote_addr = [::1]:44278
    req_id = 6c2fa8ca-bbf6-443b-9012-2bdb3d435a58
    response_code = 200
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /crucible/0/regions/d2f8d053-1032-48a7-b727-6055aa952150/snapshots/38d95da6-af8a-4762-bb5c-c3e7e5df64dd
07:59:48.903Z INFO e6bff1ff-24fb-49dc-a54e-c6a350cd4d6c (ServerContext): confirmed the snapshot exists
    dataset = 9eb738c9-2f66-4f42-9bf6-d78c6167419d (dataset)
    region = RegionId("d2f8d053-1032-48a7-b727-6055aa952150")
    saga_id = 91570677-7eaa-4af4-a8bb-c92f9ffefc02
    saga_name = snapshot-create
    snapshot = Snapshot { created: 2026-02-25T07:59:48.640304233Z, name: "38d95da6-af8a-4762-bb5c-c3e7e5df64dd" }
07:59:48.911Z TRAC omicron-dev (omicron_sled_agent::sim::Server): incoming request
    kind = storage
    local_addr = [::1]:40241
    method = POST
    remote_addr = [::1]:44278
    req_id = f04789d7-5315-4ca6-b893-61596eddbfd6
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /crucible/0/regions/d2f8d053-1032-48a7-b727-6055aa952150/snapshots/38d95da6-af8a-4762-bb5c-c3e7e5df64dd/run
07:59:48.911Z INFO omicron-dev (omicron_sled_agent::sim::Server): request completed
    kind = storage
    latency_us = 396
    local_addr = [::1]:40241
    method = POST
    remote_addr = [::1]:44278
    req_id = f04789d7-5315-4ca6-b893-61596eddbfd6
    response_code = 200
    server = b6d65341-167c-41df-9b5c-41cded99c229
    sled_id = b6d65341-167c-41df-9b5c-41cded99c229
    uri = /crucible/0/regions/d2f8d053-1032-48a7-b727-6055aa952150/snapshots/38d95da6-af8a-4762-bb5c-c3e7e5df64dd/run