Skip to content

Vshard cluster does not start in tests #432

@DifferentialOrange

Description

@DifferentialOrange

Despite all the retries and checks, sometimes vshard cluster fails to start from config. For example, see https://github.com/tarantool/crud/actions/runs/8541312165/job/23400336786

Log sample:

router | 2024-03-26 17:43:18.344 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:18.446 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:18.549 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:18.551 [446474] main/122/vshard.master_search._static_router/vshard.router init.lua:1185 E> Error during master search: {"code":32,"base_type":"LuajitError","type":"LuajitError","message":"{\"reason\":\"storage is not configured\",\"code\":33,\"type\":\"ShardingError\",\"message\":\"Storage is disabled: storage is not configured\",\"name\":\"STORAGE_IS_DISABLED\"}","trace":[{"file":"./tarantool/src/lua/utils.c","line":679}]}
router | 2024-03-26 17:43:18.651 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:18.754 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:18.856 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:18.959 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:19.061 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:19.162 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
router | 2024-03-26 17:43:19.164 [446474] main/140/main/crud.storage_info storage_info.lua:77 E> Error getting storage info for s2-master: Connection closed
    select.backend:"config".engine:"memtx".test_select_safety_too_low_limit ... 
    select.backend:"config".engine:"memtx".test_select_safety_too_low_limit ... (0.000s) error
...ithub/tarantool/crud/test/tarantool3_helpers/cluster.lua:252: crud is not bootstrapped: expected 4 running storages, got the following storage info: {"s1-master":{"status":"running","is_master":true},"s1-replica":{"status":"running","is_master":false},"s2-replica":{"status":"running","is_master":false},"s2-master":{"status":"error","message":"Connection closed","is_master":true}}

luatest_log_capture.zip

Metadata

Metadata

Assignees

No one assigned

    Labels

    2spbugSomething isn't workingflaky testThe issue describes the problem with the test being flaky

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions