[Integ-tests] Relax implicit timeout in test_slurm_scaling

hanwen-cluster · hanwen-cluster · commit 00d29e95d033 · 2023-03-07T07:43:55.000-08:00
It is a sporadic problem in integration tests code when testing in isolated regions (BTW, The sporadic failure has not happened in commercial regions).
There is an implicit timeout assuming static nodes can be replaced within 200 seconds. The speed of replacing compute nodes is not the purpose of the tests. Therefore, I extended the sleep job for 50 seconds. Therefore, the implicit timeout of static nodes replacement is increased by 50 seconds too.

Signed-off-by: Hanwen &lt;hanwenli@amazon.com&gt;
diff --git a/tests/integration-tests/tests/schedulers/test_slurm.py b/tests/integration-tests/tests/schedulers/test_slurm.py
@@ -885,7 +885,7 @@ def _test_keep_or_replace_suspended_nodes(
     )
     job_id = submit_initial_job(
         scheduler_commands,
-        "sleep 500",
+        "sleep 550",
         partition,
         dynamic_instance_type,
         num_dynamic_nodes,

Original file line number	Diff line number	Diff line change
`@@ -885,7 +885,7 @@ def _test_keep_or_replace_suspended_nodes(`
`885`	`885`	`)`
`886`	`886`	`job_id = submit_initial_job(`
`887`	`887`	`scheduler_commands,`
`888`		`- "sleep 500",`
	`888`	`+ "sleep 550",`
`889`	`889`	`partition,`
`890`	`890`	`dynamic_instance_type,`
`891`	`891`	`num_dynamic_nodes,`