Feature description

Currently, the way we handle default Dask worker profiles in Nebari can lead to unpredictable behavior due to how:

• Terraform parses JSON from the Nebari config YAML.
• Python loads the first ([0]) key from the dict of profiles when setting defaults.

This impacts the selection of the default Dask profile during deployment since dictionary key ordering can differ depending on how the configuration is loaded and processed.

Relevant Python implementation:

class Profiles(schema.Base):
    ...
    dask_worker: Dict[str, DaskWorkerProfile] = {
        "Small Worker": DaskWorkerProfile(...),
        "Medium Worker": DaskWorkerProfile(...),
    }

Relevant Terraform usage:

if config["profiles"]:
    args += [
        Select(
            "profile",
            list(config["profiles"].keys()),
            default=list(config["profiles"].keys())[0],
            label="Cluster Profile",
        )
    ]

Value and/or benefit

• Predictable deployments: Removes reliance on key order for default profile selection.
• User control: Allows explicit default selection.

Anything else?

No response