Sanitize job names with runner?

[jbusecke]

I am running repeatedly into issues with dataflow job names that are not allowed on dataflow like here.

Runner warns about this, but fails. This is acceptable in general but adds avoidable overhead for the maintainer (who often already has a steep learning curve ahead of them). It is also only an issue on Dataflow and thus would be nicely abstracted away on the runner level?

I suggest that we sanitize jobnames within runner in the following way:

• replace offending characters (e.g. _ with -)
• trim the job name to the max length (with appropriate warning)

[cisaacstern]

This block of code is supposed to be preventing this from happening. So there must be something missing from the logic here:

pangeo-forge-runner/pangeo_forge_runner/commands/bake.py

Lines 85 to 101 in e2faa93

	@validate("job_name")
	def _validate_job_name(self, proposal):
	"""
	Validate that job_name conforms to ^[a-z][-_0-9a-z]{0,62}$ regex.
	That's what is valid in dataflow job names, so let's keep everyone
	in that range.
	Dataflow job names adhere to the following GCP cloud labels requirements:
	https://cloud.google.com/resource-manager/docs/creating-managing-labels#requirements
	"""
	validating_regex = r"^[a-z][-_0-9a-z]{0,62}$"
	if not re.match(validating_regex, proposal.value):
	raise ValueError(
	f"job_name must match the regex {validating_regex}, instead found {proposal.value}"
	)
	return proposal.value

[jbusecke]

@cisaacstern I think it is 100% working! It raises exactly that error. But what I am proposing is to actually change the jobname within runner so that the error is not raised in the first place.

[jbusecke]

Just pinging this again as pretty major useability barrier for out LEAP Data ingestion. I have run into this many times, most recently in https://github.com/leap-stc/eNATL_feedstock/actions/runs/9906590482. I think this is an easily fixable issue that would greatly enhance the user experience with pgf-runner.