[Feature] CMEK support for Prompt Management with Vertex AI Python SDK

[tarrade]

It is not (yet) documented documentation but from my test CMEK is not yet supported when using Prompt management with Vertex AI Python SDK. When using the UI, you can save prompt using CMEK.

Here what I saw when using 1.74.0 (don't see anything in the release note of the newer version)

Anyway this doesn't seems to work in my code [3] base on [2]:

Traceback (most recent call last):

 File "/home/jupyter/proj_ml_at_scale_gcp_exp/src/gen-ai/vertex-ai-prompt-manangement.py", line 51, in <module>

  prompt1 = prompts.create_version(prompt=prompt)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "/home/jupyter/.conda-env/env_vertex_ai_llm/lib/python3.11/site-packages/vertexai/prompts/_prompt_management.py", line 298, in create_version

  return _create_prompt_resource(prompt=prompt, version_name=version_name)

      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "/home/jupyter/.conda-env/env_vertex_ai_llm/lib/python3.11/site-packages/vertexai/prompts/_prompt_management.py", line 445, in _create_prompt_resource

  dataset = _create_dataset(prompt=prompt, parent=parent)

       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "/home/jupyter/.conda-env/env_vertex_ai_llm/lib/python3.11/site-packages/vertexai/prompts/_prompt_management.py", line 393, in _create_dataset

  operation = prompt._dataset_client.create_dataset(

        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "/home/jupyter/.conda-env/env_vertex_ai_llm/lib/python3.11/site-packages/google/cloud/aiplatform_v1/services/dataset_service/client.py", line 943, in create_dataset

  response = rpc(

        ^^^^

 File "/home/jupyter/.conda-env/env_vertex_ai_llm/lib/python3.11/site-packages/google/api_core/gapic_v1/method.py", line 131, in __call__

  return wrapped_func(*args, **kwargs)

      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

 File "/home/jupyter/.conda-env/env_vertex_ai_llm/lib/python3.11/site-packages/google/api_core/grpc_helpers.py", line 78, in error_remapped_callable

  raise exceptions.from_grpc_error(exc) from exc

google.api_core.exceptions.FailedPrecondition: 400 The request has violated one or more Org Policies: violations {

 type: "constraints/gcp.restrictNonCmekServices"

 subject: "orgpolicy:projects/PROJECT"

 description: "Constraint `constraints/gcp.restrictNonCmekServices` violated for `projects/PROJECT` attempting to create a resource without specifying a KMS CryptoKey."

}

FR: add CMEK support for Prompt Management with Vertex AI python SDK (already supported when using the UI to save prompts). I know that this come later as the feature is still in preview but just to mention our interest as all regulated industry need such capability.