Update dependency ops to v2.15.0 [SECURITY]

[renovate]

This PR contains the following updates:

Package	Change	Age	Adoption	Passing	Confidence
ops (changelog)	2.9.0 -> 2.15.0
ops (changelog)	^2.9.0, <2.10 -> <2.16

GitHub Vulnerability Alerts

CVE-2024-41129

Summary

The issue here is that we pass the secret content as one of the args via CLI. This issue may affect any of our charms that are using: Juju (>=3.0), Juju secrets and not correctly capturing and processing subprocess.CalledProcessError.

There are two points that may log this command, in different files:

First, if there is an error during a secret handling, there will be a subprocess.CalledProcessError, which will contain the CLI comand + all its args. This is going to be logged in any logging level. This exception, if not caught by the charm, will bubble up to the /var/log/juju/ logs and syslog journal. Now, on Ubuntu 22.04, these logs are protected with:

$ juju ssh -m controller 0 -- ls -la /var/log/juju/
total 224
drwxr-xr-x 2 syslog adm      4096 Jul 14 10:59 .
drwxrwxr-x 9 root   syslog   4096 Jul 14 10:58 ..
-rw-r----- 1 syslog adm     20124 Jul 14 11:10 audit.log
-rw-r----- 1 syslog adm    110432 Jul 14 11:10 logsink.log
-rw-r----- 1 syslog adm     80783 Jul 14 11:06 machine-0.log
-rw-r----- 1 syslog adm       766 Jul 14 11:10 machine-lock.log
-rw-r--r-- 1 root   root        0 Jul 14 10:59 slow-query.log
-rw-r----- 1 syslog adm      3732 Jul 14 11:10 unit-controller-0.log

Second, certain audit setups may log terminal commands, which would result in this command being logged with its secrets. It is unknown if this is done on ubuntu security benchmarks, such as CIS hardening.

Keep in mind these logs may be copied or even backed up. Which exposes it to more services in the user's environment (e.g. CI runs in GH - although these are dummy password generated per test only).

Passing secrets straight via CLI is not advised. Here are some ways out:

1. Redacting: which commands and which args represent secrets are known, so they can be redacted. It would also mean capturing a subprocess.CalledProcessError, redacting its content and reissuing the same type of exception; this will not cover the case auditd is set to log CLI commands, if that is a risk
2. Temp files: secret-add, for example, can use a secret file instead, as can be seen here. However, if ops uses a file, ops will need to be sure to correctly remove it later
3. stdin: not sure it is accepted by secret-* commands, but generally, secrets are not shown on CLI whilst typing them; auditd may not capture that stdin

Severity Rationale

This is a CWE-532. Potentially, these secrets can lead to privilege escalation but Ubuntu default is to have logs only accessible to adm group users.

Marking this issue as "Moderate", as this report is not presenting a clear way on how to get access to the logs themselves: either getting local access to an adm group user (e.g. ubuntu) or recovering logs stored on a 3rd party service.

Details

2024-07-12T14:27:58.0175389Z unit-opensearch-3: 14:27:53 ERROR unit.opensearch/3.juju-log certificates:11: Uncaught exception while in charm code:
2024-07-12T14:27:58.0175524Z Traceback (most recent call last):
2024-07-12T14:27:58.0175957Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 3180, in _run
2024-07-12T14:27:58.0176165Z     result = subprocess.run(args, **kwargs)  # type: ignore
2024-07-12T14:27:58.0176381Z   File "/usr/lib/python3.10/subprocess.py", line 526, in run
2024-07-12T14:27:58.0176561Z     raise CalledProcessError(retcode, process.args,
2024-07-12T14:27:58.0177749Z subprocess.CalledProcessError: Command '('/var/lib/juju/tools/unit-opensearch-3/secret-add', '--label', 'opensearch:app:app-admin', '--owner', 'application', 'keystore-password-ca=aUE...')' returned non-zero exit status 1.
2024-07-12T14:27:58.0177765Z
2024-07-12T14:27:58.0178023Z The above exception was the direct cause of the following exception:
2024-07-12T14:27:58.0178038Z
2024-07-12T14:27:58.0178168Z Traceback (most recent call last):
2024-07-12T14:27:58.0178588Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/./src/charm.py", line 213, in <module>
2024-07-12T14:27:58.0178724Z     main(OpenSearchOperatorCharm)
2024-07-12T14:27:58.0179129Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 548, in main
2024-07-12T14:27:58.0179237Z     manager.run()
2024-07-12T14:27:58.0179640Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 527, in run
2024-07-12T14:27:58.0179745Z     self._emit()
2024-07-12T14:27:58.0180150Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 516, in _emit
2024-07-12T14:27:58.0180359Z     _emit_charm_event(self.charm, self.dispatcher.event_name)
2024-07-12T14:27:58.0180840Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/main.py", line 147, in _emit_charm_event
2024-07-12T14:27:58.0180974Z     event_to_emit.emit(*args, **kwargs)
2024-07-12T14:27:58.0181537Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 348, in emit
2024-07-12T14:27:58.0181656Z     framework._emit(event)
2024-07-12T14:27:58.0182091Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 860, in _emit
2024-07-12T14:27:58.0182211Z     self._reemit(event_path)
2024-07-12T14:27:58.0182659Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 950, in _reemit
2024-07-12T14:27:58.0182770Z     custom_handler(event)
2024-07-12T14:27:58.0183492Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/tls_certificates_interface/v3/tls_certificates.py", line 1811, in _on_relation_changed
2024-07-12T14:27:58.0183743Z     self.on.certificate_available.emit(
2024-07-12T14:27:58.0184173Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 348, in emit
2024-07-12T14:27:58.0184291Z     framework._emit(event)
2024-07-12T14:27:58.0184717Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 860, in _emit
2024-07-12T14:27:58.0184887Z     self._reemit(event_path)
2024-07-12T14:27:58.0186171Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/framework.py", line 950, in _reemit
2024-07-12T14:27:58.0186306Z     custom_handler(event)
2024-07-12T14:27:58.0187043Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_tls.py", line 209, in _on_certificate_available
2024-07-12T14:27:58.0187320Z     self.store_new_ca(self.charm.secrets.get_object(scope, cert_type.val))
2024-07-12T14:27:58.0187942Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_tls.py", line 444, in store_new_ca
2024-07-12T14:27:58.0188242Z     self._create_keystore_pwd_if_not_exists(Scope.APP, CertType.APP_ADMIN, "ca")
2024-07-12T14:27:58.0188981Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_tls.py", line 432, in _create_keystore_pwd_if_not_exists
2024-07-12T14:27:58.0189119Z     self.charm.secrets.put_object(
2024-07-12T14:27:58.0189738Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_secrets.py", line 359, in put_object
2024-07-12T14:27:58.0189944Z     self._add_or_update_juju_secret(scope, key, value, merge)
2024-07-12T14:27:58.0190652Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_secrets.py", line 272, in _add_or_update_juju_secret
2024-07-12T14:27:58.0190815Z     return self._add_juju_secret(scope, key, value)
2024-07-12T14:27:58.0191462Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/lib/charms/opensearch/v0/opensearch_secrets.py", line 228, in _add_juju_secret
2024-07-12T14:27:58.0191660Z     secret = scope_obj.add_secret(safe_value, label=label)
2024-07-12T14:27:58.0192110Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 477, in add_secret
2024-07-12T14:27:58.0192241Z     id = self._backend.secret_add(
2024-07-12T14:27:58.0192686Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 3624, in secret_add
2024-07-12T14:27:58.0192948Z     result = self._run('secret-add', *args, return_output=True)
2024-07-12T14:27:58.0193356Z   File "/var/lib/juju/agents/unit-opensearch-3/charm/venv/ops/model.py", line 3182, in _run
2024-07-12T14:27:58.0193489Z     raise ModelError(e.stderr) from e
2024-07-12T14:27:58.0193685Z ops.model.ModelError: ERROR this unit is not the leader
2024-07-12T14:27:58.0193692Z

From CI: https://github.com/canonical/opensearch-operator/actions/runs/9908987369/job/27376377521?pr=364

PoC

1. Deploy anything with juju
2. Run a dummy secret-add call that will fail
3. See the uncaught subprocess error

Impact

Juju secrets are generally composed of private keys, passwords, etc; generally valuable credentials that, if leaked, will likely allow an attacker to get privileged access to its target or other targets in the environment.

---

Release Notes

canonical/operator (ops)

v2.15.0

Compare Source

Features

• Add support for Pebble check-failed and check-recovered events (#​1281)

Fixes

• Pass secret data to Juju via files, rather than as command-line values (#​1290) fixing CVE-2024-41129
• Include checks and log targets when merging layers in ops.testing (#​1268)

Documentation

• Clarify distinction between maintenance and waiting status (#​1148)

CI

• Bump the Go version to match Pebble (#​1285)
• Run ruff format over charm pin update code (#​1278)
• Bump certifi from 2024.2.2 to 2024.7.4 in /docs (#​1282)
• Update charm pins (#​1269)

v2.14.1

Compare Source

Fixes

• Add connect timeout for exec websockets to avoid hanging (#​1247)
• Adjust Harness secret behaviour to align with Juju (#​1248)

Tests

• Fix TypeError when running test.pebble_cli (#​1245)
• Properly clean up after running setup_root_logging in test_log (#​1259)
• Verify that defer() is not usable on stop,remove,secret-expired,secret-rotate (#​1233)

Documentation

• Fix HACKING.md link on PyPI, and internal links (#​1261, #​1236)
• Add a section to HACKING.md on PR titles (commit messages to main) (#​1252)
• Add release step to update pinned charm tests (#​1213)
• Add a security policy (#​1266)

CI

• Only run tests once on push to PR (#​1242)
• Validate PR title against conventional commit rules in (#​1262)
• Only update ops, not all dependencies, in charm tests in (#​1275)
• Add artefact attestation (#​1267)

v2.14.0

Compare Source

Features

• Add a __str__ to ActionFailed, for better unexpected failure output (#​1209)

Fixes

• The other argument to RelatationDataContent.update(...) should be optional (#​1226)

Documentation

• Use the actual emoji character rather than GitHub markup, to show properly on PyPI (#​1221)
• Clarify that SecretNotFound may be raised for permission errors (#​1231)

Refactoring

• Refactor tests to pytest style (#​1199, #​1200, #​1203, #​1206)
• Use ruff formatter and reformat all code (#​1224)
• Don't use f-strings in logging calls (#​1227, 1234)

v2.13.0

Compare Source

Features

• Added support for user secrets in Harness (#​1176)

Fixes

• Corrected the model config types (#​1183)
• In Harness, only inspect the source file if it will be used - this fixed using Harness in a Python REPL (#​1181)

Documentation

• Updated publishing a release in HACKING.md (#​1173)
• Added tox -e docs-deps to compile requirements.txt (#​1172)
• Updated doc to note deprecated functionality in (#​1178)

Tests

• First stage of converting tests from unittest to pytest (#​1191, #​1192, #​1196, #​1193, #​1195)
• Added pebble.CheckInfo.change_id field (#​1197)

v2.12.0

Compare Source

Features

• Added Model.get_cloud_spec which uses the credential-get hook tool to get details of the cloud where the model is deployed (#​1152)

Fixes

• Update Pebble Notices get_notices parameter name to users=all (previously select=all) (#​1146)
• Warn when an observer weakref is lost (#​1142)
• More robust validation of observer signatures (#​1147)
• Change Model.relation.app type from Application|None to Application (#​1151)
• Fix attaching storage in Harness before begin (#​1150)
• Fixed an issue where pebble.Client.exec might leak a socket.timeout (builtins.TimeoutError) exception (#​1155)
• Add a consistency check and default network to add_relation (#​1138)
• Don't special-case get_relation behaviour in leader-elected (#​1156)
• Accept type: secret for config options (#​1167)

Refactoring

• Refactor main.py, creating a new _Manager class (#​1085)

Documentation

• Use "integrate with" rather than "relate to" (#​1145)
• Updated code examples in the docstring of ops.testing from unittest to pytest style (#​1157)
• Add peer relation details in Harness.add_relation docstring (#​1168)
• Update Read the Docs Sphinx Furo theme to use Canonical's latest styling (#​1163, #​1164, #​1165)

v2.11.0

Compare Source

Features

• StopEvent, RemoveEvent, and all LifeCycleEvents are no longer deferrable, and will raise a RuntimeError if defer() is called on the event object (#​1122)
• Add ActionEvent.id, exposing the JUJU_ACTION_UUID environment variable (#​1124)
• Add support for creating pebble.Plan objects by passing in a pebble.PlanDict, the
  ability to compare two Plan objects with ==, and the ability to create an empty Plan with Plan() (#​1134)

Fixes

• The remote app name (and its databag) is now consistently available in relation-broken events (#​1130)

Documentation

• Improve the can_connect() API documentation (#​1123)

Tooling

• Use ruff for linting (#​1120, #​1139, #​1114)

v2.10.0

Compare Source

Features

• Add support for Pebble Notices (PebbleCustomNoticeEvent, get_notices, and so on) (#​1086, #​1100)
• Add Relation.active, and excluded inactive relations from Model.relations (#​1091)
• Add full support for charm metadata v2 (in particular, extended ContainerMeta,
  and various info links in CharmMeta) (#​1106)
• When handling actions, print uncaught exceptions to stderr (#​1087)
• Raise ModelError in Harness if an invalid status is set (#​1107)

Fixes

• Add Pebble log targets and checks to testing plans (#​1111)
• CollectStatusEvent is now a LifecycleEvent (#​1080)

Documentation

• Update README to reflect charmcraft init changes (#​1089)
• Add information on pushing locked/bind-mount files (#​1094)
• Add instructions for using a custom version of ops to HACKING (#​1092)

Tooling

• Use pyproject.toml for building (#​1068)
• Update to the latest version of Pyright (#​1105)

---

Configuration

📅 Schedule: Branch creation - "" in timezone Etc/UTC, Automerge - At any time (no schedule defined).

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about these updates again.

---

• If you want to rebase/retry this PR, check this box

---

This PR was generated by Mend Renovate. View the repository job log.

[taurus-forever]

@carlcsaposs-canonical can you please report this upstream. This PyYAML changes is popular and annoying...

[taurus-forever]

@carlcsaposs-canonical how critical ops CVE here?
Do we need to finish #273 quickly to use the latest ops?

[carlcsaposs-canonical]

Do we need to finish #273 quickly to use the latest ops?

we are somewhat blocked by canonical/operator#1279

technically we can merge 273 and ignore canonical/operator#1279 without issues—however it could paint us into a corner or cause problems in the future, as described in canonical/operator#1279

[renovate]

Renovate Ignore Notification

Because you closed this PR without merging, Renovate will ignore this update (^2.9.0). You will get a PR once a newer version is released. To ignore this dependency forever, add it to the ignoreDeps array of your Renovate config.

If you accidentally closed this PR, or if you changed your mind: rename this PR to get a fresh replacement PR.