Community Sops Collection

The community.sops collection allows integrating mozilla/sops in Ansible.

mozilla/sops is a tool for encryption and decryption of files using secure keys (GPG, KMS). It can be leveraged in Ansible to provide an easy to use and flexible to manage way to manage ecrypted secrets' files.

Sops version compatibility

The following table shows which versions of sops were tested with which versions of the collection. Older (or newer) versions of sops can still work fine, it just means that we did not test them. In some cases, it could be that a minimal required version of sops is explicitly documented for a specific feature. Right now, that is not the case.

community.sops version	mozilla/sops version
0.1.0	3.6.1+
1.0.2	3.6.1+
main branch	3.6.1+

Tested with Ansible

• devel
• latest Ansible 2.9 release
• latest ansible-base 2.10 release

We minimally require Ansible 2.9.10. The vars plugin requires ansible-base 2.10 or later.

External requirements

You will need to install sops manually before using plugins provided by this collection.

Included content

This collection provides:

• a lookup plugin sops that allows looking up a sops-encrypted file content;
• a vars plugin sops that allows loading Ansible vars from sops-encrypted files for hosts and groups;
• an action plugin load_vars that allows loading Ansible vars from a sops-encrypted file dynamically during a playbook or role;
• a module encrypt_sops which allows to encrypt data with sops.

Using this collection

lookup plugin

The lookup plugin can be accessed with the community.sops.sops key.

Examples:

tasks:
  - name: Output secrets to screen (BAD IDEA!)
    ansible.builtin.debug:
        msg: "Content: {{ lookup('community.sops.sops', '/path/to/sops-encrypted-file.enc.yaml') }}"

  - name: Add SSH private key
    ansible.builtin.copy:
        content: "{{ lookup('community.sops.sops', user + '-id_rsa') }}"
        dest: /home/{{ user }}/.ssh/id_rsa
        owner: "{{ user }}"
        group: "{{ user }}"
        mode: 0600
    no_log: true  # avoid content to be written to log

See Lookup Plugins for more details on lookup plugins

vars plugin

Vars plugins only work in ansible >= 2.10 and require explicit enabling. One way to enable the plugin is by adding the following to the default section of your ansible.cfg:

vars_plugins_enabled = host_group_vars,community.sops.sops

See VARIABLE_PLUGINS_ENABLED for more details.

After the plugin is enabled, correctly named group and host vars files will be transparently decrypted with sops.

The files must end with one of these extensions:

• .sops.yaml
• .sops.yml
• .sops.json

Here is an example file structure

├── inventory/
│   ├── group_vars/
│   │   └── all.sops.yml
│   ├── host_vars/
│   │   ├── server1.sops.yml
│   │   └── server2/
│   │       └── data.sops.yml
│   └── hosts
├── playbooks/
│   └── setup-server.yml
└── ansible.cfg

You could execute the playbook in this example with the following command. The sops vars files would be decrypted and used.

$ ansible-playbook playbooks/setup-server.yml -i inventory/hosts

Determine when to load variables

Ansible 2.10 allows to determine when vars plugins load the data.

To run the sops vars plugin right after importing inventory, you can add the following to ansible.cfg:

[community.sops]
vars_stage = inventory

Caching variable files

By default, the sops vars plugin caches decrypted files to avoid having to decrypt them every task. If this is not wanted, it can be explicitly disabled in ansible.cfg:

[community.sops]
vars_cache = false

Please note that when using vars plugin staging, this setting only has effect if the variables are not only loaded during the inventory stage. See the documentation of the community.sops.sops vars plugin for more details.

load_vars action plugin

The load_vars action plugin can be used similarly to Ansible's include_vars, except that it right now only supports single files. Also, it does not allow to load proper variables (i.e. "unsafe" Jinja2 expressions which evaluate on usage), but only facts. It does allow to evaluate expressions on load-time though.

Examples:

tasks:
  - name: Load variables from file and store them in a variable
    community.sops.load_vars:
        file: path/to/sops-encrypted-file.sops.yaml
        name: variable_to_store_contents_in

  - name: Load variables from file into global namespace, and evaluate Jinja2 expressions
    community.sops.load_vars:
        file: path/to/sops-encrypted-file-with-jinja2-expressions.sops.yaml
        # The following allows to use Jinja2 expressions in the encrypted file!
        # They are evaluated right now, i.e. not later like when loaded with include_vars.
        expressions: evaluate-on-load

encrypt_sops module

The encrypt_sops module can be used to create and update sops encrypted files. It assumes that sops is configured via environment variables or a .sops.yaml file.

Examples:

tasks:
  - name: Store secret text sops encrypted
    community.sops.encrypt_sops:
        path: path/to/sops-encrypted-file.sops
        content_text: This is some secret text.

  - name: Store secret binary data sops encrypted
    community.sops.encrypt_sops:
        path: path/to/sops-encrypted-file.sops
        content_binary: "{{ some_secret_binary_data | b64encode }}"

  - name: Store secret JSON data
    community.sops.encrypt_sops:
        path: path/to/sops-encrypted-file.sops.json
        content_json:
            key1: value1
            key2:
                - value2
                - key3: value3
                  key4: value5

  - name: Store secret YAML data
    community.sops.encrypt_sops:
        path: path/to/sops-encrypted-file.sops.yaml
        content_yaml:
            key1: value1
            key2:
                - value2
                - key3: value3
                  key4: value5

Troubleshooting

Spurious failures during encryption and decryption with gpg

Sops calls gpg with --use-agent. When running multiple of these in parallel, for example when loading variables or looking up files for various hosts at once, some of these can randomly fail with messages such as

Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  D13xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Reading PGP message
      | failed: openpgp: incorrect key; GPG binary error: exit
      | status 2

  828xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx: FAILED
    - | could not decrypt data key with PGP key:
      | golang.org/x/crypto/openpgp error: Reading PGP message
      | failed: openpgp: incorrect key; GPG binary error: exit
      | status 2

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.

This is a limitation of gpg-agent which can be fixed by adding auto-expand-secmem to ~/.gnupg/gpg-agent.conf (reference on option, reference on config file).

(See ansible-collections#34 and https://dev.gnupg.org/T4146 for more details.)

Contributing to this collection

See CONTRIBUTING.md

Release notes

See CHANGELOG.rst.

Releasing, Versioning and Deprecation

This collection follows Semantic Versioning. More details on versioning can be found in the Ansible docs.

We plan to regularly release new minor or bugfix versions once new features or bugfixes have been implemented.

Releasing the current major version happens from the main branch. We will create a stable-1 branch for 1.x.y versions once we start working on a 2.0.0 release, to allow backporting bugfixes and features from the 2.0.0 branch (main) to stable-1. A stable-2 branch will be created once we work on a 3.0.0 release, and so on.

We currently are not planning any deprecations or new major releases like 2.0.0 containing backwards incompatible changes. If backwards incompatible changes are needed, we plan to deprecate the old behavior as early as possible. We also plan to backport at least bugfixes for the old major version for some time after releasing a new major version. We will not block community members from backporting other bugfixes and features from the latest stable version to older release branches, under the condition that these backports are of reasonable quality.

TODO

• add a role providing sops installation (with version pinning)
• a full test suite

Code of Conduct

This repository adheres to the Ansible Community code of conduct

More information

• Ansible Collection overview
• Ansible User guide
• Ansible Developer guide

Licensing

GNU General Public License v3.0 or later.

See COPYING to see the full text.