This project offers tools for managing Kerberos credentials in specialized environments, such as Batch and CI/CD.
It is comprised of:
- A privileged daemon hosted alongside the KDC which provides delegation and impersonation services:
sybild - A command line interface to interact with the sybil server and manage credentials:
sybil - A SPANK plugin that integrates with the Slurm workload manager to forward and retrieve credentials as part of the job lifecycle
When used together, this enables the use of Kerberos inside traditional HPC infrastructure and allow services which provide their own authentication mechanism to work within a Keberized environment.
For example, a CI/CD agent can leverage this to submit pipelines on behalf of a user authenticated outside of the Kerberos realm (e.g. OIDC, SAML).
This project relies among other things on GSSAPI and the Microsoft S4U protocol extensions.
# Prerequisites
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
apt install clang libkrb5-dev
make release
# Alternatively, with Slurm support
make release WITH_SLURM=1 SLURM_VERSION=24.11# Prerequisites
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cargo install cargo-generate-rpm cargo-deb
apt install clang libkrb5-dev
make deb rpm
# Alternatively, with Slurm support
make deb rpm WITH_SLURM=1 SLURM_VERSION=24.11Before running Sybil, one needs to configure the KDC appropriately with a new service principal and necessary authorizations.
Below, we provide sample configurations for the two most popular implementations, MIT Kerberos and FreeIPA.
Assuming an existing FreeIPA or RHEL IdM install:
# Create the Sybil service principal and generate its keytab
ipa service-add sybil/ipa.domain.lan
ipa-getkeytab -p sybil/ipa.domain.lan -k /etc/krb5.keytab
# Create the Sybil DNS service record
ipa dnsrecord-add --srv-priority=0 --srv-weight=100 --srv-port=57811 --srv-target=ipa.domain.lan. domain.lan _sybil._tcp
# In case user impersonation is needed (i.e. sybil kinit user@REALM)
# Allow a given host to perform impersonation against the Sybil service
ipa servicedelegationtarget-add sybil-target
ipa servicedelegationtarget-add-member --principals sybil/ipa.domain.lan sybil-target
ipa servicedelegationrule-add sybil
ipa servicedelegationrule-add-member --principals host/server.domain.lan sybil
ipa servicedelegationrule-add-target --servicedelegationtargets=sybil-target sybil
ipa host-mod --ok-to-auth-as-delegate=true server.domain.lan
# Allow delegation to the Sybil server
ipa service-mod --ok-as-delegate=true sybil/ipa.domain.lanAssuming an existing MIT Kerberos install with the LDAP backend (required for S4U to work):
# Create the Sybil service principal and generate its keytab
kadmin.local addprinc -randkey sybil/mit.domain.lan
kadmin.local ktadd -k /etc/krb5.keytab sybil/mit.domain.lan
# Create the Sybil DNS service record as follow in your DNS server
# _sybil._tcp.domain.lan. 86400 IN SRV 0 100 57811 mit.domain.lan.
# In case user impersonation is needed (i.e. sybil kinit user@REALM)
# Allow a given host to perform impersonation against the Sybil service
ldapmodify -Y EXTERNAL -H ldapi:// <<EOF
dn: krbPrincipalName=host/[email protected],cn=DOMAIN.LAN,cn=krbContainer,dc=domain,dc=lan
changetype: modify
add: krbAllowedToDelegateTo
krbAllowedToDelegateTo: sybil/mit.domain.lan
EOF
kadmin.local modprinc +ok_to_auth_as_delegate host/server.domain.lan
# Allow delegation to the Sybil server
kadmin.local modprinc +ok_as_delegate sybil/ipa.domain.lanThe Sybil server relies on the KCM protocol to store delegated credentials.
Both KCM and the Sybil server need to be deployed alongside the KDC.
Install and configure KCM to store and refresh tickets delegated to Sybil:
# Install KCM
apt install sssd-kcm
# Configure KCM to automatically renew tickets (e.g. every 30 mins)
tee /etc/sssd/conf.d/kcm.conf <<EOF
[kcm]
tgt_renewal = true
krb5_renew_interval = 30m
EOF
# Start KCM
systemctl enable --now sssd-kcmInstall and configure the Sybil server (c.f. reference configuration).
Arbitrary policies can be defined via ACL rules to restrict the set of operations available to each client principal:
# Install Sybil
apt install ./sybil_*.deb
# Configure Sybil with a set of predefined policies
tee /etc/sybil.toml <<EOF
[ticket]
cipher = "aes256-sha2"
flags = "FRA"
lifetime = "10h"
renewable_lifetime = "7d"
minimum_lifetime = "5m"
fully_qualified_user = false
cross_realm = false
# Allow any principal in the DOMAIN.LAN realm to store/fetch/list its tickets
[[acl]]
principal = '^.*@DOMAIN\.LAN$'
[acl.permissions]
fetch = true
store = true
list = true
# Allow host principals on a given subnet to fetch user tickets
[[acl]]
principal = '^host/.*@DOMAIN\.LAN$'
hosts = ["192.168.0.0/24"]
[acl.permissions]
fetch = true
masquerade = true
# Allow principals in the admin group to impersonate users, authenticating on their behalf (S4U)
[[acl]]
group = '^admin@domain\.lan$'
[acl.permissions]
kinit = true
EOF
# Start Sybil
systemctl enable --now sybilSybil ships with a SPANK plugin which allows users to automatically forward and renew Kerberos credentials as part of their Slurm jobs.
This plugin needs to be installed and configured on all the submission and compute nodes, after which a new --kerberos=[auto|yes|no|force]
parameter becomes available to the salloc/srun/sbatch commands.
# Install Sybil and its SPANK plugin
apt install ./sybil-spank*.deb
# Add Sybil to the list of Slurm plugins
install -D -m 644 /usr/share/sybil/slurm/plugstack.conf /etc/slurm/plugstack.conf.d/sybil.conf
# On the compute nodes only, configure the slurmd systemd override
install -D -m 644 /usr/share/sybil/slurm/slurm.conf /etc/systemd/system/slurmd.service.d/sybil.confThe folowing options can be adjusted in the plugstack configuration:
| Option | Description |
|---|---|
| default | Specify the default value for the --kerberos parameter |
| min_tkt_lifetime | Specify the minimum ticket lifetime required at submission time (--kerberos=force overrides this) |
# Retrieve a ticket for the host
kinit -k
# Acquire a ticket on behalf of the given principal
sybil kinit [email protected]
# Change to the user
su - user
# Verify that a ticket has been acquired
klist
Default principal: [email protected]
Valid starting Expires Service principal
09/15/2023 00:38:23 09/15/2023 10:38:23 krbtgt/[email protected]
renew until 09/22/2023 00:38:23
# Send this ticket to the server
sybil store
# List the tickets present on the server
sybil list
UID START_TIME END_TIME RENEW_UNTIL PRINCIPAL
1000 2025-03-12T15:04:27 2025-03-13T01:04:27 2025-03-17T12:18:00 [email protected]
# Submit a Slurm job, forwarding credentials to the allocated nodes
srun --kerberos=true klist
Default principal: [email protected]
Valid starting Expires Service principal
09/15/2023 00:38:23 09/15/2023 10:38:23 krbtgt/[email protected]
renew until 09/22/2023 00:38:23