Skip to content

Commit

Permalink
11.0.0+1.14.1 (#27)
Browse files Browse the repository at this point in the history
* upgrade to Cilium v1.14.1

* update README

* molecule/kvm/molecule.yml: use minikube 1.31.2 (K8s 1.27.4)

* rename Molecule scenario kvm to default

* update README

* refactor Molecule tests

* molecule/default/prepare.yml: install githubixx.cilium_cli to make Molecule verify step work

* update README

* update CHANGELOG

* Molecule tests: fix ansible-lint issues
  • Loading branch information
githubixx authored Sep 6, 2023
1 parent 8e1f445 commit 8e2e744
Show file tree
Hide file tree
Showing 38 changed files with 1,285 additions and 166 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
# Changelog

## 11.0.0+1.14.1

- upgrade to Cilium `v1.14.1`
- refactor Molecule tests

## 10.0.2+1.13.4

- upgrade to Cilium `v1.13.4`
Expand Down
34 changes: 20 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This Ansible role installs [Cilium](https://docs.cilium.io) network on a Kuberne
Versions
--------

I tag every release and try to stay with [semantic versioning](http://semver.org). If you want to use the role I recommend to checkout the latest tag. The master branch is basically development while the tags mark stable releases. But in general I try to keep master in good shape too. A tag `10.0.1+1.13.2` means this is release `10.0.1` of this role and it contains Cilium chart version `1.13.2`. If the role itself changes `X.Y.Z` before `+` will increase. If the Cilium chart version changes `X.Y.Z` after `+` will increase too. This allows to tag bugfixes and new major versions of the role while it's still developed for a specific Cilium release.
I tag every release and try to stay with [semantic versioning](http://semver.org). If you want to use the role I recommend to checkout the latest tag. The master branch is basically development while the tags mark stable releases. But in general I try to keep master in good shape too. A tag `11.0.0+1.14.1` means this is release `11.0.0` of this role and it contains Cilium chart version `1.14.1`. If the role itself changes `X.Y.Z` before `+` will increase. If the Cilium chart version changes `X.Y.Z` after `+` will increase too. This allows to tag bugfixes and new major versions of the role while it's still developed for a specific Cilium release.

Requirements
------------
Expand All @@ -28,7 +28,7 @@ Role Variables

```yaml
# Helm chart version
cilium_chart_version: "1.13.4"
cilium_chart_version: "1.14.1"

# Helm chart name
cilium_chart_name: "cilium"
Expand Down Expand Up @@ -146,13 +146,13 @@ ansible-playbook --tags=role-cilium-kubernetes --extra-vars cilium_action=instal

To check if everything was deployed use the usual `kubectl` commands like `kubectl -n <cilium_namespace> get pods -o wide`.

As [Cilium](https://docs.cilium.io) issues updates/upgrades every few weeks/months the role also can do upgrades. The role basically executes what is described in [Cilium upgrade guide](https://docs.cilium.io/en/v1.12/operations/upgrade/). That means the Cilium pre-flight check will be installed and some checks are executed before the update actually takes place. Have a look at `tasks/upgrade.yml` to see what's happening before, during and after the update. Of course you should consult [Cilium upgrade guide](https://docs.cilium.io/en/v1.12/operations/upgrade/) in general to check for major changes and stuff like that before upgrading.
As [Cilium](https://docs.cilium.io) issues updates/upgrades every few weeks/months the role also can do upgrades. The role basically executes what is described in [Cilium upgrade guide](https://docs.cilium.io/en/v1.14/operations/upgrade/). That means the Cilium pre-flight check will be installed and some checks are executed before the update actually takes place. Have a look at `tasks/upgrade.yml` to see what's happening before, during and after the update. Of course you should consult [Cilium upgrade guide](https://docs.cilium.io/en/v1.14/operations/upgrade/) in general to check for major changes and stuff like that before upgrading. Also make sure to check the [Upgrade Notes](https://docs.cilium.io/en/stable/operations/upgrade/#current-release-required-changes)!

If a upgrade wasn't successful a [Roll back](https://docs.cilium.io/en/v1.12/operations/upgrade/#step-3-rolling-back) to a previous version can be basically initiated by just changing `cilium_chart_version` variable. But you should definitely read the Cilium [roll back guide](https://docs.cilium.io/en/v1.12/operations/upgrade/#step-3-rolling-back). Switching between minor releases is normally not an issue but switching from one major release to a previous one might be not so easy.
If a upgrade wasn't successful a [Roll back](https://docs.cilium.io/en/v1.14/operations/upgrade/#step-3-rolling-back) to a previous version can be basically initiated by just changing `cilium_chart_version` variable. But you should definitely read the Cilium [roll back guide](https://docs.cilium.io/en/v1.14/operations/upgrade/#step-3-rolling-back). Switching between minor releases is normally not an issue but switching from one major release to a previous one might be not so easy.

Also check `templates/cilium_values_default_pre_flight_check.yml.j2`. If you need to adjust values for the `pre-flight` check you can either change that file or create a file `templates/cilium_values_user_pre_flight_check.yml.j2` with your own values.

Before doing the upgrade you basically only need to change `cilium_chart_version` variable e.g. from `1.10.10` to `1.11.4` to upgrade from `1.10.10` to `1.11.4`. So to do the update run
Before doing the upgrade you basically only need to change `cilium_chart_version` variable e.g. from `1.13.4` to `1.14.1` to upgrade from `1.13.4` to `1.14.1`. So to do the update run

```bash
ansible-playbook --tags=role-cilium-kubernetes --extra-vars cilium_action=upgrade k8s.yml
Expand Down Expand Up @@ -193,42 +193,48 @@ Example 2 (assign tag to role):
Testing
-------

This role has a small test setup that is created using [Molecule](https://github.com/ansible-community/molecule), libvirt (vagrant-libvirt) and QEMU/KVM. Please see my blog post [Testing Ansible roles with Molecule, libvirt (vagrant-libvirt) and QEMU/KVM](https://www.tauceti.blog/posts/testing-ansible-roles-with-molecule-libvirt-vagrant-qemu-kvm/) how to setup. The test configuration is [here](https://github.com/githubixx/ansible-role-cilium-kubernetes/tree/master/molecule/kvm).
This role has a small test setup that is created using [Molecule](https://github.com/ansible-community/molecule), libvirt (vagrant-libvirt) and QEMU/KVM. Please see my blog post [Testing Ansible roles with Molecule, libvirt (vagrant-libvirt) and QEMU/KVM](https://www.tauceti.blog/posts/testing-ansible-roles-with-molecule-libvirt-vagrant-qemu-kvm/) how to setup. The test configuration is [here](https://github.com/githubixx/ansible-role-cilium-kubernetes/tree/master/molecule/default).

Afterwards molecule can be executed. The following command will do a basic setup and create a template of the resources (default action see above) that will be created:

```bash
molecule converge -s kvm
molecule converge
```

Installing `Cilium` and the required resources. This will setup a virtual machine (VM) and installs a minimal Kubernetes setup using `minikube`. That setup will be used to install `Cilium` by using this role.
Installing `Cilium` and the required resources. This will setup a few virtual machines (VM) and installs a Kubernetes cluster. That setup will be used to install `Cilium` by using this role.

```bash
molecule converge -s kvm -- --extra-vars cilium_action=install
molecule converge -- --extra-vars cilium_action=install
```

The following command can be used to install [CoreDNS](https://github.com/githubixx/ansible-kubernetes-playbooks/tree/master/coredns) for Kubernetes DNS stuff and taints controller nodes to only run Cilium pods:

```bash
molecule converge -- --extra-vars cilium_setup_networking=install
```

Upgrading `Cilium` or changing parameters:

```bash
molecule converge -s kvm -- --extra-vars cilium_action=upgrade
molecule converge -- --extra-vars cilium_action=upgrade
```

Deleting `Cilium` and its resources:

```bash
molecule converge -s kvm -- --extra-vars cilium_action=delete
molecule converge -- --extra-vars cilium_action=delete
```

To run a few tests use
To run a few tests use (optionally add `-v` for more output):

```bash
molecule verify -s kvm
molecule verify
```

To clean up run

```bash
molecule destroy -s kvm
molecule destroy
```

License
Expand Down
4 changes: 2 additions & 2 deletions defaults/main.yml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
# Helm chart version (uses Cilium v1.13.4)
cilium_chart_version: "1.13.4"
# Helm chart version (uses Cilium v1.14.1)
cilium_chart_version: "1.14.1"

# Helm release name
cilium_release_name: "cilium"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
# Copyright (C) 2022 Robert Wimmer
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

collections:
- ansible.posix
- kubernetes.core
- community.docker
40 changes: 40 additions & 0 deletions molecule/default/converge.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

- name: Setup Cilium
hosts: k8s_worker
become: true
gather_facts: true
tasks:
- name: Include Cilium role
ansible.builtin.include_role:
name: githubixx.cilium_kubernetes
vars:
cilium_action: "install"

- name: Setup tooling to make worker nodes usable
hosts: test-assets
become: true
gather_facts: true
tasks:
- name: Setup tooling
when:
- cilium_setup_networking is defined
- cilium_setup_networking == "install"
block:
- name: Waiting for Cilium to become ready
ansible.builtin.include_tasks:
file: tasks/cilium_status.yml

- name: Control plane nodes should only run Cilium pods
ansible.builtin.include_tasks:
file: tasks/taint_controller_nodes.yml

- name: Install CoreDNS
ansible.builtin.include_tasks:
file: tasks/coredns.yml

- name: Waiting for CoreDNS to become ready
ansible.builtin.include_tasks:
file: tasks/coredns_status.yml
165 changes: 165 additions & 0 deletions molecule/default/group_vars/all.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,165 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

harden_linux_ntp: "systemd-timesyncd"

# Password for user "root" and "cilium" is "cilium"
harden_linux_root_password: "$6$rounds=656000$mysecretsalt$IPK2dn/YVQ4vSNE7y/KlZOUY1kBo3V0pMDKSwVwrJk2R6s5IeAnoucTR2YcOvttZCjLkg3PuIp8Flx16kLNQy."
harden_linux_deploy_user: "cilium"
harden_linux_deploy_user_password: "$6$rounds=656000$mysecretsalt$IPK2dn/YVQ4vSNE7y/KlZOUY1kBo3V0pMDKSwVwrJk2R6s5IeAnoucTR2YcOvttZCjLkg3PuIp8Flx16kLNQy."
harden_linux_deploy_user_home: "/home/cilium"

harden_linux_sysctl_settings_user:
"net.ipv4.ip_forward": 1
"net.ipv6.conf.default.forwarding": 1
"net.ipv6.conf.all.forwarding": 1

harden_linux_sshd_settings_user:
"^Port ": "Port 22"
"^PasswordAuthentication": "PasswordAuthentication yes"
"^PermitRootLogin": "PermitRootLogin yes"

harden_linux_ufw_rules:
- rule: "allow"
to_port: "22"
protocol: "tcp"
- rule: "allow"
to_port: "51820"
protocol: "udp"
- rule: "allow"
to_port: "80"
protocol: "tcp"
- rule: "allow"
to_port: "443"
protocol: "tcp"
- rule: "allow"
to_port: "25"
protocol: "tcp"

harden_linux_ufw_allow_networks:
- "10.0.0.0/8"
- "172.16.0.0/12"
- "192.168.0.0/16"

harden_linux_ufw_logging: 'on'

harden_linux_ufw_defaults_user:
"^DEFAULT_FORWARD_POLICY": 'DEFAULT_FORWARD_POLICY="ACCEPT"'

harden_linux_sshguard_whitelist:
- "127.0.0.0/8"
- "::1/128"
- "10.0.0.0/8"

etcd_ca_conf_directory: "/tmp/k8s"
etcd_conf_dir: "/etc/etcd"
etcd_interface: "{{ k8s_interface }}"
etcd_settings_user:
"heartbeat-interval": "250"
"election-timeout": "2500"
etcd_cert_hosts:
- localhost
- 127.0.0.1
- 10.32.0.1
- kubernetes
- kubernetes.default
- kubernetes.default.svc
- kubernetes.default.svc.cluster
- kubernetes.svc.cluster.local
- 192.168.10.5
- 192.168.10.10
- 192.168.10.20
- 192.168.10.30
- 192.168.10.100
- 192.168.10.110
- 192.168.10.120
- 192.168.10.130
- 10.10.10.5
- 10.10.10.10
- 10.10.10.20
- 10.10.10.30
- 10.10.10.100
- 10.10.10.110
- 10.10.10.120
- 10.10.10.130
- test-controller1
- test-controller2
- test-controller3
- test-worker1
- test-worker2
- test-worker3

etcd_additional_clients:
- k8s-apiserver-etcd
- cilium

k8s_ca_conf_directory: "/tmp/k8s"
k8s_ca_conf_directory_perm: "0775"
k8s_ca_file_perm: "0666"
k8s_ca_certificate_owner: "{{ k8s_config_owner }}"
k8s_ca_certificate_group: "{{ k8s_config_group }}"

k8s_release: "1.27.5"
k8s_interface: "wg0"
k8s_controller_delegate_to: "test-assets"

k8s_config_cluster_name: "cilium"
k8s_config_directory: "/tmp/k8s"
k8s_config_directory_perm: "0777"
k8s_config_file_perm: "0666"
k8s_config_owner: "root"
k8s_config_group: "root"

k8s_encryption_config_directory: "{{k8s_config_directory}}"
k8s_encryption_config_key: "Y29uZmlndXJhdGlvbjIyCg=="
k8s_encryption_config_owner: "{{ k8s_config_owner }}"
k8s_encryption_config_group: "{{ k8s_config_owner }}"
k8s_encryption_config_directory_perm: "{{ k8s_config_directory_perm }}"
k8s_encryption_config_file_perm: "{{ k8s_config_file_perm }}"

k8s_conf_dir: "/var/lib/kubernetes"

k8s_worker_kubelet_conf_dir: "/var/lib/kubelet"

k8s_apiserver_secure_port: "6443"

k8s_apiserver_settings_user:
"enable-aggregator-routing": "true"

k8s_worker_kubelet_settings:
"config": "{{k8s_worker_kubelet_conf_dir}}/kubelet-config.yaml"
"node-ip": "{{hostvars[inventory_hostname]['ansible_' + k8s_interface].ipv4.address}}"
"kubeconfig": "{{k8s_worker_kubelet_conf_dir}}/kubeconfig"
"seccomp-default": ""

containerd_flavor: "k8s"
containerd_tmp_directory: "/tmp"
containerd_runc_binary_directory: "/usr/local/sbin"
containerd_crictl_config_file: "crictl.yaml"
containerd_crictl_config_directory: "/etc"
containerd_cni_binary_directory: "/opt/cni/bin"

cilium_etcd_enabled: "true"
cilium_delegate_to: "test-assets"
cilium_helm_show_commands: true
cilium_etcd_interface: "{{ k8s_interface }}"
cilium_etcd_client_port: 2379
cilium_etcd_nodes_group: "k8s_etcd"

cilium_etcd_secrets_name: "cilium-etcd-secrets"
cilium_etcd_cert_directory: "{{ k8s_ca_conf_directory }}"
cilium_etcd_cafile: "ca-etcd.pem"
cilium_etcd_certfile: "cert-cilium.pem"
cilium_etcd_keyfile: "cert-cilium-key.pem"

coredns_delegate_to: "test-assets"

ca_etcd_csr_cn: "etcd"
ca_k8s_apiserver_csr_cn: "kubernetes"
etcd_server_csr_cn: "etcd"
etcd_peer_csr_cn: "etcd"
etcd_client_csr_cn_prefix: "etcd"
k8s_apiserver_csr_cn: "kubernetes"
k8s_admin_csr_cn: "admin"
k8s_controller_manager_sa_csr_cn: "service-accounts"
8 changes: 8 additions & 0 deletions molecule/default/host_vars/test-assets.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

wireguard_address: "10.10.10.5/24"
wireguard_port: 51820
wireguard_persistent_keepalive: "30"
wireguard_endpoint: "192.168.10.5"
8 changes: 8 additions & 0 deletions molecule/default/host_vars/test-controller1.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

wireguard_address: "10.10.10.10/24"
wireguard_port: 51820
wireguard_persistent_keepalive: "30"
wireguard_endpoint: "192.168.10.10"
8 changes: 8 additions & 0 deletions molecule/default/host_vars/test-controller2.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

wireguard_address: "10.10.10.20/24"
wireguard_port: 51820
wireguard_persistent_keepalive: "30"
wireguard_endpoint: "192.168.10.20"
8 changes: 8 additions & 0 deletions molecule/default/host_vars/test-controller3.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

wireguard_address: "10.10.10.30/24"
wireguard_port: 51820
wireguard_persistent_keepalive: "30"
wireguard_endpoint: "192.168.10.30"
8 changes: 8 additions & 0 deletions molecule/default/host_vars/test-worker1.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

wireguard_address: "10.10.10.100/24"
wireguard_port: 51820
wireguard_persistent_keepalive: "30"
wireguard_endpoint: "192.168.10.100"
8 changes: 8 additions & 0 deletions molecule/default/host_vars/test-worker2.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
# Copyright (C) 2023 Robert Wimmer
# SPDX-License-Identifier: GPL-3.0-or-later

wireguard_address: "10.10.10.110/24"
wireguard_port: 51820
wireguard_persistent_keepalive: "30"
wireguard_endpoint: "192.168.10.110"
Loading

0 comments on commit 8e2e744

Please sign in to comment.