Skip to content

Commit 973f88b

Browse files
committed
Update to localityMappings
1 parent 8c7ec0d commit 973f88b

File tree

1 file changed

+18
-13
lines changed

1 file changed

+18
-13
lines changed

src/current/v25.2/deploy-cockroachdb-with-kubernetes-operator.md

Lines changed: 18 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -42,9 +42,11 @@ If you want to secure your cluster to use TLS certificates for all network commu
4242

4343
### Localities
4444

45-
CockroachDB clusters use locality labels to efficiently distribute replicas. This is especially important in multi-region deployments. In cloud provider deployments (e.g., [GKE](#hosted-gke), [EKS](#hosted-eks), or [AKS](#hosted-aks)), the [`topology.kubernetes.io/region`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesioregion) and [`topology.kubernetes.io/zone`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone) labels are applied implicitly to Kubernetes nodes and populated by the regions and zones specific to the cloud provider. For further granularity, you can define arbitrary locality labels (e.g., `province`, `datacenter`, `rack`), but these need to be applied individually to the Kubernetes node when initialized so that CockroachDB can understand where the node lives and distribute replicas accordingly.
45+
CockroachDB clusters use localities to efficiently distribute replicas. This is especially important in multi-region deployments. With the {{ site.data.products.cockroachdb-operator }}, you specify mappings between locality levels and the location on a Kubernetes node where the value for that locality can be found.
4646

47-
On bare metal Kubernetes deployments, you must plan a hierarchy of locality labels that suit your CockroachDB node distribution, then apply these labels individually to nodes when they are initialized. Although you can set most of these values arbitrarily, you must set region and zone locations in the reserved `topology.kubernetes.io/region` and `topology.kubernetes.io/zone` namespaces, respectively.
47+
In cloud provider deployments (e.g., [GKE](#hosted-gke), [EKS](#hosted-eks), or [AKS](#hosted-aks)), the [`topology.kubernetes.io/region`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesioregion) and [`topology.kubernetes.io/zone`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone) values on Kubernetes nodes are populated by cloud provider. For further granularity, you can define arbitrary locality labels (e.g., `province`, `datacenter`, `rack`), but these need to be applied individually to the Kubernetes node when initialized so that CockroachDB can understand where the node lives and distribute replicas accordingly.
48+
49+
On bare metal Kubernetes deployments, you must plan a hierarchy of localities that suit your CockroachDB node distribution, then apply these values individually to nodes when they are initialized. Although you can set most of these values arbitrarily, you must set region and zone locations in the reserved `topology.kubernetes.io/region` and `topology.kubernetes.io/zone` namespaces, respectively.
4850

4951
For more information on how locality labels are used by CockroachDB, refer to the [`--locality` documentation](cockroach-start.html#locality).
5052

@@ -382,29 +384,32 @@ For bare metal deployments, the specific Kubernetes infrastructure deployment st
382384

383385
For a detailed tutorial of a TLS configuration with manual certificates, refer to [Example: Authenticate with cockroach cert](#example-authenticate-with-cockroach-cert).
384386

385-
1. In `cockroachdb.crdbCluster.localityLabels`, provide [locality labels](#localities) that specify where the locality information of each Kubernetes node is stored. When CockroachDB is initialized on a node, it processes these values as though they are provided through the [`cockroach start --locality`](cockroach-start#locality) flag.
387+
1. In `cockroachdb.crdbCluster.localityMappings`, provide [locality mappings](#localities) that define locality levels and map them to node labels where the locality information of each Kubernetes node is stored. When CockroachDB is initialized on a node, it processes these values as though they are provided through the [`cockroach start --locality`](cockroach-start#locality) flag.
386388

387-
The default configuration uses the `region` and `zone` locality levels, which are set differently depending on the deployment type:
388-
- In cloud provider deployments, the [`topology.kubernetes.io/region`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesioregion) and [`topology.kubernetes.io/zone`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone) locality labels are applied implicitly to Kubernetes nodes and populated by the regions and zones specific to the cloud provider.
389-
- In bare metal deployments, the `topology.kubernetes.io/region` and `topology.kubernetes.io/zone` values are not set implicitly by a cloud provider when initializing the node, so you must set them manually or configure custom locality labels.
389+
The default configuration uses the `region` and `zone` locality labels, mapped implicitly to the [`topology.kubernetes.io/region`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesioregion) and [`topology.kubernetes.io/zone`](https://kubernetes.io/docs/reference/labels-annotations-taints/#topologykubernetesiozone) node labels.
390+
- In cloud provider deployments, the `topology.kubernetes.io/region` and `topology.kubernetes.io/zone` values on a node are populated by the cloud provider.
391+
- In bare metal deployments, the `topology.kubernetes.io/region` and `topology.kubernetes.io/zone` node label values are not set implicitly by a cloud provider when initializing the node, so you must set them manually or configure custom locality labels.
390392

391-
To add more granular levels of locality to your nodes or use different locality labels, add custom locality levels as values in the `cockroachdb.crdbCluster.localityLabels` list. Any custom `localityLabels` configuration overrides the default `region` and `zone` configuration, so if you append an additional locality level but wish to keep the `region` and `zone` labels you must declare them manually.
393+
To add more granular levels of locality to your nodes or use different locality labels, add custom locality levels as values in the `cockroachdb.crdbCluster.localityMappings` list. Any custom `localityMappings` configuration overrides the default `region` and `zone` configuration, so if you append an additional locality level but wish to keep the `region` and `zone` labels you must declare them manually.
392394

393-
The following example uses the existing `region` and `zone` labels and adds an additional `datacenter` locality label that is more granular than `zone`. This example declares that the `datacenter` locality information is stored in the `example.datacenter.locality` variable on the node:
395+
The following example uses the existing `region` and `zone` labels and adds an additional `datacenter` locality mapping that is more granular than `zone`. This example declares that the `dc` locality information is stored in the `example.datacenter.locality` node label:
394396

395397
~~~ yaml
396398
cockroachdb:
397399
crdbCluster:
398-
localityLabels:
399-
- topology.kubernetes.io/region
400-
- topology.kubernetes.io/zone
401-
- example.datacenter.locality
400+
localityMappings:
401+
- nodeLabel: "topology.kubernetes.io/region"
402+
localityLabel: "region"
403+
- nodeLabel: "topology.kubernetes.io/zone"
404+
localityLabel: "zone"
405+
- nodeLabel: "example.datacenter.locality"
406+
localityLabel: "dc"
402407
~~~
403408

404409
In this example, if a Kubernetes node is initialized in the `us-central1` region, `us-central1-c` zone, and `dc2` datacenter, its `cockroach start --locality` flag would be equivalent to the following:
405410

406411
~~~ shell
407-
cockroach start --locality region=us-central1,zone=us-central1-c,example.datacenter.locality=dc2
412+
cockroach start --locality region=us-central1,zone=us-central1-c,dc=dc2
408413
~~~
409414

410415
Optionally, review the `cockroachdb.crdbCluster.topologySpreadConstraints` configuration and set `topologyKey` to a locality variable that will have distinct values for each node. By default the lowest locality level is `zone`, so the following configuration sets that value as the `topologyKey`:

0 commit comments

Comments
 (0)