Remove support for chunks storage entirely: Take 2 (#4812)

* Fix updateCachedShippedBlocks - new thanos

Signed-off-by: Alan Protasio <[email protected]>

* Remove support for chunks ingestion

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove blocksconvert

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove most all of chunks storage

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove old legacy object store client and uses

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Start removing chunks docs

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove HTTP APIs that were chunks-only

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove more chunks docs

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove deprecated and chunks configs

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Draft changelog

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Fix up integration tests

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Clean up lint warnings

Signed-off-by: Andrew Bloomgarden <[email protected]>

* go mod tidy && go mod vendor

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove chunks storage from bug report template

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove most chunk encoding/decoding code

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Remove unused cardinality limit flag

Signed-off-by: Andrew Bloomgarden <[email protected]>

* Document removed flags in the CHANGELOG

Signed-off-by: Andrew Bloomgarden <[email protected]>

* rebasing

Signed-off-by: Alan Protasio <[email protected]>

* Fix Ingester Proto

Signed-off-by: Alan Protasio <[email protected]>

* Removing chunk storage config

Signed-off-by: Alan Protasio <[email protected]>

* Removing tsdb_test.go

Signed-off-by: Alan Protasio <[email protected]>

* go mod

Signed-off-by: Alan Protasio <[email protected]>

* Removing cassandra referenves from the integ tests

Signed-off-by: Alan Protasio <[email protected]>

* Keep MIgrating Storage Documentation

Signed-off-by: Alan Protasio <[email protected]>

* Removing dangling chunks config

Signed-off-by: Alan Protasio <[email protected]>

* Update Changelog

Signed-off-by: Alan Protasio <[email protected]>

* Removing blocks convert from gitignore and makfile local

Signed-off-by: Alan Protasio <[email protected]>

* Remove dead code

Signed-off-by: Alan Protasio <[email protected]>

Signed-off-by: Alan Protasio <[email protected]>
Signed-off-by: Andrew Bloomgarden <[email protected]>
Co-authored-by: Andrew Bloomgarden <[email protected]>
Co-authored-by: Alvin Lin <[email protected]>

Author: 3 people

.github/ISSUE_TEMPLATE/bug_report.md

@@ -22,9 +22,5 @@ A clear and concise description of what you expected to happen.
  - Infrastructure: [e.g., Kubernetes, bare-metal, laptop]
  - Deployment tool: [e.g., helm, jsonnet]
 
-**Storage Engine**
-- [ ] Blocks
-- [ ] Chunks
-
 **Additional Context**
 <!--  Additional relevant info which can help us debug this issue easily like Logs, Configuration etc. -->

.github/workflows/test-build-deploy.yml

@@ -35,13 +35,6 @@ jobs:
     runs-on: ubuntu-20.04
     container:
       image: quay.io/cortexproject/build-image:update-hugo-0.94.x-96d1d7cdd
-    services:
-      cassandra:
-        image: cassandra:3.11
-        env:
-          JVM_OPTS: "-Xms1024M -Xmx1024M"
-        ports:
-          - 9042:9042
     steps:
       - name: Checkout Repo
         uses: actions/checkout@v2
@@ -50,7 +43,7 @@ jobs:
           mkdir -p /go/src/github.com/cortexproject/cortex
           ln -s $GITHUB_WORKSPACE/* /go/src/github.com/cortexproject/cortex
       - name: Run Tests
-        run: CASSANDRA_TEST_ADDRESSES=cassandra:9042 make BUILD_IN_CONTAINER=false test
+        run: make BUILD_IN_CONTAINER=false test
 
   build:
     runs-on: ubuntu-20.04
@@ -138,7 +131,6 @@ jobs:
           docker pull quay.io/cortexproject/cortex:v1.9.0
           docker pull quay.io/cortexproject/cortex:v1.10.0
           docker pull shopify/bigtable-emulator:0.1.0
-          docker pull rinscy/cassandra:3.11.0
           docker pull memcached:1.6.1
           docker pull bouncestorage/swift-aio:55ba4331
       - name: Integration Tests

.gitignore

@@ -1,7 +1,6 @@
 cmd/test-exporter/test-exporter
 cmd/cortex/cortex
 cmd/query-tee/query-tee
-cmd/blocksconvert/blocksconvert
 cmd/thanosconvert/thanosconvert
 .uptodate
 .pkg

CHANGELOG.md

@@ -1,15 +1,49 @@
 # Changelog
 
 ## master / unreleased
+  **This release removes support for chunks storage. See below for more.**
+* [CHANGE] Remove support for chunks storage entirely. If you are using chunks storage on a previous version, you must [migrate your data](https://github.com/cortexproject/cortex/blob/v1.11.1/docs/blocks-storage/migrate-from-chunks-to-blocks.md) on version 1.12 or earlier. Before upgrading to this release, you should also remove any deprecated chunks-related configuration, as this release will no longer accept that. The following flags are gone:
+  - `-dynamodb.*`
+  - `-metrics.*`
+  - `-s3.*`
+  - `-azure.*`
+  - `-bigtable.*`
+  - `-gcs.*`
+  - `-cassandra.*`
+  - `-boltdb.*`
+  - `-local.*`
+  - some `-ingester` flags:
+    - `-ingester.wal-enabled`
+    - `-ingester.checkpoint-enabled`
+    - `-ingester.recover-from-wal`
+    - `-ingester.wal-dir`
+    - `-ingester.checkpoint-duration`
+    - `-ingester.flush-on-shutdown-with-wal-enabled`
+    - `-ingester.max-transfer-retries`
+    - `-ingester.max-samples-per-query`
+    - `-ingester.min-chunk-length`
+    - `-ingester.flush-period`
+    - `-ingester.retain-period`
+    - `-ingester.max-chunk-idle`
+    - `-ingester.max-stale-chunk-idle`
+    - `-ingester.flush-op-timeout`
+    - `-ingester.max-chunk-age`
+    - `-ingester.chunk-age-jitter`
+    - `-ingester.concurrent-flushes`
+    - `-ingester.spread-flushes`
+    - `-store.*` except `-store.engine` and `-store.max-query-length`
+    - `-store.query-chunk-limit` was deprecated and replaced by `-querier.max-fetched-chunks-per-query`
+  - `-deletes.*`
+  - `-grpc-store.*`
+  - `-flusher.wal-dir`, `-flusher.concurrent-flushes`, `-flusher.flush-op-timeout`
+* [CHANGE] Remove support for alertmanager and ruler legacy store configuration. Before upgrading, you need to convert your configuration to use the `alertmanager-storage` and `ruler-storage` configuration on the version that you're already running, then upgrade.
 * [ENHANCEMENT] Querier/Ruler: Retry store-gateway in case of unexpected failure, instead of failing the query. #4532
 * [ENHANCEMENT] Ring: DoBatch prioritize 4xx errors when failing. #4783
 * [FEATURE] Compactor: Added `-compactor.block-files-concurrency` allowing to configure number of go routines for download/upload block files during compaction. #4784
 * [FEATURE] Compactor: Added -compactor.blocks-fetch-concurrency` allowing to configure number of go routines for blocks during compaction. #4787
 * [FEATURE] Compactor: Added configurations for Azure MSI in blocks-storage, ruler-storage and alertmanager-storage. #4818
 * [BUGFIX] Memberlist: Add join with no retrying when starting service. #4804
 
-
-
 ## 1.13.0 2022-07-14
 
 * [CHANGE] Changed default for `-ingester.min-ready-duration` from 1 minute to 15 seconds. #4539

Makefile

@@ -96,15 +96,12 @@ pkg/frontend/v1/frontendv1pb/frontend.pb.go: pkg/frontend/v1/frontendv1pb/fronte
 pkg/frontend/v2/frontendv2pb/frontend.pb.go: pkg/frontend/v2/frontendv2pb/frontend.proto
 pkg/querier/queryrange/queryrange.pb.go: pkg/querier/queryrange/queryrange.proto
 pkg/querier/stats/stats.pb.go: pkg/querier/stats/stats.proto
-pkg/chunk/storage/caching_index_client.pb.go: pkg/chunk/storage/caching_index_client.proto
 pkg/distributor/ha_tracker.pb.go: pkg/distributor/ha_tracker.proto
 pkg/ruler/rulespb/rules.pb.go: pkg/ruler/rulespb/rules.proto
 pkg/ruler/ruler.pb.go: pkg/ruler/ruler.proto
 pkg/ring/kv/memberlist/kv.pb.go: pkg/ring/kv/memberlist/kv.proto
 pkg/scheduler/schedulerpb/scheduler.pb.go: pkg/scheduler/schedulerpb/scheduler.proto
 pkg/storegateway/storegatewaypb/gateway.pb.go: pkg/storegateway/storegatewaypb/gateway.proto
-pkg/chunk/grpc/grpc.pb.go: pkg/chunk/grpc/grpc.proto
-tools/blocksconvert/scheduler.pb.go: tools/blocksconvert/scheduler.proto
 pkg/alertmanager/alertmanagerpb/alertmanager.pb.go: pkg/alertmanager/alertmanagerpb/alertmanager.proto
 pkg/alertmanager/alertspb/alerts.pb.go: pkg/alertmanager/alertspb/alerts.proto
 
@@ -359,7 +356,7 @@ dist/$(UPTODATE)-packages: dist $(wildcard packaging/deb/**) $(wildcard packagin
 			--before-remove packaging/deb/control/prerm \
 			--package dist/cortex-$(VERSION)_$$arch.deb \
 			dist/cortex-linux-$$arch=/usr/local/bin/cortex \
-			docs/chunks-storage/single-process-config.yaml=/etc/cortex/single-process-config.yaml \
+			docs/configuration/single-process-config-blocks.yaml=/etc/cortex/single-process-config.yaml \
 			packaging/deb/default/cortex=/etc/default/cortex \
 			packaging/deb/systemd/cortex.service=/etc/systemd/system/cortex.service; \
 		$(FPM_OPTS) -t rpm  \
@@ -368,7 +365,7 @@ dist/$(UPTODATE)-packages: dist $(wildcard packaging/deb/**) $(wildcard packagin
 			--before-remove packaging/rpm/control/preun \
 			--package dist/cortex-$(VERSION)_$$arch.rpm \
 			dist/cortex-linux-$$arch=/usr/local/bin/cortex \
-			docs/chunks-storage/single-process-config.yaml=/etc/cortex/single-process-config.yaml \
+			docs/configuration/single-process-config-blocks.yaml=/etc/cortex/single-process-config.yaml \
 			packaging/rpm/sysconfig/cortex=/etc/sysconfig/cortex \
 			packaging/rpm/systemd/cortex.service=/etc/systemd/system/cortex.service; \
 	done

Makefile.local.example

@@ -3,8 +3,5 @@
 BUILD_IMAGE ?= quay.io/cortexproject/build-image
 IMAGE_PREFIX ?= custom-prefix/
 
-blocksconvert-push: cmd/blocksconvert/.uptodate
-	docker push $(IMAGE_PREFIX)blocksconvert:$(IMAGE_TAG)
-
 cortex-push: cmd/cortex/.uptodate
 	docker push $(IMAGE_PREFIX)cortex:$(IMAGE_TAG)

cmd/blocksconvert/Dockerfile

@@ -28,7 +28,7 @@ For the sake of clarity, in this document we have grouped API endpoints by servi
 | [Remote write](#remote-write) | Distributor | `POST /api/v1/push` |
 | [Tenants stats](#tenants-stats) | Distributor | `GET /distributor/all_user_stats` |
 | [HA tracker status](#ha-tracker-status) | Distributor | `GET /distributor/ha_tracker` |
-| [Flush chunks / blocks](#flush-chunks--blocks) | Ingester | `GET,POST /ingester/flush` |
+| [Flush blocks](#flush-blocks) | Ingester | `GET,POST /ingester/flush` |
 | [Shutdown](#shutdown) | Ingester | `GET,POST /ingester/shutdown` |
 | [Ingesters ring status](#ingesters-ring-status) | Ingester | `GET /ingester/ring` |
 | [Instant query](#instant-query) | Querier, Query-frontend | `GET,POST <prometheus-http-prefix>/api/v1/query` |
@@ -40,7 +40,6 @@ For the sake of clarity, in this document we have grouped API endpoints by servi
 | [Get metric metadata](#get-metric-metadata) | Querier, Query-frontend | `GET <prometheus-http-prefix>/api/v1/metadata` |
 | [Remote read](#remote-read) | Querier, Query-frontend | `POST <prometheus-http-prefix>/api/v1/read` |
 | [Get tenant ingestion stats](#get-tenant-ingestion-stats) | Querier | `GET /api/v1/user_stats` |
-| [Get tenant chunks](#get-tenant-chunks) | Querier | `GET /api/v1/chunks` |
 | [Ruler ring status](#ruler-ring-status) | Ruler | `GET /ruler/ring` |
 | [Ruler rules ](#ruler-rule-groups) | Ruler | `GET /ruler/rule_groups` |
 | [List rules](#list-rules) | Ruler | `GET <prometheus-http-prefix>/api/v1/rules` |
@@ -60,9 +59,6 @@ For the sake of clarity, in this document we have grouped API endpoints by servi
 | [Get Alertmanager configuration](#get-alertmanager-configuration) | Alertmanager | `GET /api/v1/alerts` |
 | [Set Alertmanager configuration](#set-alertmanager-configuration) | Alertmanager | `POST /api/v1/alerts` |
 | [Delete Alertmanager configuration](#delete-alertmanager-configuration) | Alertmanager | `DELETE /api/v1/alerts` |
-| [Delete series](#delete-series) | Purger | `PUT,POST <prometheus-http-prefix>/api/v1/admin/tsdb/delete_series` |
-| [List delete requests](#list-delete-requests) | Purger | `GET <prometheus-http-prefix>/api/v1/admin/tsdb/delete_series` |
-| [Cancel delete request](#cancel-delete-request) | Purger | `PUT,POST <prometheus-http-prefix>/api/v1/admin/tsdb/cancel_delete_request` |
 | [Tenant delete request](#tenant-delete-request) | Purger | `POST /purger/delete_tenant` |
 | [Tenant delete status](#tenant-delete-status) | Purger | `GET /purger/delete_tenant_status` |
 | [Store-gateway ring status](#store-gateway-ring-status) | Store-gateway | `GET /store-gateway/ring` |
@@ -247,7 +243,7 @@ Displays a web page with the current status of the HA tracker, including the ele
 
 ## Ingester
 
-### Flush chunks / blocks
+### Flush blocks
 
 ```
 GET,POST /ingester/flush
@@ -256,11 +252,11 @@ GET,POST /ingester/flush
 GET,POST /flush
 ```
 
-Triggers a flush of the in-memory time series data (chunks or blocks) to the long-term storage. This endpoint triggers the flush also when `-ingester.flush-on-shutdown-with-wal-enabled` or `-blocks-storage.tsdb.flush-blocks-on-shutdown` are disabled.
+Triggers a flush of the in-memory time series data to the long-term storage. This endpoint triggers the flush also when `-ingester.flush-on-shutdown-with-wal-enabled` or `-blocks-storage.tsdb.flush-blocks-on-shutdown` are disabled.
 
-When using blocks storage, this endpoint accepts `tenant` parameter to specify tenant whose blocks are compacted and shipped. This parameter may be specified multiple times to select more tenants. If no tenant is specified, all tenants are flushed.
+This endpoint accepts `tenant` parameter to specify tenant whose blocks are compacted and shipped. This parameter may be specified multiple times to select more tenants. If no tenant is specified, all tenants are flushed.
 
-Flush endpoint now also accepts `wait=true` parameter, which makes the call synchronous – it will only return after flushing has finished. Note that returned status code does not reflect the result of flush operation. This parameter is only available when using blocks storage.
+Flush endpoint now also accepts `wait=true` parameter, which makes the call synchronous – it will only return after flushing has finished. Note that returned status code does not reflect the result of flush operation.
 
 ### Shutdown
 
@@ -427,25 +423,6 @@ Returns realtime ingestion rate, for the authenticated tenant, in `JSON` format.
 
 _Requires [authentication](#authentication)._
 
-### Get tenant chunks
-
-```
-GET /api/v1/chunks
-
-# Legacy
-GET <legacy-http-prefix>/chunks
-```
-
-Fetch a compressed tar of all the chunks containing samples for the given time range and label matchers. This endpoint is supported only by the **chunks storage**, requires `-querier.ingester-streaming=true` and should **not be exposed to users** but just used for debugging purposes.
-
-| URL query parameter | Description |
-| ------------------- | ----------- |
-| `start` | Start timestamp, in RFC3339 format or unix epoch. |
-| `end` | End timestamp, in RFC3339 format or unix epoch. |
-| `matcher` | Label matcher that selects the series for which chunks should be fetched. |
-
-_Requires [authentication](#authentication)._
-
 ## Ruler
 
 The ruler API endpoints require to configure a backend object storage to store the recording rules and alerts. The ruler API uses the concept of a "namespace" when creating rule groups. This is a stand in for the name of the rule file in Prometheus and rule groups must be named uniquely within a namespace.
@@ -810,52 +787,7 @@ _Requires [authentication](#authentication)._
 
 ## Purger
 
-The Purger service provides APIs for requesting deletion of series in chunks storage and managing delete requests. For more information about it, please read the [Delete series Guide](../guides/deleting-series.md).
-
-### Delete series
-
-```
-PUT,POST <prometheus-http-prefix>/api/v1/admin/tsdb/delete_series
-
-# Legacy
-PUT,POST <legacy-http-prefix>/api/v1/admin/tsdb/delete_series
-```
-
-Prometheus-compatible delete series endpoint.
-
-_For more information, please check out the Prometheus [delete series](https://prometheus.io/docs/prometheus/latest/querying/api/#delete-series) documentation._
-
-_Requires [authentication](#authentication)._
-
-### List delete requests
-
-```
-GET <prometheus-http-prefix>/api/v1/admin/tsdb/delete_series
-
-# Legacy
-GET <legacy-http-prefix>/api/v1/admin/tsdb/delete_series
-```
-
-List all the delete requests.
-
-_Requires [authentication](#authentication)._
-
-### Cancel delete request
-
-```
-PUT,POST <prometheus-http-prefix>/api/v1/admin/tsdb/cancel_delete_request
-
-# Legacy
-PUT,POST <legacy-http-prefix>/api/v1/admin/tsdb/cancel_delete_request
-```
-
-Cancel a delete request while the request is still in the grace period (before the request is effectively processed by the purger and time series data is hard-deleted from the storage).
-
-| URL query parameter | Description |
-| ------------------- | ----------- |
-| `request_id` | Deletion request ID to cancel. Can be obtained by the [List delete requests](#list-delete-requests) endpoint. |
-
-_Requires [authentication](#authentication)._
+The Purger service provides APIs for requesting deletion of tenants.
 
 ### Tenant Delete Request