Skip to content

Commit 93c3a03

Browse files
DEV: (RediSearch) doc expiration capabilities in Redis 8 (#1792)
* DEV: (RediSearch) doc expiration capabilities in Redis 8 * Apply suggestions from code review Co-authored-by: andy-stark-redis <[email protected]> * Apply more suggestions from code review --------- Co-authored-by: andy-stark-redis <[email protected]>
1 parent 5c28113 commit 93c3a03

File tree

6 files changed

+90
-0
lines changed

6 files changed

+90
-0
lines changed

content/commands/expire.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,10 @@ still take the full state of the expires existing in the dataset, so when a
266266
replica is elected to master it will be able to expire the keys independently,
267267
fully acting as a master.
268268

269+
### Redis Query Engine and expiration
270+
271+
Starting with Redis 8, the Redis Query Engine has enhanced behavior when handling expiring keys. For detailed information about how [`FT.SEARCH`]({{< relref "/commands/ft.search" >}}) and [`FT.AGGREGATE`]({{< relref "/commands/ft.aggregate" >}}) commands interact with expiring keys, see [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}}).
272+
269273
## Return information
270274

271275
{{< multitabs id="expire-return-info"

content/commands/ft.aggregate.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -496,5 +496,6 @@ Next, count GitHub events by user (actor), to produce the most active users.
496496
## Related topics
497497

498498
- [Aggregations]({{< relref "/develop/ai/search-and-query/advanced-concepts/aggregations" >}})
499+
- [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}})
499500
- [RediSearch]({{< relref "/develop/ai/search-and-query" >}})
500501

content/commands/ft.search.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -860,5 +860,6 @@ Query with `CONTAINS` operator:
860860

861861
- [Extensions]({{< relref "/develop/ai/search-and-query/administration/extensions" >}})
862862
- [Highlighting]({{< relref "/develop/ai/search-and-query/advanced-concepts/highlight" >}})
863+
- [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}})
863864
- [Query syntax]({{< relref "/develop/ai/search-and-query/query/" >}})
864865
- [RediSearch]({{< relref "/develop/ai/search-and-query/" >}})

content/commands/hexpire.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,10 @@ You can call `HEXPIRE` using as argument a field that already has an
113113
existing TTL set.
114114
In this case, the time to live is _updated_ to the new value.
115115

116+
## Redis Query Engine and field expiration
117+
118+
Starting with Redis 8, the Redis Query Engine has enhanced behavior when handling expiring hash fields. For detailed information about how [`FT.SEARCH`]({{< relref "/commands/ft.search" >}}) and [`FT.AGGREGATE`]({{< relref "/commands/ft.aggregate" >}}) commands interact with expiring hash fields, see [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}}).
119+
116120
## Example
117121

118122
```

content/develop/ai/search-and-query/advanced-concepts/_index.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@ Redis Open Source supports the following Redis Query Engine features. This artic
4242
* Numeric filters and ranges
4343
* Geo-filtering using Redis [geo commands]({{< relref "/commands/" >}}?group=geo)
4444
* [Vector search]({{< relref "/develop/ai/search-and-query/vectors" >}})
45+
* [Key and field expiration behavior]({{< relref "/develop/ai/search-and-query/advanced-concepts/expiration" >}})
4546

4647

4748
## Full-text search features
Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
---
2+
categories:
3+
- docs
4+
- develop
5+
- stack
6+
- oss
7+
- rs
8+
- rc
9+
- oss
10+
- kubernetes
11+
- clients
12+
description: How the Redis Query Engine handles expiring keys and hash fields
13+
linkTitle: Key and field expiration
14+
title: Key and field expiration behavior
15+
weight: 8
16+
---
17+
18+
The Redis Query Engine behavior with expiring keys and hash fields has been enhanced starting with Redis 8 to provide more consistent and predictable results.
19+
20+
## Key expiration
21+
22+
### Expiration times
23+
24+
**Before Redis 8**: Expiration times were not taken into account when computing the result set.
25+
26+
**Redis 8 and later**: The query engine returns only documents that are valid (not expired) at the time when the query or cursor read started.
27+
28+
### Active expiration
29+
30+
Active expiration can affect the number of results returned. For example, if a user requests 100 documents but 10 are actively expired during query execution, only 90 documents will be returned.
31+
32+
**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts.
33+
34+
### Passive expiration
35+
36+
**Before Redis 8**: A query could return `nil` as a document name in the result set for a key that was passively expired.
37+
38+
**Redis 8 and later**: Only valid document names will be returned. Passively expired keys are filtered out from the result set.
39+
40+
## Field expiration
41+
42+
Field expiration was introduced in Redis 7.4 and provides fine-grained control over hash field lifecycles.
43+
44+
### Expiration times
45+
46+
**Before Redis 8**: Field expiration times were not taken into account when computing the result set.
47+
48+
**Redis 8 and later**: The query engine returns only documents that are valid (fields not expired) at the time when the query or cursor read started.
49+
50+
### Active expiration
51+
52+
Similar to key expiration, active field expiration can affect the number of results returned. If fields that match query criteria are actively expired during execution, fewer results will be returned than requested.
53+
54+
**Note**: This behavior did not change in Redis 8 - active expiration has always affected result counts.
55+
56+
### Passive expiration
57+
58+
**Before Redis 8**: The query engine could return documents without fields that were passively expired, even if the expired field caused the document to match the query. This behavior depended to some degree on the `SORTABLE` keyword usage.
59+
60+
**Redis 8 and later**: Documents will return with all fields that existed when the query or cursor read started. Passively expired fields are handled consistently.
61+
62+
## Best practices
63+
64+
* **Consistent timing**: Be aware that query results are based on the state at query start time, not when individual documents are processed.
65+
66+
* **Result count expectations**: Be aware that when you use expiring keys or fields, the actual number of results may be less than you expected due to active expiration during the execution of the query.
67+
68+
* **Field-level expiration**: Use hash field expiration (available since Redis 7.4) for fine-grained control over document field lifecycles without affecting the entire document.
69+
70+
* **Query planning**: Consider expiration patterns when designing queries, especially for time-sensitive applications where expired data should not appear in results.
71+
72+
## Related commands
73+
74+
The following commands are directly related to key and field expiration:
75+
76+
- [`EXPIRE`]({{< relref "/commands/expire" >}}) - Set expiration time for keys
77+
- [`HEXPIRE`]({{< relref "/commands/hexpire" >}}) - Set expiration time for hash fields
78+
- [`FT.SEARCH`]({{< relref "/commands/ft.search" >}}) - Search queries affected by expiration behavior
79+
- [`FT.AGGREGATE`]({{< relref "/commands/ft.aggregate" >}}) - Aggregation queries affected by expiration behavior

0 commit comments

Comments
 (0)