Skip to content

DEV: update examples on two JSON pages #1305

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Apr 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions content/commands/json.debug-memory/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ Get the values' memory usage in bytes.

{{< highlight bash >}}
redis> JSON.DEBUG MEMORY item:2
(integer) 253
(integer) 573
{{< / highlight >}}
</details>

Expand All @@ -81,4 +81,3 @@ redis> JSON.DEBUG MEMORY item:2

* [RedisJSON]({{< relref "/develop/data-types/json/" >}})
* [Index and search JSON documents]({{< relref "/develop/interact/search-and-query/indexing/" >}})

102 changes: 53 additions & 49 deletions content/develop/data-types/json/ram.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,7 @@ categories:
- oss
- kubernetes
- clients
description: 'Debugging memory consumption

'
description: Debugging memory consumption
linkTitle: Memory Usage
title: Redis JSON RAM Usage
weight: 6
Expand All @@ -21,96 +19,102 @@ Every key in Redis takes memory and requires at least the amount of RAM to store
well as some per-key overhead that Redis uses. On top of that, the value in the key also requires
RAM.

Redis JSON stores JSON values as binary data after deserializing them. This representation is often more
expensive, size-wise, than the serialized form. The JSON data type uses at least 24 bytes (on
64-bit architectures) for every value, as can be seen by sampling an empty string with the
[`JSON.DEBUG MEMORY`]({{< relref "commands/json.debug-memory/" >}}) command:
Redis JSON stores JSON values as binary data after deserialization. This representation is often more
expensive, size-wise, than the serialized form. All JSON values occupy at least 8 bytes (on 64-bit architectures) because each is represented as a thin wrapper around a pointer. The type information is stored in the lower bits of the pointer, which are guaranteed to be zero due to alignment restrictions. This allows those bits to be repurposed to store some auxiliary data.

For some types of JSON values, 8 bytes is all that’s needed. Nulls and booleans don’t require any additional storage. Small integers are stored in static memory because they’re frequently used, so they also use only the initial 8 bytes. Similarly, empty strings, arrays, and objects don’t require any bookkeeping. Instead, they point to static instances of a _null_ string, array, or object. Here are some examples that use the [JSON.DEBUG MEMORY]({{< relref "/commands/json.debug-memory" >}}) command to report on memory consumption:

```
127.0.0.1:6379> JSON.SET boolean . 'true'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY boolean
(integer) 8

127.0.0.1:6379> JSON.SET null . null
OK
127.0.0.1:6379> JSON.DEBUG MEMORY null
(integer) 8

127.0.0.1:6379> JSON.SET emptystring . '""'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY emptystring
(integer) 24
```
(integer) 8

This RAM requirement is the same for all scalar values, but strings require additional space
depending on their actual length. For example, a 3-character string will use 3 additional bytes:
127.0.0.1:6379> JSON.SET emptyarr . '[]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY emptyarr
(integer) 8

```
127.0.0.1:6379> JSON.SET foo . '"bar"'
127.0.0.1:6379> JSON.SET emptyobj . '{}'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY foo
(integer) 27
127.0.0.1:6379> JSON.DEBUG MEMORY emptyobj
(integer) 8
```

Empty containers take up 32 bytes to set up:
This RAM requirement is the same for all scalar values, but strings require additional space
depending on their length. For example, a 3-character string will use 3 additional bytes:

```
127.0.0.1:6379> JSON.SET arr . '[]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 32
127.0.0.1:6379> JSON.SET obj . '{}'
127.0.0.1:6379> JSON.SET foo . '"bar"'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY obj
(integer) 32
127.0.0.1:6379> JSON.DEBUG MEMORY foo
(integer) 11
```

The actual size of a container is the sum of sizes of all items in it on top of its own
overhead. To avoid expensive memory reallocations, containers' capacity is scaled by multiples of 2
until a treshold size is reached, from which they grow by fixed chunks.
In the following four examples, each array requires 56 bytes. This breaks down as:
- 8 bytes for the initial array value pointer
- 16 bytes of metadata: 8 bytes for the allocated capacity and 8 bytes for the point-in-time size of the array
- 32 bytes for the array. The initial capacity of an array is 4. Therefore, the calculation is `4 * 8` bytes

A container with a single scalar is made up of 32 and 24 bytes, respectively:
```
127.0.0.1:6379> JSON.SET arr . '[""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 56
```

A container with two scalars requires 40 bytes for the container (each pointer to an entry in the
container is 8 bytes), and 2 * 24 bytes for the values themselves:
```
127.0.0.1:6379> JSON.SET arr . '["", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 88
(integer) 56
```

A 3-item (each 24 bytes) container will be allocated with capacity for 4 items, i.e. 56 bytes:

```
127.0.0.1:6379> JSON.SET arr . '["", "", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 128
(integer) 56
```

The next item will not require an allocation in the container, so usage will increase only by that
scalar's requirement, but another value will scale the container again:

```
127.0.0.1:6379> JSON.SET arr . '["", "", "", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 152
(integer) 56
```

Once the current capacity is insufficient to fit a new value, the array reallocates to double its capacity. An array with 5 elements will have a capacity of 8, therefore consuming `8 + 16 + 8 * 8 = 88` bytes.

```
127.0.0.1:6379> JSON.SET arr . '["", "", "", "", ""]'
OK
127.0.0.1:6379> JSON.DEBUG MEMORY arr
(integer) 208
(integer) 88
```

This table gives the size (in bytes) of a few of the test files on disk and when stored using
JSON. The _MessagePack_ column is for reference purposes and reflects the length of the value
when stored using MessagePack.

| File | Filesize | Redis JSON | MessagePack |
| -------------------------------------- | --------- | ------ | ----------- |
| /tests/files/pass-100.json | 380 | 1079 | 140 |
| /tests/files/pass-jsonsl-1.json | 1441 | 3666 | 753 |
| /tests/files/pass-json-parser-0000.json | 3468 | 7209 | 2393 |
| /tests/files/pass-jsonsl-yahoo2.json | 18446 | 37469 | 16869 |
| /tests/files/pass-jsonsl-yelp.json | 39491 | 75341 | 35469 |
Because reallocation operations can be expensive, Redis grows JSON arrays geometrically rather than linearly. This approach spreads the cost across many insertions.

This table gives the size (in bytes) of a few of the test files from the [module repo](https://github.com/RedisJSON/RedisJSON/tree/master/tests/files), stored using
JSON. The _MessagePack_ column is for reference purposes and reflects the length of the value when stored using [MessagePack](https://msgpack.org/index.html).

| File | File size | Redis JSON | MessagePack |
| --------------------------------------- | --------- | ---------- | ----------- |
| /tests/files/pass-100.json | 381 | 1069 | 140 |
| /tests/files/pass-jsonsl-1.json | 1387 | 2190 | 757 |
| /tests/files/pass-json-parser-0000.json | 3718 | 5469 | 2393 |
| /tests/files/pass-jsonsl-yahoo2.json | 22466 | 26901 | 16869 |
| /tests/files/pass-jsonsl-yelp.json | 46333 | 57513 | 35529 |

> Note: In the current version, deleting values from containers **does not** free the container's
allocated memory.