fix(cache): Estimate size of posting lists

[xqqp]

Description

The maximum cost of the Ristretto cache is set to a number of megabytes, but when inserting a posting list item into the cache, the cost is set to 1. Because of this the cache only grows and no items get evicted. This results in high memory consumption, especially with the new UID cache (see #9513).

This PR introduces a cost function which estimates the memory size of each item. For more details see predictable-labs#6. Credits to @darkcoderrises for the implementation.

The default cache size is changed from 1024 to 4096, to reflect the more accurate cost estimation. See below for how the size of the cache relates to the occupied memory by the Dgrpah process, when this PR is applied.

size-mb=1024

Dgraph process occupies up to 3.7 GiB of memory

heap:

  344.49MB 30.34% 30.34%   407.26MB 35.87%  github.com/hypermodeinc/dgraph/v25/posting.(*List).calculateUids
  130.37MB 11.48% 41.83%   130.37MB 11.48%  github.com/dgraph-io/ristretto/v2.newCmRow
   83.20MB  7.33% 49.15%    83.20MB  7.33%  github.com/dgraph-io/badger/v4/skl.newArena
   69.50MB  6.12% 55.28%    69.50MB  6.12%  github.com/hypermodeinc/dgraph/v25/posting.(*List).Uids.func1
   65.16MB  5.74% 61.02%    65.16MB  5.74%  github.com/dgraph-io/ristretto/v2/z.(*Bloom).Size
   53.27MB  4.69% 65.71%    53.27MB  4.69%  github.com/hypermodeinc/dgraph/v25/posting.(*List).calculateUids.func1

size-mb=2048

Dgraph process occupies up to 5.4 GiB of memory

heap:

  637.07MB 33.68% 33.68%   699.33MB 36.97%  github.com/hypermodeinc/dgraph/v25/posting.(*List).calculateUids
  260.63MB 13.78% 47.46%   260.63MB 13.78%  github.com/dgraph-io/ristretto/v2.newCmRow
  130.04MB  6.88% 54.34%   130.04MB  6.88%  github.com/dgraph-io/ristretto/v2/z.(*Bloom).Size
   99.01MB  5.23% 59.57%    99.01MB  5.23%  github.com/hypermodeinc/dgraph/v25/posting.(*List).Uids.func1
   89.89MB  4.75% 64.32%    89.89MB  4.75%  github.com/dgraph-io/ristretto/v2.(*lockedMap[go.shape.*uint8]).Set
   83.20MB  4.40% 68.72%    83.20MB  4.40%  github.com/dgraph-io/badger/v4/skl.newArena

size-mb=4096

Dgraph process occupies up to 8.4 GiB of memory

heap:

 1343.44MB 41.61% 41.61%  1404.71MB 43.51%  github.com/hypermodeinc/dgraph/v25/posting.(*List).calculateUids
  520.15MB 16.11% 57.72%   520.15MB 16.11%  github.com/dgraph-io/ristretto/v2.newCmRow
     260MB  8.05% 65.77%      260MB  8.05%  github.com/dgraph-io/ristretto/v2/z.(*Bloom).Size
  136.70MB  4.23% 70.01%   136.70MB  4.23%  github.com/dgraph-io/ristretto/v2.(*lockedMap[go.shape.*uint8]).Set
  106.51MB  3.30% 73.31%   106.51MB  3.30%  reflect.New
   83.20MB  2.58% 75.88%    83.20MB  2.58%  github.com/dgraph-io/badger/v4/skl.newArena

size-mb=8192

Dgraph process occupies up to 13.3 GiB of memory

heap:

 2539.78MB 45.10% 45.10%  2601.10MB 46.19%  github.com/hypermodeinc/dgraph/v25/posting.(*List).calculateUids
 1040.01MB 18.47% 63.56%  1040.01MB 18.47%  github.com/dgraph-io/ristretto/v2.newCmRow
     520MB  9.23% 72.80%      520MB  9.23%  github.com/dgraph-io/ristretto/v2/z.(*Bloom).Size
  223.52MB  3.97% 76.77%   223.52MB  3.97%  reflect.New
  157.97MB  2.81% 79.57%   157.97MB  2.81%  github.com/dgraph-io/ristretto/v2.(*lockedMap[go.shape.*uint8]).Set
  136.01MB  2.42% 81.99%   259.03MB  4.60%  github.com/hypermodeinc/dgraph/v25/posting.copyList

Closes #9513

[xqqp]

Test failures seem unrelated to this change. @matthewmcneely when you have time please take a look, without this patch v25 is unusable for me, because memory quickly fills up.

[matthewmcneely]

Test failures seem unrelated to this change. @matthewmcneely when you have time please take a look, without this patch v25 is unusable for me, because memory quickly fills up.

@xqqp Yeah, the shared runners we are now using don't seem to be up to the task. We're working on it.