Sync to version 3.20.7

Author: Oldes

CHANGES.md

@@ -3,6 +3,57 @@
 This is just generated output from commits in [this repository](https://github.com/Oldes/Rebol3). For full log use [GitHub commits](https://github.com/Oldes/Rebol3/commits/master).
 
 
+## 2025 December
+
+### Features:
+
+* [`7df1d4`](https://github.com/Oldes/Rebol3/commit/7df1d445e4e73591f16b4f1720d6a38649f97c24) Add `recycle/pool` to free empty memory pool segments
+* [`80a468`](https://github.com/Oldes/Rebol3/commit/80a4683c10306245314d1c0cf961ea5b8d95d508) Add optional LZ4 de/compression
+
+### Changes:
+
+* [`bc42bf`](https://github.com/Oldes/Rebol3/commit/bc42bfc0f35f56209ade1a77d8b4fb10eb1132ad) Refactoring of SYS-LOAD and SYS-BASE (#153)
+
+  No substantive changes—intent is to better elucidate the LOAD and DO processes with an eye to review and modification. Where possible, word names are expanded and reflect what each value represents, not what they are.
+* [`ce497a`](https://github.com/Oldes/Rebol3/commit/ce497af725f018b6ea5e2ca439d9def16b69218d) Update link to the latest MiniAudio extension (version 0.11.23)
+* [`bf8dbc`](https://github.com/Oldes/Rebol3/commit/bf8dbcbdf685d3cd0654d868fe9d269f5812f314) Using `libdeflate` sources for `deflate`, `zlib` and `gzip` compressions
+
+  Here are test results from my notebook:
+  ```
+  c:\Dev\Builder\tree\rebol\Rebol\src\tests>rebol_3.20.7 -s test-deflate.r3
+  Rebol/version: 3.20.7
+  Data length: 1039872
+  Test: [checksum data 'crc32]
+  Time: 0.000112
+  Test: [checksum data 'adler32]
+  Time: 0.000077
+  Test: [compress/level data 'deflate 9]
+  Time: 0.028716
+  
+  c:\Dev\Builder\tree\rebol\Rebol\src\tests>rebol_3.20.0 -s test-deflate.r3
+  Rebol/version: 3.20.0
+  Data length: 1039872
+  Test: [checksum data 'crc32]
+  Time: 0.003945
+  Test: [checksum data 'adler32]
+  Time: 0.000667
+  Test: [compress/level data 'deflate 9]
+  Time: 0.076221
+  ```
+* [`f14224`](https://github.com/Oldes/Rebol3/commit/f14224009b0cc7313f2f99444a4e9aabc525b34f) Gracefully release memory in debug builds to help detect leaks
+
+  - Gracefully disposes all memory only when `quit` is used, and only in builds compiled with the `DEBUG` define.
+  - Some guru meditation `evoke` chants are available only in `DEBUG` builds.
+  - `recycle` now returns the number of bytes released from the heap (memory released to pools is not reported).
+  - `stats` returns the total number of bytes currently allocated from the heap.
+  - In debug builds, it is possible to monitor memory allocations using  `evoke 'watch-alloc`
+
+### Fixes:
+
+* [`07bfe8`](https://github.com/Oldes/Rebol3/commit/07bfe85740b43f44e55fc2715a7d73f2e43bd79b) Properly parse double quotes in Unicode escape sequences (#154)
+* [`25ba57`](https://github.com/Oldes/Rebol3/commit/25ba5731cef1bb8b96c8f073f8e13563f7de7e6d) Do not dispose cryptography when it is not included
+
+
 
 ## 2025 November
 
@@ -14,12 +65,16 @@ This is just generated output from commits in [this repository](https://github.c
 
 ### Changes:
 
+* [`d34ea4`](https://github.com/Oldes/Rebol3/commit/d34ea4d317bc87f5a388a64daae9f5c3edf3dfc5) Update xxhash source to version 0.8.3
+* [`a7d7e3`](https://github.com/Oldes/Rebol3/commit/a7d7e304c41ef6ea1e5577b34ca9a48165b346f3) Additional fixes related to Register_Compress_Method integration
 * [`e349a7`](https://github.com/Oldes/Rebol3/commit/e349a7e29136db1e1300944762e4b5b0b5a837ec) Initialize optional compression methods using Register_Compress_Method
 * [`57b590`](https://github.com/Oldes/Rebol3/commit/57b590799344f467abea5ca69491131ca7a70e29) Update Brotli source to version 1.2.0
 * [`76425d`](https://github.com/Oldes/Rebol3/commit/76425d1e31d776162741e8e81c39da584ad6e84c) Unify definitions of endianness-related macros
 
 ### Fixes:
 
+* [`683095`](https://github.com/Oldes/Rebol3/commit/683095bc2e86a77bb17dd2742e368b4c5b77f971) Handle TLS handshake record received in multiple chunks
+* [`da4162`](https://github.com/Oldes/Rebol3/commit/da416289d1322082e607265c541470caf79d6f98) Pass none as the error argument when an extension returns an error without a message
 * [`47fd36`](https://github.com/Oldes/Rebol3/commit/47fd3671a61a11c4975cbff6dabfdc79e1134eaa) Validate handle data is not NULL before releasing
 * [`c8397b`](https://github.com/Oldes/Rebol3/commit/c8397b8b8755e52281142c2daf0a3820c4416f5f) Missing argument for throw-http-error in the HTTP scheme
 * [`67f5da`](https://github.com/Oldes/Rebol3/commit/67f5da5d8d730b442a058daccdb6d242d55cc3aa) TLS protocol: handle site using TLS1.2 with rss_pss signature algorithm

NOTICE

@@ -1,6 +1,6 @@
 REBOL [R3] Language Interpreter and Run-time Environment
 Copyright 2012 REBOL Technologies
-Copyright 2012-2021 Rebol Open Source Developers
+Copyright 2012-2025 Rebol Open Source Developers
 REBOL is a trademark of REBOL Technologies
 Licensed under the Apache License, Version 2.0
 See included LICENSE file for details
@@ -16,10 +16,19 @@ Credits for Non-REBOL orginated C files and modules
 * Unicode encoding/decoding functions:
   Copyright 2001-2004 Unicode, Inc.
 
-* ZLIB general purpose compression library:
+* libdeflate:
+  Copyright 2016 Eric Biggers
+  Copyright 2024 Google LLC
+  MIT - https://github.com/ebiggers/libdeflate
+
+* ZLIB general purpose compression library (optional):
   Version 1.3.1.1, 1-Apr-2025
   Copyright 1995-2024 Jean-loup Gailly and Mark Adler
 
+* LZ4 (optional):
+  Copyright (c) Yann Collet
+  BSD - https://github.com/lz4/lz4
+
 * LZAV (optional):
   Copyright (C) 2023-2025, Aleksey Vaneev
   MIT - https://github.com/avaneev/lzav

make/rebol3.nest

@@ -18,7 +18,10 @@ temp:     %make/tmp/
 stack-size: 4194304 ;= 4MB (4 * 1024 * 1024)
 optimize:   2
 
-version: 3.20.5
+version: 3.20.7
+
+;flags: "-ggdb3"
+;define: DEBUG
 
 #if Linux?   [ defines: TO_LINUX   ]
 #if macOS?   [ defines: TO_MACOS   ]
@@ -270,7 +273,6 @@ core-files: [
 ;	%core/u-rc4.c             ;optional, use: include-cryptography
 ;	%core/u-rsa.c             ;optional, use: include-cryptography
 ;	%core/u-uECC.c            ;optional, use: include-cryptography
-	%core/u-zlib.c
 ;	%core/u-wav.c             ;optional, use: include-native-wav-codec
 
 	%core/u-mbedtls.c
@@ -451,7 +453,7 @@ include-image-natives: [
 ]
 
 include-native-bmp-codec: [config: INCLUDE_BMP_CODEC core-files: %core/u-bmp.c]
-include-native-png-codec: [config: INCLUDE_PNG_CODEC core-files: %core/u-png.c]
+include-native-png-codec: [config: INCLUDE_PNG_CODEC core-files: %core/u-png.c core-files: %core/u-zlib.c]
 include-native-qoi-codec: [config: INCLUDE_QOI_CODEC core-files: %core/u-qoi.c]
 include-native-jpg-codec: [config: INCLUDE_JPG_CODEC core-files: %core/u-jpg.c]
 include-native-gif-codec: [config: INCLUDE_GIF_CODEC core-files: %core/u-gif.c]
@@ -786,6 +788,15 @@ include-lzma-compression: [
 	core-files: %core/u-lzma.c
 ]
 
+include-lz4-compression: [
+	config: INCLUDE_LZ4
+	include: %src/core/lz4/
+	files: [
+		%core/lz4/lz4.c
+		%core/lz4/lz4hc.c
+	]
+]
+
 include-lzav-compression: [
 	config: INCLUDE_LZAV
 ]
@@ -841,6 +852,28 @@ include-brotli-compression: [
 	]
 ]
 
+include-zlib-compression: [
+	;= Original but slower Adler's implementation.
+	config: INCLUDE_DEPRECATED_ZLIB
+	core-files: %core/u-zlib.c
+]
+include-deflate-compression: [
+	config: INCLUDE_DEFLATE
+	core-files: [
+		%core/deflate/lib/adler32.c
+		%core/deflate/lib/crc32.c
+		%core/deflate/lib/deflate_compress.c
+		%core/deflate/lib/deflate_decompress.c
+		%core/deflate/lib/gzip_compress.c
+		%core/deflate/lib/gzip_decompress.c
+		%core/deflate/lib/utils.c
+		%core/deflate/lib/zlib_compress.c
+		%core/deflate/lib/zlib_decompress.c
+		%core/deflate/lib/x86/cpu_features.c
+		%core/deflate/lib/arm/cpu_features_arm.c
+	]
+]
+
 include-png-filter-native: [
 	config: INCLUDE_PNG_FILTER
 	core-files: %core/u-png-filter.c
@@ -944,6 +977,11 @@ include-prebol:              [mezz-lib-files: %modules/prebol.reb           ]
 include-native-oid: [core-files: %core/n-oid.c] ; `form-oid` native (used in DER codec)
 
 
+;- Deflate is part of the Base!
+
+;:include-zlib-compression   ;= Original Adler's implementation
+:include-deflate-compression ;= libdeflate's zlib, gzip and deflate
+
 ;- Product specifications                                                       
 
 include-rebol-core: [
@@ -977,6 +1015,7 @@ include-rebol-bulk: [
 	:include-rebol-core
 	:include-optional-checksums
 	:include-image-natives
+	:include-lz4-compression
 	:include-lzav-compression
 	:include-lzma-compression   ;= cca 17kB
 	:include-lzw-compression    ;= cca 2kB

src/boot/natives.reb

@@ -250,6 +250,7 @@ recycle: native [
 	/ballast {Trigger for auto-recycle (memory used)}
 	size [integer!]
 	/torture {Constant recycle (for internal debugging)}
+	/pools {Release empty memory pool segments}
 ]
 
 release: native [

src/boot/strings.reb

@@ -133,9 +133,14 @@ errs:
 	"** Near: "
 
 watch:
+;; Used with: evoke 'watch-recycle
 	"RECYCLING: "
 	"%d series"
 	"obj-copy: %d %m"
+	"EMPTY(pool=%d, seg=%x, size=%d)"
+;; Used with: evoke 'watch-alloc (only in debug builds)
+	"ALLOC: %d"
+	"FREE:  %d"
 
 extension:
 	"RX_Init"

src/boot/sysobj.reb

@@ -120,10 +120,12 @@ catalog: object [
 		quoted-printable: #(bitset! #{FFFFFFFFFFFFFFFBFFFFFFFFFFFFFFFF})
 	]
 	structs: make map! [] ;; filled using `register` native function
+	compressions: [
+		; will be filled on boot from `Init_Compression` in `u-compress.c`
+	]
 	checksums: [
 		; will be filled on boot from `Init_Crypt` in `n-crypt.c
 	]
-	compressions: [deflate zlib gzip] ;; these are always available
 	elliptic-curves: [
 		; will be filled on boot from `Init_Crypt` in `n-crypt.c`
 	]
@@ -266,7 +268,7 @@ modules: object [
 	blurhash:      https://github.com/Siskin-framework/Rebol-BlurHash/releases/download/1.0.0/
 	easing:        https://github.com/Siskin-framework/Rebol-Easing/releases/download/1.0.0/
 	mathpresso:    https://github.com/Siskin-framework/Rebol-MathPresso/releases/download/0.1.0/
-	miniaudio:     https://github.com/Oldes/Rebol-MiniAudio/releases/download/0.11.18.0/
+	miniaudio:     https://github.com/Oldes/Rebol-MiniAudio/releases/download/0.11.23.0/
 	speak:         https://github.com/Oldes/Rebol-Speak/releases/download/0.0.1/
 	sqlite:        https://github.com/Siskin-framework/Rebol-SQLite/releases/download/3.46.0.0/
 	triangulate:   https://github.com/Siskin-framework/Rebol-Triangulate/releases/download/1.6.0.0/

src/boot/words.reb

@@ -288,7 +288,9 @@ bits
 crash
 crash-dump
 watch-recycle
+watch-expand
 watch-obj-copy
+watch-alloc
 stack-size
 
 ;call/info

src/core/a-lib.c

@@ -172,11 +172,16 @@ extern int Do_Callback(REBSER *obj, u32 name, RXIARG *args, RXIARG *result);
 	REBSER *ser;
 
 	if (bin) {
+#ifdef INCLUDE_DEPRECATED_ZLIB
 		spec.data = bin;
 		spec.tail = len;
-		ser = DecompressZlib(&spec, 0, -1, 0, 0);
+		ser = DecompressZlibDeprecated(&spec, 0, -1, 0, 0);
+#else
+		REBINT err = 0;
+		DecompressZlib(bin, len, -1, &ser, &err);
+		if (err) return 1;
+#endif
 		if (!ser) return 1;
-
 		val = BLK_SKIP(Sys_Context, SYS_CTX_BOOT_HOST);
 		Set_Binary(val, ser);
 	}
@@ -322,12 +327,19 @@ extern int Do_Callback(REBSER *obj, u32 name, RXIARG *args, RXIARG *result);
 #endif
 
 	//Cloak(TRUE, code, NAT_SPEC_SIZE, &key[0], 20, TRUE);
+#ifdef INCLUDE_DEPRECATED_ZLIB
 	spec.data = bin;
 	spec.tail = length;
-	text = DecompressZlib(&spec, 0, -1, 0, 0);
+	text = DecompressZlibDeprecated(&spec, 0, -1, 0, 0);
+#else
+	REBINT err = 0;
+	DecompressZlib(bin, length, -1, &text, &err);
+	if (err) return FALSE;
+#endif
 	if (!text) return FALSE;
 	Append_Byte(text, 0);
 
+
 #ifdef DUMP_INIT_SCRIPT
 	f = _open("host-boot.reb", _O_CREAT | _O_RDWR, _S_IREAD | _S_IWRITE );
 	_write(f, STR_HEAD(text), LEN_BYTES(STR_HEAD(text)));

src/core/b-init.c

@@ -123,7 +123,7 @@ extern const REBYTE Str_Banner[];
 	if (rargs->options & RO_VERS) {
 		Out_Str(cb_cast(REBOL_VERSION), 0, FALSE);
 		Out_Str(cb_cast("\n"), 0, FALSE);
-		OS_Exit(0);
+		OS_Exit(0, 0);
 	}
 }
 
@@ -169,18 +169,21 @@ extern const REBYTE Str_Banner[];
 	// first four bytes of Native_Specs is a little-endian 32-bit
 	// length of the uncompressed spec data.
 	{
-		REBSER spec;
-		REBSER *text;
-		REBCNT textlen;
+		REBSER* text;
+		REBCNT textlen = Bytes_To_REBCNT(Native_Specs);
 
+#ifdef INCLUDE_DEPRECATED_ZLIB
+		REBSER spec;
 		// REVIEW: This is a nasty casting away of a const.  But there's
 		// nothing that can be done about it as long as Decompress takes
 		// a REBSER, as the data field is not const
 		spec.data = ((REBYTE*)Native_Specs) + 4;
 		spec.tail = NAT_SPEC_SIZE;
-
-		textlen = Bytes_To_REBCNT(Native_Specs);
-		text = DecompressZlib(&spec, 0, -1, textlen, 0);
+		text = DecompressZlibDeprecated(&spec, 0, -1, textlen, 0);
+#else
+		REBINT err = 0;
+		DecompressZlib(((REBYTE*)Native_Specs) + 4, NAT_SPEC_SIZE, textlen, &text, &err);
+#endif
 		if (!text || (STR_LEN(text) != textlen)) Crash(RP_BOOT_DATA);
 		boot = Scan_Source(STR_HEAD(text), textlen);
 		//Dump_Block_Raw(boot, 0, 2);
@@ -1009,6 +1012,10 @@ static void Set_Option_File(REBCNT field, REBYTE* src, REBOOL dir )
 	PG_Boot_Level = BOOT_LEVEL_FULL;
 	PG_Mem_Usage = 0;
 	PG_Mem_Limit = 0;
+#ifdef DEBUG
+	PG_Mem_Make = 0;
+	PG_Mem_Free = 0;
+#endif
 	PG_Reb_Stats = Make_CMem(sizeof(*PG_Reb_Stats));
 	Halt_State = 0;
 	Reb_Opts = Make_CMem(sizeof(*Reb_Opts));
@@ -1115,6 +1122,11 @@ static void Set_Option_File(REBCNT field, REBYTE* src, REBOOL dir )
 	if(!Task_Series)
 		return; // can happen when close button, shutdown, etc.
 
+#ifdef DEBUG
+	// Turn off watching memory (not possible after releasing output buffers)
+	Reb_Opts->watch_alloc = 0;
+#endif
+
 	Free_Series(Task_Series);
 	if (PG_Boot_Phase > BOOT_START) {
 		Free_Series(Bind_Table);
@@ -1147,9 +1159,20 @@ static void Set_Option_File(REBCNT field, REBYTE* src, REBOOL dir )
 		Dispose_Ports();
 		Dispose_Mold();
 		Dispose_CRC();
-		Free_Mem(PG_Boot_Strs, 0);
+#ifdef INCLUDE_CRYPTOGRAPHY
+		Dispose_Crypt();
+#endif
+		Dispose_Compression();
+		Dispose_Handles();
+		Free_Mem(PG_Boot_Strs, RS_MAX * sizeof(REBYTE*));
 	}
 	Dispose_StdIO();
-	Free_Mem(PG_Reb_Stats, 0);
-	Free_Mem(Reb_Opts, 0);
+	Dispose_Pools();
+	Free_Mem(PG_Reb_Stats, sizeof(*PG_Reb_Stats));
+	Free_Mem(Reb_Opts, sizeof(*Reb_Opts));
+#ifdef DEBUG
+	if (PG_Mem_Make != PG_Mem_Free || PG_Mem_Usage > 0)
+		printf("PG_Mem_Make: %llu free: %llu used: %llu\n", PG_Mem_Make, PG_Mem_Free, PG_Mem_Usage);
+#endif
+	ASSERT1(PG_Mem_Free == PG_Mem_Make, RP_MISC);
 }