[Feature] Add Redis-backed TensorDict (RedisTensorDict)

Adds RedisTensorDict, a TensorDictBase subclass that stores tensors in a
Redis instance for out-of-core storage. This enables datasets that exceed
local RAM to be accessed with the familiar TensorDict interface.

MVP features:
- Async redis.asyncio client on a background event loop with sync wrappers
- Zero-copy serialization via torch.frombuffer / untyped_storage bytes
- Metadata (shape, dtype) stored in Redis Hashes for fast introspection
- Redis Cluster-compatible key schema using hash tags
- Pipelined batch I/O for all get/set operations
- Nested TensorDict support via lightweight prefix-based views
- Pickle support for multi-process usage
- from_dict() factory, to_local() / to_tensordict() materialization
- CI: install redis package and start redis-server before tests


ghstack-source-id: 6cb2eff
Pull-Request: pytorch#1567

Author: vmoens

.github/unittest/linux/scripts/environment.yml

@@ -22,3 +22,4 @@ dependencies:
     - ninja
     - numpy<2.0.0
     - mosaicml-streaming
+    - redis

.github/unittest/linux/scripts/run_test.sh

@@ -26,6 +26,23 @@ export TORCHDYNAMO_INLINE_INBUILT_NN_MODULES=1
 export TD_GET_DEFAULTS_TO_NONE=1
 export LIST_TO_STACK=1
 
+# Start Redis server for test_redis.py (non-fatal if unavailable)
+if command -v redis-server &> /dev/null; then
+    redis-server --daemonize yes --save "" --appendonly no || true
+else
+    case "$(uname -s)" in
+        Linux*)
+            apt update -y && apt install -y redis-server && redis-server --daemonize yes --save "" --appendonly no || echo "Redis server not available, redis tests will be skipped"
+            ;;
+        Darwin*)
+            brew install redis 2>/dev/null && redis-server --daemonize yes --save "" --appendonly no || echo "Redis server not available, redis tests will be skipped"
+            ;;
+        *)
+            echo "Redis server not available on this platform, redis tests will be skipped"
+            ;;
+    esac
+fi
+
 coverage run -m pytest test/smoke_test.py -v --durations 20
 coverage run -m pytest --runslow --instafail -v --durations 20 --timeout 120
 coverage run -m pytest ./benchmarks --instafail -v --durations 20

.github/unittest/linux/scripts/setup_env.sh

@@ -94,7 +94,7 @@ if [ "${PYTHON_VERSION}" == "3.14t" ]; then
     # Install test dependencies (mirrors environment.yml)
     pip install pybind11 numpy expecttest pyyaml hypothesis future cloudpickle \
         pytest pytest-benchmark pytest-cov pytest-mock pytest-instafail \
-        pytest-rerunfailures pytest-timeout coverage h5py orjson ninja protobuf
+        pytest-rerunfailures pytest-timeout coverage h5py orjson ninja protobuf redis
     # Note: mosaicml-streaming may not be available for 3.14t yet, skip if fails
     pip install mosaicml-streaming || echo "mosaicml-streaming not available for Python 3.14t, skipping"
 else

tensordict/__init__.py

@@ -47,6 +47,7 @@
 )
 from tensordict.memmap import MemoryMappedTensor
 from tensordict.persistent import PersistentTensorDict
+from tensordict.redis import RedisTensorDict
 from tensordict.tensorclass import (
     from_dataclass,
     MetaData,
@@ -105,6 +106,7 @@
     "TensorClass",
     "MemoryMappedTensor",
     "PersistentTensorDict",
+    "RedisTensorDict",
     "NestedKey",
     # Factory functions
     "from_dict",