Skip to content

Add async memory operations with AsyncFileMem#27

Open
konard wants to merge 4 commits intomainfrom
issue-26-90c80e263d5a
Open

Add async memory operations with AsyncFileMem#27
konard wants to merge 4 commits intomainfrom
issue-26-90c80e263d5a

Conversation

@konard
Copy link
Member

@konard konard commented Jan 13, 2026

Summary

Implements GitHub issue #26: async memory access support with benchmarks.

  • Add AsyncFileMem<T> type for async file-backed memory operations using tokio
  • Add async feature flag to optionally enable async functionality
  • Create comprehensive benchmarks comparing sync vs async memory operations
  • Update documentation with async installation, usage examples, and API reference

Implementation Details

AsyncFileMem<T>

A new async file-backed memory type that provides:

  • create(path) / open(path) / temp() - async constructors
  • grow() / grow_filled() / grow_zeroed() / grow_from_slice() - async grow operations
  • shrink() - async shrink operation
  • get() / set() / as_slice() / as_slice_mut() - synchronous in-memory buffer access
  • sync() / flush() - async persistence to disk

Why Not Async Mmap?

Memory-mapped file I/O cannot be truly asynchronous because page faults trigger synchronous disk operations. For truly async file access, AsyncFileMem uses explicit read/write operations with an in-memory buffer.

Feature Flags

# Enable async memory operations (uses tokio)
platform-mem = { version = "0.1", features = ["async"] }

Benchmarks

Added benchmarks in benches/memory_benchmarks.rs comparing:

  • Sync allocator-based memory vs file-mapped memory
  • Async file memory with sync operations
  • Random read and sequential write patterns
  • Grow/shrink cycles

Test plan

  • All existing tests pass (72 coverage tests + 3 other tests)
  • All new async tests pass (9 async-specific tests)
  • Benchmarks compile and run
  • Tests pass with and without async feature
  • CI passes on all platforms (Ubuntu, macOS, Windows)

Files Changed

  • Cargo.toml - Added async feature flag and tokio dependency
  • src/lib.rs - Added conditional async module export
  • src/async_mem.rs - New async memory implementation (493 lines)
  • benches/memory_benchmarks.rs - Comprehensive benchmarks (280 lines)
  • experiments/async_example.rs - Usage example
  • README.md - Updated documentation

Fixes #26

🤖 Generated with Claude Code

Adding CLAUDE.md with task information for AI processing.
This file will be removed when the task is complete.

Issue: #26
@konard konard self-assigned this Jan 13, 2026
Implements GitHub issue #26: async memory access support.

- Add AsyncFileMem<T> type for async file-backed memory using tokio
- Add 'async' feature flag to enable async functionality
- Add 'io-uring' feature flag placeholder for future io_uring support
- Create comprehensive benchmarks comparing sync vs async operations
- Add async usage example in experiments/async_example.rs
- Update documentation with async installation and usage examples
- Add AsyncFileMem to the Memory Types table

AsyncFileMem provides:
- create/open/temp async constructors
- async grow/grow_filled/grow_zeroed/grow_from_slice/shrink operations
- get/set/as_slice/as_slice_mut for in-memory buffer access
- sync/flush for persisting changes to disk

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@konard konard changed the title [WIP] In addition to sync mmap access also support async memory access as efficiently as possible, add bechmarks Add async memory operations with AsyncFileMem Jan 13, 2026
The io_uring feature was causing CI failures on macOS and Windows
because tokio-uring only compiles on Linux. Since the io_uring
implementation is not yet complete, remove the feature for now.
The async feature with tokio works cross-platform.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@konard konard marked this pull request as ready for review January 13, 2026 12:34
@konard
Copy link
Member Author

konard commented Jan 13, 2026

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

  • Public pricing estimate: $8.004834 USD
  • Calculated by Anthropic: $5.706724 USD
  • Difference: $-2.298110 (-28.71%)
    📎 Log file uploaded as Gist (1071KB)
    🔗 View complete solution draft log

Now working session is ended, feel free to review and add any feedback on the solution draft.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

In addition to sync mmap access also support async memory access as efficiently as possible, add bechmarks

1 participant