Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: hold runtime ref and handle to prevent spawn after shutdown #736

Merged
merged 3 commits into from
Sep 24, 2024

Conversation

MrCroxx
Copy link
Collaborator

@MrCroxx MrCroxx commented Sep 24, 2024

What's changed and what's your intention?

Please explain IN DETAIL what the changes are in this PR and why they are needed. :D

As title.

Checklist

  • I have written the necessary rustdoc comments
  • I have added the necessary unit tests and integration tests
  • I have passed make all (or make fast instead if the old tests are not modified) in my local environment.

Related issues or PRs (optional)

fix #735

@MrCroxx MrCroxx added bug Something isn't working refactor labels Sep 24, 2024
@MrCroxx MrCroxx added this to the v0.11 milestone Sep 24, 2024
@MrCroxx MrCroxx self-assigned this Sep 24, 2024
Signed-off-by: MrCroxx <[email protected]>
Copy link

codecov bot commented Sep 24, 2024

Codecov Report

Attention: Patch coverage is 93.97590% with 10 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
foyer-storage/src/store.rs 66.66% 6 Missing ⚠️
foyer-memory/src/generic.rs 87.50% 1 Missing ⚠️
foyer-storage/src/device/direct_fs.rs 85.71% 1 Missing ⚠️
foyer-storage/src/device/mod.rs 66.66% 1 Missing ⚠️
foyer-storage/src/large/generic.rs 96.00% 1 Missing ⚠️
Files with missing lines Coverage Δ
foyer-common/src/asyncify.rs 50.00% <100.00%> (ø)
foyer-common/src/runtime.rs 84.21% <100.00%> (+12.78%) ⬆️
foyer-memory/src/cache.rs 86.18% <100.00%> (ø)
foyer-storage/src/device/direct_file.rs 84.91% <100.00%> (-0.17%) ⬇️
foyer-storage/src/device/monitor.rs 78.64% <100.00%> (ø)
foyer-storage/src/large/flusher.rs 85.14% <100.00%> (ø)
foyer-storage/src/large/reclaimer.rs 85.48% <100.00%> (ø)
foyer-storage/src/large/recover.rs 90.29% <100.00%> (ø)
foyer-storage/src/large/scanner.rs 75.70% <100.00%> (-0.02%) ⬇️
foyer-storage/src/large/tombstone.rs 96.55% <100.00%> (+0.08%) ⬆️
... and 7 more

@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Sep 24, 2024

Wait for risingwavelabs/risingwave#18652 to pass.

@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Sep 24, 2024

image

This PR fixed the bug. But there is performance regression. Let me check why.

@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Sep 24, 2024

image

Make it back to normal. It is because the disk io was not using dedicated foyer runtime, which is a bug. After this PR fixed it, the --read/write--runtime-max-blocking-thread works and limited the threads count to a low number and limited the performance. After removing the option from foyer-bench, the performance looks fine.

@MrCroxx
Copy link
Collaborator Author

MrCroxx commented Sep 24, 2024

The new benchmark command:

cargo build --release --features jemalloc && PATH=/home/mrcroxx/.local/bin:$PATH RUST_BACKTRACE=1 RUST_LOG=info flamegraph -- ./target/release/foyer-bench --file /dev/nvme2n1 --mem 64MiB --disk 100GiB --region-size 64MiB --get-range 100000 --flushers 4 --reclaimers 4 --time 60 --writers 256 --w-rate 4MiB --admission-rate-limit 500MiB --readers 32 --r-rate 32MiB --metrics --warm-up 2 --runtime separated --user-runtime-worker-threads 8 --read-runtime-worker-threads 8 --write-runtime-worker-threads 4

@MrCroxx MrCroxx merged commit b49e275 into main Sep 24, 2024
22 checks passed
@MrCroxx MrCroxx deleted the xx/runtime branch September 24, 2024 16:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working refactor
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

bug: flusher spawn task after runtime is dropped
1 participant