Skip to content

feat(executor): add query execution stats #18345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jul 24, 2025

Conversation

dqhl76
Copy link
Collaborator

@dqhl76 dqhl76 commented Jul 11, 2025

I hereby agree to the terms of the CLA available at: https://docs.databend.com/dev/policies/cla/

Summary

feat(executor): add query execution stats

root@localhost:8000/default/default> from system.query_execution;

from
  system.query_execution

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│          node          │          timestamp         │               query_id               │ process_rows │ process_time_in_micros │
│         String         │          Timestamp         │                String                │    UInt64    │         UInt64         │
├────────────────────────┼────────────────────────────┼──────────────────────────────────────┼──────────────┼────────────────────────┤
│ vU7wCmWh9Zo1Vq2iGZbyN1 │ 2025-07-15 07:42:32.000000 │ bc8ddfc1-75c1-458d-8104-d4cd97b6bc69 │            2 │                    766 │
│ vU7wCmWh9Zo1Vq2iGZbyN1 │ 2025-07-15 07:42:38.000000 │ 77e2ad37-ae73-4a1d-a2b0-4cfdce4cceec │      9791888 │                 383508 │
│ vU7wCmWh9Zo1Vq2iGZbyN1 │ 2025-07-15 07:42:39.000000 │ 77e2ad37-ae73-4a1d-a2b0-4cfdce4cceec │    291175190 │                9891921 │
│ vU7wCmWh9Zo1Vq2iGZbyN1 │ 2025-07-15 07:42:40.000000 │ 79909746-d352-477a-8891-cc12a88d520e │            8 │                    448 │
│ vU7wCmWh9Zo1Vq2iGZbyN1 │ 2025-07-15 07:42:40.000000 │ 77e2ad37-ae73-4a1d-a2b0-4cfdce4cceec │    290045495 │                9887434 │
│ vU7wCmWh9Zo1Vq2iGZbyN1 │ 2025-07-15 07:42:41.000000 │ 77e2ad37-ae73-4a1d-a2b0-4cfdce4cceec │    291243909 │                9888362 │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯

Tests

  • Unit Test
  • Logic Test
  • Benchmark Test
  • No Test - Explain why

Type of change

  • Bug Fix (non-breaking change which fixes an issue)
  • New Feature (non-breaking change which adds functionality)
  • Breaking Change (fix or feature that could cause existing functionality not to work as expected)
  • Documentation Update
  • Refactoring
  • Performance Improvement
  • Other (please describe):

This change is Reviewable

@github-actions github-actions bot added the pr-feature this PR introduces a new feature to the codebase label Jul 11, 2025
@dqhl76 dqhl76 force-pushed the refactor-executor-2 branch 7 times, most recently from d8a9c27 to 15ac573 Compare July 14, 2025 06:15
@dqhl76 dqhl76 added the ci-benchmark Benchmark: run all test label Jul 14, 2025
Copy link
Contributor

Docker Image for PR

  • tag: pr-18345-e6916cc-1752477930

note: this image tag is only available for internal use.

@dqhl76 dqhl76 force-pushed the refactor-executor-2 branch from 15ac573 to 5696744 Compare July 15, 2025 06:59
This reverts commit 5696744.
@dqhl76 dqhl76 removed the ci-benchmark Benchmark: run all test label Jul 15, 2025
@dqhl76 dqhl76 marked this pull request as ready for review July 15, 2025 07:51
@dqhl76 dqhl76 requested a review from zhang2014 July 15, 2025 07:51
@dqhl76 dqhl76 force-pushed the refactor-executor-2 branch from 0176290 to 2e6f3a1 Compare July 15, 2025 09:36
@wubx
Copy link
Member

wubx commented Jul 16, 2025

Why not add it to system.processes?

@dqhl76
Copy link
Collaborator Author

dqhl76 commented Jul 16, 2025

Why not add it to system.processes?

The key difference is that system.processes shows a real-time snapshot of what's currently running, whereas system.query_execution displays performance metrics from the last 10 seconds. A finish query will also have a record.
Another reason is that in a distributed environment, system.processes only shows one record for a given query. The new table is designed to show a record for each node involved, giving a much more detailed, per-node view.
To not break the current system.processes, I choose to create a new table.

@zhang2014 zhang2014 merged commit 00a0693 into databendlabs:main Jul 24, 2025
164 of 166 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
pr-feature this PR introduces a new feature to the codebase
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants