Skip to content

Conversation

@MichaelScofield
Copy link
Collaborator

I hereby agree to the terms of the GreptimeDB CLA.

Refer to a related PR or issue link (optional)

What's changed and what's your intention?

following #7096
for postgres

PR Checklist

Please convert it to a draft if some of the following conditions are not met.

  • I have written the necessary rustdoc comments.
  • I have added the necessary unit tests and integration tests.
  • This PR requires documentation updates.
  • API changes are backward compatible.
  • Schema or data changes are backward compatible.

@github-actions github-actions bot added size/L docs-not-required This change does not impact docs. labels Oct 22, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR refactors the PostgreSQL type encoding logic to convert data directly from Arrow arrays instead of going through intermediate Value representations. This follows a similar optimization made in PR #7096.

Key Changes

  • Replaced value-based encoding with direct Arrow array conversion for better performance
  • Introduced RecordBatchRowIterator to iterate over record batches and encode rows directly from Arrow arrays
  • Eliminated intermediate Value conversions by reading directly from typed Arrow arrays

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 7 comments.

File Description
src/servers/src/postgres/types.rs Refactored encoding functions to work directly with Arrow arrays; added RecordBatchRowIterator for row-by-row encoding; updated tests to use Arrow schema and vectors
src/servers/src/postgres/handler.rs Simplified stream processing to use new RecordBatchRowIterator instead of manual row encoding

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

@MichaelScofield MichaelScofield force-pushed the refactor/pg-write-arrow-directly branch 2 times, most recently from 70323c6 to df2efc0 Compare October 23, 2025 06:54
@MichaelScofield MichaelScofield force-pushed the refactor/pg-write-arrow-directly branch from df2efc0 to 24fd6e5 Compare October 23, 2025 07:05
@MichaelScofield MichaelScofield marked this pull request as ready for review October 23, 2025 07:33
@MichaelScofield MichaelScofield requested a review from a team as a code owner October 23, 2025 07:33
@sunng87
Copy link
Member

sunng87 commented Oct 23, 2025

LGTM. And I think next step we can switch to my datafusion-postgres ecosystem library arrow-pg

Signed-off-by: luofucong <[email protected]>
@MichaelScofield
Copy link
Collaborator Author

@killme2008 PTAL

@killme2008 killme2008 enabled auto-merge October 24, 2025 03:08
@killme2008 killme2008 added this pull request to the merge queue Oct 24, 2025
Merged via the queue into main with commit 6ad23bc Oct 24, 2025
42 checks passed
@killme2008 killme2008 deleted the refactor/pg-write-arrow-directly branch October 24, 2025 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs-not-required This change does not impact docs. size/L

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants