WIP: Implement COPY … FROM STDIN
#566
Conversation
This is still WIP. To do items include: - [ ] Test the various error cases, I have mostly focused on the success case so far - [ ] Test the backpressure support - [ ] Change the public API to accept the table + columns to copy into as well as options so that we can build the `COPY` query instead of letting the user write it - [ ] Add an API that allows binary transfer of data
| /// A handle to send | ||
| public struct PostgresCopyFromWriter: Sendable { | ||
| private let channelHandler: NIOLoopBound<PostgresChannelHandler> | ||
| private let context: NIOLoopBound<ChannelHandlerContext> |
There was a problem hiding this comment.
use the handlerContext: ChannelHandlerContext? with ! in PostgresChannelHandler instead.
There was a problem hiding this comment.
Would you prefer to make PostgresChannelHandler.handlerContext internal? Not sure whether it’s supposed to be private.
There was a problem hiding this comment.
You don't need to pass the context to sendCopyDone or copyData. Instead use the context that already exists in the PostgresChannelHandler.handlerContext.
There was a problem hiding this comment.
Oops, not sure how I missed that.
Sources/PostgresNIO/New/Connection State Machine/ExtendedQueryStateMachine.swift
Outdated
Show resolved
Hide resolved
| @@ -88,7 +88,23 @@ struct ConnectionStateMachine { | |||
| case sendParseDescribeBindExecuteSync(PostgresQuery) | |||
| case sendBindExecuteSync(PSQLExecuteStatement) | |||
| case failQuery(EventLoopPromise<PSQLRowStream>, with: PSQLError, cleanupContext: CleanUpContext?) | |||
| /// Fail a query's execution by throwing an error on the given continuation. | |||
| case failQueryContinuation(any AnyErrorContinuation, with: PSQLError, cleanupContext: CleanUpContext?) | |||
There was a problem hiding this comment.
existential, where Existential isn't necessary.
There was a problem hiding this comment.
The alternative is to have two cases, one for CheckedContinuation<PostgresCopyFromWriter, any Error> and one for CheckedContinuation<Void, any Error>. Since this is the failure case, it’s not performance sensitive and I’d argue that the improved readability is worth the extra allocation in the failure case. Happy to adjust if you disagree though.
There was a problem hiding this comment.
Can we make this an enum instead? Best of both worlds?
There was a problem hiding this comment.
you can even implement the fail on it directly.
| context.flush() | ||
| } | ||
| context.write(self.wrapOutboundOut(self.encoder.flushBuffer()), promise: nil) |
There was a problem hiding this comment.
This doesn't look right. It would mean we only flush once the channel has become not writable.
This is still WIP. To do items include:
COPYquery instead of letting the user write itPostgresCopyFromWriterto reduce the number ofCopyDatamessages we need to send (and thus the protocol overhead). Alternatively, we can leave that kind of optimization to the client.Deferred to follow-up PRs:
COPY FROM.Fixes #290