Anything else to add?

[brianc]

@charmander thanks for putting the 8.0 milestone on a few of the issues. Anything else that comes to mind you want to break in the newest release? I want to keep churn somewhat down on the 8.0 release...one of the main things I want to do is drop support for all versions of node earlier than any active LTS versions. There was some talk about making BigInt support opt-in in 8.0...but since it'll be opt-in for now I don't think we need to land it directly on the major version bump.

It looks like 8.0 is going end of life in like...3 days! So I think we could drop official support for anything earlier than node 10 which is the currently oldest LTS version...what do you think? reference

Now that this is a monorepo with most of the main dependent modules included I can have a much easier time doing sweeping changes to the internal APIs and weird timing behaviors. I want to hit the entire monorepo w/ a prettier format & enforce prettier formatting via eslint because the lint rules haven't been consistent between all the repos...and the tests inside pg (some of which are very old) haven't been linted at all...so...I'm going to fix all the lint stuff before 8.0 just to keep things slightly more organized.

Thanks again for all your help & support.

[uasan]

Remove callback interfaces, save only promise interfaces.
This clearing your code base and use async native JS functions.

[vitaly-t]

I just wanted to add my main concern here...

There are literally thousands of examples and manuals out there, explaining how to use such external modules as pg-query-stream. And merging the module in has just invalidated all that. I predict the confusion over using it as external module will echo here accordingly, 1000-fold. And nobody is going to update the plethora of external examples and documentation for it.

Explained by the author below.

[vitaly-t]

@brianc And as to what to else to add, I will again sneak in with my connection-string module 😄

[brianc]

merging the module in has just invalidated all that

As I mentioned here and just to clear up any confusion...definitely not merging the modules together nor do I have any desire to do that. If anything I'd like even more modules for certain things that may want to be reused in other places....I am only merging the repos because me dancing between 4-5 github repositories w/ their own issue trackers and trying to managing breaking and semver changes across interconnected modules is harder and I miss a lot more things than I'd like because of that I think....so from a consumer's perspective absolutely nothing should change...from a maintainer's perspective...the plan is it'll be easier for me 🤞 and clearer for users on where to go for help/issues. Good thoughts on keeping backwards compatibility with regard to existing documentation too - yet another reason to try to avoid breaking changes unless the good strongly outweighs the bad!

[brianc]

And as to what to else to add, I will again sneak in with my connection-string module 😄

ah nice I think I had a couple issues around unix socket stuff...I'll take a look at this in the future! 👍

[vitaly-t]

@brianc Thanks for clarifications! 👍

[brianc]

Remove callback interfaces, save only promise interfaces.
This clearing your code base and use async native JS functions.

Yup, I'm going to do something like that in version 9.0...probably going to significantly change the internals of the client, focusing on some pretty big performance gains I've been experimenting with. I'll likely build a compatibility layer you can opt into which will still have callbacks.

[uasan]

Batch mode and query pipelining (like PgJDBC, but unsupported libpq)
https://www.2ndquadrant.com/en/blog/postgresql-latency-pipelining-batching/

Patch for libpq
https://commitfest.postgresql.org/14/1024/

This is fastest when the client sends a lot of small requests and when the database server removed.

[brianc]

Yup batching (pipelining I've called it) is on the horizon as well, absolutely.

[uasan]

Yup, I'm going to do something like that in version 9.0...probably going to significantly change the internals of the client, focusing on some pretty big performance gains I've been experimenting with. I'll likely build a compatibility layer you can opt into which will still have callbacks.

Why wait for version 9?
I think those who upgrade to version 8, they no longer need callbacks.

But respect anyway 👍

[brianc]

Why wait for version 9?

Well...two reasons. 1) it's a lot of work & I don't want to delay 8 for it...8's already been held back for quite a while as we were letting some breaking changes kinda "pile up" for it. and 2) It's going to be a big change & I'd like to release it and nothing else so the upgrade guide is clear and its not "oh we deprecated this, that, this one thing changed, also: rewrote EVERYTHING" just to keep issue triage after the fact (hopefully there aren't any...but you never know) a bit easier to do. Plus I dunno...this module is really heavily used in a lot of places & I don't subscribe to the "move fast and break things" mindset w/ it at all...rather go a bit on the slow side than fast side....but we'll get there! It's always been more of a marathon than a sprint kinda mindset for me...helps me not burn out. 😄

[johanneswuerbach]

I would really like to see something like brianc/node-pg-pool#119 included as we historically had a lot of bugs with accidentally re-adding broken clients to the pool just to get random errors later. The client is already aware of the client state so that could be prevented by default with probably some warning logged.

[brianc]

I would really like to see something like brianc/node-pg-pool#119 included as we historically had a lot of bugs with accidentally re-adding broken clients to the pool just to get random errors later. The client is already aware of the client state so that could be prevented by default with probably some warning logged.

excellent point - will port the issue & code over here after merging #2048 and get that in. It's kinda a bummer that's even a breaking change but hey - we'll get it in!

[charmander]

It looks like 8.0 is going end of life in like...3 days! So I think we could drop official support for anything earlier than node 10 which is the currently oldest LTS version...what do you think?

That sounds good!

Another pool breaking change that I think would be nice to have in some major version (maybe not 8) would be to return a new instance wrapping the client for each pool, to stop clients from being reused incorrectly after being released/having event listeners incorrectly left attached.

[brianc]

Kk sweet. I was at a good cadence but i was chopping veggies for dinner and cut the end of my left middle finger off! Not down to the bone thankfully, but it’s currently wrapped in a big mound of gause so i will probably be down and out for at least another few days. Then steady march to 8.0 and beyond!

…

On Wed, Jan 1, 2020 at 9:37 PM Charmander ***@***.***> wrote: It looks like 8.0 is going end of life in like...3 days! So I think we could drop official support for anything earlier than node 10 which is the currently oldest LTS version...what do you think? That sounds good! Another pool breaking change that I think would be nice to have in some major version (maybe not 8) would be to return a new instance wrapping the client for each pool, to stop clients from being reused incorrectly after being released/having event listeners incorrectly left attached. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2047?email_source=notifications&email_token=AAAMHILJ2TQ5S2JPBDIVQNLQ3VOOJA5CNFSM4J77VVR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH5TMLA#issuecomment-570111532>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAMHIOJPDHGQSDVXCOKMY3Q3VOOJANCNFSM4J77VVRQ> .

[charmander]

Yikes :( I hope you have a good, not-too-painful recovery.

[abenhamdine]

Hi Brianc,
Don't know if it's the right place here, but it would be great to consider to solve #1568 for version 8.

[sehrope]

+1 to wrapping pooled clients.

-1 to kitchen accidents. Hope you get better.

[abenhamdine]

Not down to the bone thankfully, but it’s currently wrapped in a big mound of gause so i will probably be down and out for at least another few days

Oh sorry, I didn't really read your comment !
Hope you will be ok !

[jgeurts]

Based on some deprecation messages, I’m guessing this is included. If not, please put me down for a +1 for #1996

[jgeurts]

Also, not sure if it’s doable for v8, but I’d like to see pg-pool decoupled from pg. We use a different pooling library and pg-pool is an unnecessary companion.

[brianc]

Based on some deprecation messages, I’m guessing this is included. If not, please put me down for a +1 for #1996

yah that's definitely part of it.

Also, not sure if it’s doable for v8, but I’d like to see pg-pool decoupled from pg. We use a different pooling library and pg-pool is an unnecessary companion.

I don't think this will ever happen - if you don't want to use the pool...just instantiate clients directly! But its way way way too baked into the library to remove at this point. If I had a time machine I might have done things differently, but node was a different thing back then...the whole "micro-modules" thing hadn't even become a craze yet.

edit: actually...if I had a time machine I'd probably just go back in time, buy some stocks, come back to now and retire. 😅

[jgeurts]

:) figured it was worth a shot!

[zispo]

@brianc please don't remove the callback interfaces as others have suggested. We use your module in a large project where performance is important. The promise interfaces are slower and cause more allocations. If you remove the callback interfaces for 9.0 we'll be stuck in 8.0 and we'll have to start a private fork :(
PS: If you are determined to remove the callback interfaces, please let us know, knowing this will be extremely helpful for us to reconsider our options, thanks!

[brianc]

I definitely don't plan on removing callback interface in 9.0. My primary focus in fact will be on performance...making things faster, sometimes by quite a large margin if my initial experiments are correct. I just need to climb out from under this large mound of covid-19 anxiety and get back to work. But don't worry, your callbacks are safe for the foreseeable future.

[zispo]

Hey @brianc thanks for the word on our callbacks :) The reason I was a bit concerned about is that it seemed like according to this comment of yours:

I'll likely build a compatibility layer you can opt into which will still have callbacks.

That the idea is to have the library internally using promises and just having a compatibility layer on top of that that interfaces with said promises and converts that to a callback interface. This would defeat the performance advantage of not having promises, so it's not what we are looking for. Ideally what we would want is the minimum amount of allocations between the point that the query is requested and when the resulting callback is called.
Take it easy!

[dpkirchner]

If brianc/node-pg-pool#119 (referenced in another comment) can't be merged, moving the client.release = ...[0] call up so it's just before the emit('connect', client) call would allow us to implement our own release logic (saving the error emission and using it in our overridden release call, for example) before the client is returned to our code. This might be a breaking change, but I kind of doubt there's a lot of people that rely on release being set only after connect is emitted.

[0] https://github.com/brianc/node-postgres/blob/master/packages/pg-pool/index.js#L255