Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better documentation for connection pooling #5

Closed
paul-gaspar opened this issue Jan 3, 2020 · 1 comment
Closed

Better documentation for connection pooling #5

paul-gaspar opened this issue Jan 3, 2020 · 1 comment
Assignees
Labels

Comments

@paul-gaspar
Copy link

@paul-gaspar paul-gaspar commented Jan 3, 2020

First of all - thank you very much for this great piece of work! This is by far the best approach to a postgres client.

Can we have a bit more information about connection pooling please?

As far as I can see there is no need for releasing a connection, even in case of a failed query? Does postgres handle all those cases?

What happens if the pool is full? Are queries queued and executed in a fifo manner?

Is there a timeout for queries?

What happens if a db becomes unavailable (temporarily or permanently), or if the master node changes in a high availability setup? Does it try to reconnect and to continue executing the queue?

@porsager
Copy link
Owner

@porsager porsager commented Jan 3, 2020

Thanks a lot !

And thank you for the issue, that'll be good to have in the documentation.

As a starter for the docs I'll answer your questions and describe it here 😉

Yes, connections are handled on failed queries so it shouldn't be a user concern except maybe deciding on the seconds of timeout for idle connections.

The connection pool is implicit and lazy. The only thing defining it is the max number of connections specified in options.

Connections are created lazily once a query is created. This means that simply doing const sql = postgres(...) won't have any effect other than instantiating a new sql instance. No connection will be made until a query is made. This means that we get a much simpler story for error handling and reconnections. Queries will be sent over the wire immediately on the next available connection in the pool. Connections are automatically taken out of the pool if you sql.begin() a transaction, and returned to the pool once queries / the async callback function resolves.

That means that any query which was already sent over the wire will be rejected if the connection is lost. It'll automatically defer to the error handling you have in your app, and since connections are lazy it'll automatically try to reconnect the next time a query is made. The benefit is no weird generic "onerror" handler that tries to get things back to normal, and also simpler application code since you don't have to handle code paths out of context.

There are no guarantees about queries executing in order unless using a transaction with sql.begin() or setting max: 1. Of course doing a series of queries, one awaiting the other will work as expected, but that's just due to the nature of js async/promise handling, so it's not necessary for this library to be concerned with ordering (outside transactions that is).

I have timeout for queries on my todo as well as cancelable queries. The usage will most likely look like this:

timeout

await sql`select pg_sleep(10)`.timeout(5)
// throws after 5 seconds

cancellable

const { promise, cancel } = sql`select pg_sleep(10)`.cancellable()
cancel()
// throws and sends a cancel request to postgres

I'm also working on multihost support which will work like specified for libpq here: https://www.postgresql.org/docs/10/libpq-connect.html#LIBPQ-MULTIPLE-HOSTS
That should tie in well with the way pools work now, so it will try to reconnect and continue executing queries.

Also related — sql.listen(...) uses its own dedicated connection which is also lazily created the first time sql.listen(...) is called. It does this because we always want to be able to receive NOTIFY messages regardless of any queue of queries / transactions blocking a connection.

I hope that covered most of your questions :)

@porsager porsager self-assigned this Jan 3, 2020
@porsager porsager closed this in 88e4a97 Jan 20, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.