Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: initial implementation of a libsql driver #79

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

hbjydev
Copy link

@hbjydev hbjydev commented Mar 13, 2025

This PR aims to implement #78 by introducing a new toasty-libsql driver, supporting libsql.

It uses a lot of carbon-copy code from the rusqlite implementation, as minimal changes are needed to port between SQLite APIs and LibSQL by design.

Outstanding things to check/implement;

  • I don't think libsql::Database is Sync, but libsql::Connection is. However, this means we have one connection for the entire driver instance. I don't want to go as far as opening one connection per execution, but I'd like to look into some method of pooling the connections at some point.
  • Token authentication to a LibSQL service needs a second parameter, but I don't know how to pass it just yet. I think some string manipulating/including the token in a query param then taking that parameter and passing it as the token might work. Implementing this via impl From<libsql::Connection> for LibSQL in the driver crate.
  • I'm doing a cast from usize to i32 in the code for the driver, but I have no idea if that's sensible in the slightest; if there's a more 'safe' way to implement that type conversion I'd be very appreciative, but for now it got the code running.

The new driver currently works properly with all the example codebases, running them with --features libsql and setting TOASTY_CONNECTION_URL to libsql::memory: succeeds without any errors.

Let me know if I'm barking up the right tree here or not! :)

@hbjydev hbjydev mentioned this pull request Mar 13, 2025
@hbjydev hbjydev marked this pull request as draft March 14, 2025 13:30
@hbjydev
Copy link
Author

hbjydev commented Mar 14, 2025

I didn't see the multi-thread error locally... will investigate

@carllerche
Copy link
Member

Thanks for the PR. What are the pros / cons of libsql vs sqlite?

@hbjydev
Copy link
Author

hbjydev commented Mar 14, 2025

Thanks for the PR. What are the pros / cons of libsql vs sqlite?

So libsql primarily shines in a couple bigger places;

  • Server component -- it has an HTTP implementation layer on top of the Sqlite codebase that lets you run it as a server instead of being stuck using it locally
  • Ability to do multi-instance replication
  • Full compatibility with SQLite because it's just a fork of it
  • It has the ability to continuously snapshot itself to S3 if configured to do so
  • Design-wise, it's focused on query latency and HA.
  • Support for at-rest encryption

If you want Turso's argument, their README lists a few more features it adds over plain SQLite, and their 'manifesto' provides the motivation behind building it.

@hbjydev
Copy link
Author

hbjydev commented Mar 14, 2025

Also, r.e. this:

Token authentication to a LibSQL service needs a second parameter, but I don't know how to pass it just yet.

I think I might implement From<libsql::Database> to let people bring their own DB configurations...

@hbjydev
Copy link
Author

hbjydev commented Mar 15, 2025

Okay @carllerche I think I've figured out what breaks this... Apparently running libsql and rusqlite in the same thread has a bit of an issue attached; tursodatabase/libsql#526

I'm not entirely sure how to resolve this in the context of cargo test, given it just runs them side-by-side.

Would you be okay with me splitting the test command to do --feature libsql, --feature sqlite, etc. individually? The tests do pass, this is just an issue with how rusqlite links its binary from what I can gather.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants