Skip to content

Statically linked with libsqlite3.a with LTO enabled #16

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 28 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
8dcece0
Add .cargo/config to use linker-plugin-lto
NobodyXu Jul 26, 2021
be7cd34
Add submodule sqlite3
NobodyXu Jul 26, 2021
64b2cf7
Add once_cell to build-dependencies
NobodyXu Jul 26, 2021
8654d95
Add `fn main` to src/bin/common.rs to suppress error
NobodyXu Jul 26, 2021
57b4840
Add '*.pc' to .gitignore
NobodyXu Jul 26, 2021
c86098b
Replace submodule sqlite3 with amalgamation source code
NobodyXu Jul 26, 2021
98e6e65
Add c objects and archives to .gitignore
NobodyXu Jul 26, 2021
0f20f54
Remove build-dep once_cell
NobodyXu Jul 26, 2021
c9e9664
Add target `build_rust` to Makefile
NobodyXu Jul 26, 2021
d8aaa06
Write sqlite3.pc to sqlite3/pkgconfig
NobodyXu Jul 26, 2021
3a30d66
Set SQLITE3_LIB_DIR for dep crate rusqlite
NobodyXu Jul 26, 2021
495f165
Update Makefile: Add sqlite3/compile.sh as dep for libsqlite3.a
NobodyXu Jul 26, 2021
24b363b
Fix sqlite3/compile.sh: Use `-p` when mkdir pkgconfig
NobodyXu Jul 26, 2021
3356440
Enable feature buildtime_bindgen for rusqlite
NobodyXu Jul 26, 2021
a97d353
Update Cargo.lock
NobodyXu Jul 26, 2021
1876432
Create new feature async-sql to disable sqlx and tokio
NobodyXu Jul 27, 2021
e6b3475
Fix env required to use custom libsqlite3.a in cargo_build.sh
NobodyXu Jul 27, 2021
7e6f443
Enable LTO, set panic to "abort" and codegen-units to 1
NobodyXu Jul 27, 2021
6a7174f
Add more rust flags to try to ensure libsqlite3.a is linked with LTO
NobodyXu Jul 27, 2021
09f4f96
Compile `sqlite3/sqlite3.c` with `-march=native`
NobodyXu Jul 27, 2021
207bffc
Set default sym vis to hidden when compiling sqlite3.c
NobodyXu Jul 27, 2021
63af5b0
Fix the lto linking by moving the sqlite3 binding to workspace
NobodyXu Aug 4, 2021
76f1d81
Upload flamegraph for binary basic
NobodyXu Aug 4, 2021
ae2c1f4
Add flamegraph for basic_batched
NobodyXu Aug 4, 2021
385b019
Add flamegraph for basic_batched_wp
NobodyXu Aug 4, 2021
187c93a
Add flamegraph for basic_prep
NobodyXu Aug 4, 2021
f2207cb
Add flamegraph for threaded_batched
NobodyXu Aug 4, 2021
37c78c1
Add flamegraph for threaded_str_bached
NobodyXu Aug 4, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -156,3 +156,11 @@ dmypy.json

# These are backup files generated by rustfmt
**/*.rs.bk

# pkg-config files
*.pc

# c objects and archive
*.o
*.a
*.so
Empty file added .gitmodules
Empty file.
109 changes: 104 additions & 5 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 15 additions & 4 deletions Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,13 +1,24 @@
[package]
name = "fast-sqlite3-inserts"
name = "fast_sqlite3_inserts"
version = "0.1.0"
authors = ["avi <[email protected]>"]
edition = "2018"

[workspace]
members = ["sqlite3-binding"]

[profile.release]
lto = true
panic = "abort"
codegen-units = 1

[features]
async-sql = ["sqlx", "tokio"]

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
sqlx = { version = "0.5.2", features = ["runtime-tokio-native-tls", "sqlite"]}
tokio = {version = "1.5.0", features = ["full"]}
sqlx = { version = "0.5.2", features = ["runtime-tokio-native-tls", "sqlite"], optional = true }
tokio = { version = "1.5.0", features = ["full"], optional = true }
rand = "0.8.3"
num_cpus = "1.0"
rusqlite = "0.25.3"
sqlite3-binding = { path = "sqlite3-binding" }
11 changes: 10 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
@@ -1,8 +1,17 @@
# runs each of the scripts one after another, prints the measurements to stdout
.SILENT:

.PHONY: build_rust

export TZ := ":Asia/Kolkata"

build_rust: sqlite3/libsqlite3.a
./cargo_build.sh

sqlite3/libsqlite3.a: sqlite3/sqlite3.c sqlite3/sqlite3.h sqlite3/compile.sh
sqlite3/compile.sh
#cargo clean -p rusqlite

busy-python:
echo
echo "$$(date)" "[PYTHON] busy_loop.py (100_000_000) iterations"
Expand Down Expand Up @@ -30,4 +39,4 @@ busy-py-all: busy-python busy-pypy
busy-rust-all: busy-rust busy-rust-thread

busy-all: busy-py-all busy-rust-all


18 changes: 18 additions & 0 deletions cargo_build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/sh -ex

base="$(dirname `realpath $0`)"

cd "$base"

# SQLITE3_LIB_DIR is necessary the directory of statically linked library.
# Otherwise, libsqlite3-sys would just use the global dynamic library
export SQLITE3_LIB_DIR="$base/sqlite3"
# SQLITE3_STATIC is necessary to statically linked with libsqlite3.a
export SQLITE3_STATIC=1
# SQLITE3_INCLUDE_DIR is necessary since libsqlite3-sys does not use sqlite3.pc
# found using `SQLITE3_LIB_DIR` for locating the header.
export SQLITE3_INCLUDE_DIR="$SQLITE3_LIB_DIR"

export PKG_CONFIG_PATH="$SQLITE3_LIB_DIR/pkgconfig"

exec cargo build --release -vv $@
Loading