Skip to content

Commit 5c04502

Browse files
usamoiusamoi
authored
feat: fp16 vector (#178)
* feat: fp16 vector Signed-off-by: usamoi <[email protected]> * feat: detect avx512fp16 Signed-off-by: usamoi <[email protected]> * fix: install clang-16 for ci Signed-off-by: usamoi <[email protected]> * fix: clippy Signed-off-by: usamoi <[email protected]> * fix: rename c to pgvectorsc Signed-off-by: usamoi <[email protected]> * feat: hand-writing avx512fp16 Signed-off-by: usamoi <[email protected]> * fix: index on fp16 Signed-off-by: usamoi <[email protected]> * feat: hand-writing avx2 Signed-off-by: usamoi <[email protected]> * fix: clippy Signed-off-by: usamoi <[email protected]> * fix: add rerun in build script Signed-off-by: usamoi <[email protected]> * fix: cross compilation Signed-off-by: usamoi <[email protected]> * fix: do not leave uninitialized bytes in datatype input function Signed-off-by: usamoi <[email protected]> * fix: compiler built-in function calling convention workaround Signed-off-by: usamoi <[email protected]> * fix: cross compile on aarch64 Signed-off-by: usamoi <[email protected]> * fix: fix detect avx512fp16 Signed-off-by: usamoi <[email protected]> * fix: avx512 codegen by multiversion Signed-off-by: usamoi <[email protected]> * fix: enable more target features for c Signed-off-by: usamoi <[email protected]> * fix: use tensorchord/stdarch Signed-off-by: usamoi <[email protected]> * fix: ci Signed-off-by: usamoi <[email protected]> * fix: remove no-run cross test Signed-off-by: usamoi <[email protected]> * fix: vbase Signed-off-by: usamoi <[email protected]> * fix: error and document Signed-off-by: usamoi <[email protected]> * [skip ci] Signed-off-by: usamoi <[email protected]> --------- Signed-off-by: usamoi <[email protected]>
1 parent 2ab7611 commit 5c04502

File tree

146 files changed

+7435
-4107
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

146 files changed

+7435
-4107
lines changed

‎.cargo/config.toml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,3 +4,13 @@ rustdocflags = ["--document-private-items"]
44
[target.'cfg(target_os="macos")']
55
# Postgres symbols won't be available until runtime
66
rustflags = ["-Clink-arg=-Wl,-undefined,dynamic_lookup"]
7+
8+
[target.x86_64-unknown-linux-gnu]
9+
linker = "x86_64-linux-gnu-gcc"
10+
11+
[target.aarch64-unknown-linux-gnu]
12+
linker = "aarch64-linux-gnu-gcc"
13+
14+
[env]
15+
BINDGEN_EXTRA_CLANG_ARGS_x86_64_unknown_linux_gnu = "-isystem /usr/x86_64-linux-gnu/include/ -ccc-gcc-name x86_64-linux-gnu-gcc"
16+
BINDGEN_EXTRA_CLANG_ARGS_aarch64_unknown_linux_gnu = "-isystem /usr/aarch64-linux-gnu/include/ -ccc-gcc-name aarch64-linux-gnu-gcc"

‎.github/workflows/check.yml

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ on:
66
paths:
77
- ".cargo/**"
88
- ".github/**"
9+
- "crates/**"
910
- "scripts/**"
1011
- "src/**"
1112
- "tests/**"
@@ -18,6 +19,7 @@ on:
1819
paths:
1920
- ".cargo/**"
2021
- ".github/**"
22+
- "crates/**"
2123
- "scripts/**"
2224
- "src/**"
2325
- "tests/**"
@@ -90,11 +92,16 @@ jobs:
9092
- name: Format check
9193
run: cargo fmt --check
9294
- name: Semantic check
93-
run: cargo clippy --no-default-features --features "pg${{ matrix.version }} pg_test"
95+
run: |
96+
cargo clippy --no-default-features --features "pg${{ matrix.version }} pg_test" --target x86_64-unknown-linux-gnu
97+
cargo clippy --no-default-features --features "pg${{ matrix.version }} pg_test" --target aarch64-unknown-linux-gnu
9498
- name: Debug build
95-
run: cargo build --no-default-features --features "pg${{ matrix.version }} pg_test"
99+
run: |
100+
cargo build --no-default-features --features "pg${{ matrix.version }} pg_test" --target x86_64-unknown-linux-gnu
101+
cargo build --no-default-features --features "pg${{ matrix.version }} pg_test" --target aarch64-unknown-linux-gnu
96102
- name: Test
97-
run: cargo test --all --no-default-features --features "pg${{ matrix.version }} pg_test" -- --nocapture
103+
run: |
104+
cargo test --all --no-fail-fast --no-default-features --features "pg${{ matrix.version }} pg_test" --target x86_64-unknown-linux-gnu
98105
- name: Install release
99106
run: ./scripts/ci_install.sh
100107
- name: Sqllogictest

‎.github/workflows/release.yml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -112,15 +112,17 @@ jobs:
112112
- uses: mozilla-actions/[email protected]
113113
- name: Prepare
114114
run: |
115-
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
115+
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" >> /etc/apt/sources.list.d/pgdg.list'
116+
sudo sh -c 'echo "deb http://apt.llvm.org/$(lsb_release -cs)/ llvm-toolchain-$(lsb_release -cs)-16 main" >> /etc/apt/sources.list'
116117
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
118+
wget --quiet -O - https://apt.llvm.org/llvm-snapshot.gpg.key | sudo apt-key add -
117119
sudo apt-get update
118120
sudo apt-get -y install libpq-dev postgresql-${{ matrix.version }} postgresql-server-dev-${{ matrix.version }}
121+
sudo apt-get -y install clang-16
119122
cargo install cargo-pgrx --git https://github.com/tensorchord/pgrx.git --rev $(cat Cargo.toml | grep "pgrx =" | awk -F'rev = "' '{print $2}' | cut -d'"' -f1)
120123
cargo pgrx init --pg${{ matrix.version }}=/usr/lib/postgresql/${{ matrix.version }}/bin/pg_config
121124
if [[ "${{ matrix.arch }}" == "arm64" ]]; then
122125
sudo apt-get -y install crossbuild-essential-arm64
123-
rustup target add aarch64-unknown-linux-gnu
124126
fi
125127
- name: Build Release
126128
id: build_release
@@ -130,8 +132,6 @@ jobs:
130132
mkdir ./artifacts
131133
cargo pgrx package
132134
if [[ "${{ matrix.arch }}" == "arm64" ]]; then
133-
export CARGO_TARGET_AARCH64_UNKNOWN_LINUX_GNU_LINKER=aarch64-linux-gnu-gcc
134-
export BINDGEN_EXTRA_CLANG_ARGS_aarch64_unknown_linux_gnu="-target aarch64-unknown-linux-gnu -isystem /usr/aarch64-linux-gnu/include/ -ccc-gcc-name aarch64-linux-gnu-gcc"
135135
cargo build --target aarch64-unknown-linux-gnu --release --features "pg${{ matrix.version }}" --no-default-features
136136
mv ./target/aarch64-unknown-linux-gnu/release/libvectors.so ./target/release/vectors-pg${{ matrix.version }}/usr/lib/postgresql/${{ matrix.version }}/lib/vectors.so
137137
fi

‎.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,4 +6,5 @@
66
.vscode
77
.ignore
88
__pycache__
9-
.pytest_cache
9+
.pytest_cache
10+
rustc-ice-*.txt

0 commit comments

Comments
 (0)