Skip to content

Commit

Permalink
fix: revert to dynlib on Windows
Browse files Browse the repository at this point in the history
  • Loading branch information
@chaaz
chaaz committed Aug 19, 2021
1 parent 392751e commit 69a2efb
Show file tree
Hide file tree
Showing 13 changed files with 192 additions and 175 deletions.
127 changes: 38 additions & 89 deletions .github/snippets/get-cargo-os.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,50 +6,20 @@ value:
- name: Install MacOS dependencies
if: matrix.os == 'macos-latest'
run: brew update; brew install gpgme
- name: Setup Msys2 on Windows
if: matrix.os == 'windows-latest'
uses: msys2/setup-msys2@v2
with:
update: true
msystem: mingw64
path-type: inherit
install: >-
base-devel
git
mingw-w64-x86_64-toolchain
- name: Install Windows dependencies
if: matrix.os == 'windows-latest'
env:
RUNNER_TEMP: ${{ runner.temp }}
working-directory: ${{ runner.temp }}
shell: msys2 {0}
# Short version, the choco install only provideds 32-bit dynamic
# libraries for some reason. We want to try 64-bit static links, so
# we have to build all the deps ourselves. Also, the gpgme python
# install scripts use subprocess.Popen and friends, which are all
# horked in msys2, so we'll stick to only C and C++ libs.
# Use choco to install gnupg; directly downloading + running the
# gnupg.exe installer silently fails somehow after setting the
# registry, but before/during writing to the `C:\Program Files
# (x86)\gnupg` filesystem. Also see
# https://github.com/actions/virtual-environments/issues/2876 to
# ensure that the $env:PATH doesn't run out of characters when
# running choco.
run: |
curl https://gnupg.org/ftp/gcrypt/libgpg-error/libgpg-error-1.42.tar.bz2 -o libgpg-error-1.42.tar.bz2
tar -jxf libgpg-error-1.42.tar.bz2
curl https://gnupg.org/ftp/gcrypt/libassuan/libassuan-2.5.5.tar.bz2 -o libassuan-2.5.5.tar.bz2
tar -jxf libassuan-2.5.5.tar.bz2
curl https://gnupg.org/ftp/gcrypt/gpgme/gpgme-1.16.0.tar.bz2 -o gpgme-1.16.0.tar.bz2
tar -jxf gpgme-1.16.0.tar.bz2
pushd libgpg-error-1.42
./configure --enable-static
make
make install
popd
pushd libassuan-2.5.5
./configure --enable-static
make
make install
popd
pushd gpgme-1.16.0
./configure --enable-static --enable-languages=cl,cpp
make
make install
popd
$env:PATH = "C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\ProgramData\chocolatey\bin"
[Environment]::SetEnvironmentVariable("Path", $env:PATH, "Machine")
choco install -y gnupg
echo "C:\Program Files (x86)\GnuPG\bin" >> $env:GITHUB_PATH
- name: Get cargo stable
if: matrix.os != 'windows-latest'
uses: actions-rs/toolchain@v1
Expand All @@ -63,69 +33,48 @@ value:
toolchain: nightly
components: rustfmt
- name: Get Windows cargo stable
# After dozens of stabs at this, we have decided to build the gpgme
# dependencies ourselves, in order to provide a 64-bit statically
# linked binary. (On windows, the gpgme pre-built packages from
# chocolatey etc only provide 32-bit dynamic libs.) There are still
# plenty of issues on this route, though, including:
#
# 1. You can't use the msvc target to build gnupg; the gpg libraries
# are built with the GNU toolchain, and are apparently nigh-on
# impossible to build in Studio.
# 2. The GNU toolchain included in the Windows runner is
# different than the builtin rust installation. That *shouldn't*
# be a problem (as of 2020-02, rust is supposed to by favor the
# system toolchain), but somehow it is.
# 3. The GNU runner toolchain still seems to be half-baked, and is
# missing libraries. Also, its compiler isn't even on the PATH,
# making it impossible for cargo to find, and thus to include its
# associated libs. You can't even build GNU C programs
# without explicitly adding it yourself before you compile. See
# https://github.com/actions/virtual-environments/issues/2549
# I give up. We are going to distribute Windows Versio as a 32-bit
# dynamic-link to gpgme, and include documentation that gpgme is
# required to run.
#
# However, there is a `msys2/setup-msys2@v2` action available, which
# provides a more complete dev environment for building gnu-based
# windows applications, and handles many (but not quite all) of the
# path problems.
# Commit `88e4066` (v0.5.3, 20210818023324Z) of this repo contains a
# static-link for all three platforms build on GitHub Actions. This
# is a big step over the previous successful build `d0d3533`
# (20210803124432T-0600) which inadvertantly dynamically linked the
# gpgme libraries on all platforms.
#
# So, it looks like the best way to build this is to use the msvc
# toolchain to cross-compile to the gnu target inside the msys2
# shell. This was actually recommended by various reputable internet
# sources (none of which I can find right now). Also, we need to
# ensure that the correct target is selected on Windows builds, and
# that the cross-compile toolchain can be found. So for Windows
# builds, we need to
# For Windows especially, `88e4066` was the culmination of a lot
# of effort: we downloaded and built the gpg-error, assuan, and
# gpgme dependencies manually, since the publically-available
# pre-built dependencies (via chocolatey, et al) only contain
# dynamic 32-bit libs, and had to use the msys2 shell to incorporate
# the gnu toolchain. However, even then the resulting Windows
# executable couldn't bind to `gpg-agent` or fork `gpgme-w32spawn`,
# likely as a result of the static linkage that the gpgme system was
# not designed to accomodate.
#
# - install the msys2 shell and development tools. (see the `uses:
# msys2/setup-msys2@v2` up above)
# - ensure that the stable- or nightly-x86_64-pc-windows-msvc
# cargo toolchain is installed and selected.
# - ensure that the cross-compile x86_64-pc-windows-gnu target is
# installed for that cargo toolchain.
# - ensure that the cross-compile toolchain tools are actually
# available (`windows-latest` does satisfy this with msys2/mingw)
# - use the msys2 shell and `--release --target
# x86_64-pc-windows-gnu` for all cargo commands.
# - remember that this will build to the
# `target/i686-pc-windows-gnu/release` directory.
# So, we've reverted the Windows build steps back to their dynamic
# 32-bit instructions, which are still somewhat convoluted: we need
# to use the 32-bit-msvc toolchain to cross-compile to the
# 32-bit-gnu target, and explicitly add the 32-bit compiler on PATH
# a la https://github.com/actions/virtual-environments/issues/2549.
if: matrix.os == 'windows-latest'
uses: actions-rs/toolchain@v1
with:
toolchain: stable-x86_64-pc-windows-msvc
target: x86_64-pc-windows-gnu
toolchain: stable-i686-pc-windows-msvc
target: i686-pc-windows-gnu
components: clippy
default: true
- name: Get Windows cargo nightly
if: matrix.os == 'windows-latest'
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-x86_64-pc-windows-msvc
target: x86_64-pc-windows-gnu
toolchain: nightly-i686-pc-windows-msvc
target: i686-pc-windows-gnu
components: rustfmt
- name: Update Windows target configuration
if: matrix.os == 'windows-latest'
shell: msys2 {0}
run: rustup set default-host x86_64-pc-windows-gnu
run: rustup set default-host i686-pc-windows-gnu
- name: Find paths
id: cargo-find-paths
run: 'echo ::set-output name=cargo-lock-glob::"${{ matrix.root }}"/**/Cargo.lock'
Expand Down
28 changes: 14 additions & 14 deletions .github/snippets/job-publish-versio.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ value:
rustflags: '-D warnings'
bin_name: 'versio'
- os: windows-latest
target: 'x86_64-pc-windows-gnu.exe'
target: 'x86_64-pc-win32.exe'
root: '.'
rustflags: '-D warnings'
bin_name: 'versio.exe'
Expand Down Expand Up @@ -62,23 +62,23 @@ value:
GPGME_INCLUDE: /usr/local/opt/libassuan/include:/usr/local/opt/gpgme/include
GPGME_LIB_DIR: /usr/local/opt/libassuan/lib:/usr/local/opt/gpgme/lib
GPGME_LIBS: static=gpgme:static=assuan
- name: Add mingw32 to PATH
# See comments in `../snippets/get-cargo-os.yml`
if: matrix.os == 'windows-latest'
run: |
echo "C:\msys64\mingw32\bin" >> $env:GITHUB_PATH
- name: Build Windows binary
# We're not going to try to statically link for Windows; see
# comments in `../snippets/get-cargo-os.yml`. Because we aren't
# statically linking, we don't have to specify the build locations
# in the environment; the build scripts will find them via the
# registry and/or the `gpgme-config` / `gpg-error-config`
# programs.
if: matrix.os == 'windows-latest'
shell: msys2 {0}
run: cargo build --release --target x86_64-pc-windows-gnu
run: cargo build --release --target i686-pc-windows-gnu
working-directory: ${{ matrix.root }}
env:
RUSTFLAGS: ${{ matrix.rustflags }}
# See comments in the above "Build Ubuntu binary" step to
# explain these env. Now that we're building the gpg libs
# ourselves, we need to use the locations of where `make
# install` in the `Install Windows dependencies` step puts them.
LIBGPG_ERROR_INCLUDE: '/mingw64/include'
LIBGPG_ERROR_LIB_DIR: '/mingw64/lib'
LIBGPG_ERROR_LIBS: 'static=gpg-error'
GPGME_INCLUDE: '/mingw64/include'
GPGME_LIB_DIR: '/mingw64/lib'
GPGME_LIBS: 'static=gpgme;static=assuan'
- name: Upload binary
if: matrix.os != 'windows-latest'
uses: actions/upload-release-asset@v1
Expand All @@ -92,6 +92,6 @@ value:
uses: actions/upload-release-asset@v1
with:
upload_url: ${{ needs.github-publish.outputs.upload_url }}
asset_path: target/x86_64-pc-windows-gnu/release/${{ matrix.bin_name }}
asset_path: target/i686-pc-windows-gnu/release/${{ matrix.bin_name }}
asset_name: versio__${{ matrix.target }}
asset_content_type: application/octet-stream
10 changes: 10 additions & 0 deletions .github/snippets/steps-github-publish.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,16 @@ value:
New to Versio? The [repository](https://github.com/chaaz/versio) is the best place to learn about Versio and what it can do. If you want to report a bug or request a feature, you can do so at our [Issues](https://github.com/chaaz/versio/issues) link, but we ask you first read the [Troubleshooting](https://github.com/chaaz/versio/blob/main/docs/troubleshooting.md) page learn about problems and their solutions.
DEPENDENCIES:
On MacOS and Linux, you must have GPG tools installed. These tools are installed by default on most distributions of Linux and MacOS. If you can run `git`, then you probably have the necessary tools already installed.
On Windows, you need to install the GpgME package, which is available [here](https://gnupg.org/ftp/gcrypt/binary/gnupg-w32-2.3.1_20210420.exe) (Signature and checksum available from its [parent directory](https://gnupg.org/ftp/gcrypt/binary/)).
If you have any questions, you should view the [Dependency page](https://github.com/chaaz/versio/blob/main/docs/dependencies.md) for information about installing and testing dependencies.
INSTALLATION:
To install, follow the instructions for your platform; some files might be zipped for improved download speed. For example, on MacOS you can do something like this (assuming `~/bin` exists and is in your PATH):
```sh
Expand Down
43 changes: 13 additions & 30 deletions .github/workflows/github-publish.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -102,7 +102,7 @@ jobs:
with:
tag_name: "${{ steps.find-version.outputs.version }}"
release_name: "(TODO) RELEASE FOR ${{ steps.find-version.outputs.version }}"
body: "(TODO) SUMMARY\n- (TODO) FEATURE1\n\nNew to Versio? The [repository](https://github.com/chaaz/versio) is the best place to learn about Versio and what it can do. If you want to report a bug or request a feature, you can do so at our [Issues](https://github.com/chaaz/versio/issues) link, but we ask you first read the [Troubleshooting](https://github.com/chaaz/versio/blob/main/docs/troubleshooting.md) page learn about problems and their solutions.\n\nTo install, follow the instructions for your platform; some files might be zipped for improved download speed. For example, on MacOS you can do something like this (assuming `~/bin` exists and is in your PATH):\n\n```sh\ncurl -L https://github.com/chaaz/versio/releases/download/${{ steps.find-version.outputs.version }}/versio__x86_64-apple-darwin -o ~/bin/versio\nchmod +x ~/bin/versio\n```\n\n**MacOS:** download `versio__x86_64-apple-darwin`, copy to `versio` in your PATH.\n**GNU Linux 64:** download `versio__x86_64-unknown-linux-gnu`, copy to `versio` in your PATH.\n**Windows:** download `versio__x86_64-pc-win32.exe`, copy to `versio.exe` in your %PATH.\n"
body: "(TODO) SUMMARY\n- (TODO) FEATURE1\n\nNew to Versio? The [repository](https://github.com/chaaz/versio) is the best place to learn about Versio and what it can do. If you want to report a bug or request a feature, you can do so at our [Issues](https://github.com/chaaz/versio/issues) link, but we ask you first read the [Troubleshooting](https://github.com/chaaz/versio/blob/main/docs/troubleshooting.md) page learn about problems and their solutions.\n\nDEPENDENCIES:\n\nOn MacOS and Linux, you must have GPG tools installed. These tools are installed by default on most distributions of Linux and MacOS. If you can run `git`, then you probably have the necessary tools already installed.\n\nOn Windows, you need to install the GpgME package, which is available [here](https://gnupg.org/ftp/gcrypt/binary/gnupg-w32-2.3.1_20210420.exe) (Signature and checksum available from its [parent directory](https://gnupg.org/ftp/gcrypt/binary/)).\n\nIf you have any questions, you should view the [Dependency page](https://github.com/chaaz/versio/blob/main/docs/dependencies.md) for information about installing and testing dependencies.\n\nINSTALLATION: \n\nTo install, follow the instructions for your platform; some files might be zipped for improved download speed. For example, on MacOS you can do something like this (assuming `~/bin` exists and is in your PATH):\n\n```sh\ncurl -L https://github.com/chaaz/versio/releases/download/${{ steps.find-version.outputs.version }}/versio__x86_64-apple-darwin -o ~/bin/versio\nchmod +x ~/bin/versio\n```\n\n**MacOS:** download `versio__x86_64-apple-darwin`, copy to `versio` in your PATH.\n**GNU Linux 64:** download `versio__x86_64-unknown-linux-gnu`, copy to `versio` in your PATH.\n**Windows:** download `versio__x86_64-pc-win32.exe`, copy to `versio.exe` in your %PATH.\n"
draft: true
prerelease: false
commitish: main
Expand All @@ -122,7 +122,7 @@ jobs:
rustflags: "-D warnings"
bin_name: versio
- os: windows-latest
target: x86_64-pc-windows-gnu.exe
target: x86_64-pc-win32.exe
root: "."
rustflags: "-D warnings"
bin_name: versio.exe
Expand All @@ -139,21 +139,9 @@ jobs:
- name: Install MacOS dependencies
if: "matrix.os == 'macos-latest'"
run: brew update; brew install gpgme
- name: Setup Msys2 on Windows
if: "matrix.os == 'windows-latest'"
uses: msys2/setup-msys2@v2
with:
update: true
msystem: mingw64
path-type: inherit
install: base-devel git mingw-w64-x86_64-toolchain
- name: Install Windows dependencies
if: "matrix.os == 'windows-latest'"
env:
RUNNER_TEMP: "${{ runner.temp }}"
working-directory: "${{ runner.temp }}"
shell: "msys2 {0}"
run: "curl https://gnupg.org/ftp/gcrypt/libgpg-error/libgpg-error-1.42.tar.bz2 -o libgpg-error-1.42.tar.bz2\ntar -jxf libgpg-error-1.42.tar.bz2\ncurl https://gnupg.org/ftp/gcrypt/libassuan/libassuan-2.5.5.tar.bz2 -o libassuan-2.5.5.tar.bz2\ntar -jxf libassuan-2.5.5.tar.bz2\ncurl https://gnupg.org/ftp/gcrypt/gpgme/gpgme-1.16.0.tar.bz2 -o gpgme-1.16.0.tar.bz2\ntar -jxf gpgme-1.16.0.tar.bz2\npushd libgpg-error-1.42\n./configure --enable-static\nmake\nmake install\npopd\npushd libassuan-2.5.5\n./configure --enable-static\nmake\nmake install\npopd\npushd gpgme-1.16.0\n./configure --enable-static --enable-languages=cl,cpp\nmake\nmake install\npopd\n"
run: "$env:PATH = \"C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\ProgramData\\chocolatey\\bin\"\n[Environment]::SetEnvironmentVariable(\"Path\", $env:PATH, \"Machine\")\nchoco install -y gnupg\necho \"C:\\Program Files (x86)\\GnuPG\\bin\" >> $env:GITHUB_PATH\n"
- name: Get cargo stable
if: "matrix.os != 'windows-latest'"
uses: actions-rs/toolchain@v1
Expand All @@ -170,21 +158,20 @@ jobs:
if: "matrix.os == 'windows-latest'"
uses: actions-rs/toolchain@v1
with:
toolchain: stable-x86_64-pc-windows-msvc
target: x86_64-pc-windows-gnu
toolchain: stable-i686-pc-windows-msvc
target: i686-pc-windows-gnu
components: clippy
default: true
- name: Get Windows cargo nightly
if: "matrix.os == 'windows-latest'"
uses: actions-rs/toolchain@v1
with:
toolchain: nightly-x86_64-pc-windows-msvc
target: x86_64-pc-windows-gnu
toolchain: nightly-i686-pc-windows-msvc
target: i686-pc-windows-gnu
components: rustfmt
- name: Update Windows target configuration
if: "matrix.os == 'windows-latest'"
shell: "msys2 {0}"
run: rustup set default-host x86_64-pc-windows-gnu
run: rustup set default-host i686-pc-windows-gnu
- name: Find paths
id: cargo-find-paths
run: "echo ::set-output name=cargo-lock-glob::\"${{ matrix.root }}\"/**/Cargo.lock"
Expand Down Expand Up @@ -217,19 +204,15 @@ jobs:
GPGME_INCLUDE: "/usr/local/opt/libassuan/include:/usr/local/opt/gpgme/include"
GPGME_LIB_DIR: "/usr/local/opt/libassuan/lib:/usr/local/opt/gpgme/lib"
GPGME_LIBS: "static=gpgme:static=assuan"
- name: Add mingw32 to PATH
if: "matrix.os == 'windows-latest'"
run: "echo \"C:\\msys64\\mingw32\\bin\" >> $env:GITHUB_PATH\n"
- name: Build Windows binary
if: "matrix.os == 'windows-latest'"
shell: "msys2 {0}"
run: cargo build --release --target x86_64-pc-windows-gnu
run: cargo build --release --target i686-pc-windows-gnu
working-directory: "${{ matrix.root }}"
env:
RUSTFLAGS: "${{ matrix.rustflags }}"
LIBGPG_ERROR_INCLUDE: /mingw64/include
LIBGPG_ERROR_LIB_DIR: /mingw64/lib
LIBGPG_ERROR_LIBS: static=gpg-error
GPGME_INCLUDE: /mingw64/include
GPGME_LIB_DIR: /mingw64/lib
GPGME_LIBS: static=gpgme;static=assuan
- name: Upload binary
if: "matrix.os != 'windows-latest'"
uses: actions/upload-release-asset@v1
Expand All @@ -243,6 +226,6 @@ jobs:
uses: actions/upload-release-asset@v1
with:
upload_url: "${{ needs.github-publish.outputs.upload_url }}"
asset_path: "target/x86_64-pc-windows-gnu/release/${{ matrix.bin_name }}"
asset_path: "target/i686-pc-windows-gnu/release/${{ matrix.bin_name }}"
asset_name: "versio__${{ matrix.target }}"
asset_content_type: application/octet-stream
Loading

0 comments on commit 69a2efb

Please sign in to comment.