Skip to content

Snow-2043523: windows 313 support #2275

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 72 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
195908a
deprecate python 3.8
sfc-gh-mkeller Nov 11, 2024
859db24
adding Python 3.13 builds
sfc-gh-mkeller Nov 13, 2024
0f3e6a4
pre-commit autoupdate
sfc-gh-mkeller Nov 13, 2024
8db96c5
fixing all files with pre-commit
sfc-gh-mkeller Nov 13, 2024
a707ef4
bump cibuildwheel used
sfc-gh-mkeller Nov 13, 2024
40e555d
changelog entry
sfc-gh-mkeller Nov 13, 2024
01384a6
update numpy.nan usage
sfc-gh-mkeller Nov 13, 2024
92ee331
fix olddriver tests
sfc-gh-mkeller Nov 14, 2024
7390bd9
update windows image used for builds and tests
sfc-gh-mkeller Nov 14, 2024
fb4b1a8
use already built wheels
sfc-gh-mkeller Nov 18, 2024
c0c1df2
pre-commit fixes
sfc-gh-mkeller Jan 29, 2025
1ea64c6
bump numpy to <2.1.0
sfc-gh-mkeller Jan 29, 2025
a5fd0dc
adding Python 3.13 Dockerfile
sfc-gh-mkeller Jan 29, 2025
580db9d
fix numpy issues
sfc-gh-mkeller Feb 5, 2025
e1faded
SNOW-1763673: Merged main
sfc-gh-fpawlowski Mar 4, 2025
0536a8e
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 5, 2025
bdc2956
SNOW-1763673: Added running python 313 tests on Jenkins and depricate…
sfc-gh-fpawlowski Mar 5, 2025
ffdf66b
Merge remote-tracking branch 'origin/mkeller/SNOW-1763673/python3.13-…
sfc-gh-fpawlowski Mar 5, 2025
fffc947
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 11, 2025
e8655b8
SNOW-1763673: Added python 313 to tox dependencies
sfc-gh-fpawlowski Mar 11, 2025
c8a5aab
Merge remote-tracking branch 'origin/mkeller/SNOW-1763673/python3.13-…
sfc-gh-fpawlowski Mar 11, 2025
f37ad7e
SNOW-1763673: Reverted path env settinh
sfc-gh-fpawlowski Mar 11, 2025
1d3e227
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 17, 2025
15527f7
SNOW-1763673: Fixed failing old test with ubuntu downgrade
sfc-gh-fpawlowski Mar 17, 2025
b843cbb
SNOW-1763673: Fixed fips dockerfile to include the new python version…
sfc-gh-fpawlowski Mar 17, 2025
a3876d8
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 17, 2025
8651faa
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 18, 2025
b8fa4ad
SNOW-1763673: Hardcoded list of versions of python supported fixed in…
sfc-gh-fpawlowski Mar 18, 2025
6132c8b
SNOW-1763673: Build darwin cleanup
sfc-gh-fpawlowski Mar 18, 2025
f5aea74
SNOW-1763673: Added Jenkins builds
sfc-gh-fpawlowski Mar 18, 2025
985108d
SNOW-1763673: Added Jenkins builds
sfc-gh-fpawlowski Mar 18, 2025
13dd00c
SNOW-1763673: Added Jenkins builds
sfc-gh-fpawlowski Mar 18, 2025
86c839c
SNOW-1763673: Removed installpkg dist/*.whl to run tox tests command …
sfc-gh-fpawlowski Mar 18, 2025
66cbee6
SNOW-1763673: Upgraded windows image version to make it capable of co…
sfc-gh-fpawlowski Mar 18, 2025
19787f1
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 18, 2025
d2afbf3
SNOW-1763673: Dockerfile moved forward to allow building numpy for 3.13
sfc-gh-fpawlowski Mar 19, 2025
14f4c22
SNOW-1763673: Dockerfile tools added to allow building numpy for 3.13
sfc-gh-fpawlowski Mar 25, 2025
573f975
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 25, 2025
f1f61b2
SNOW-1763673: Dockerfile tools added to allow building numpy for 3.13
sfc-gh-fpawlowski Mar 25, 2025
0aa1c03
SNOW-1763673: Printing logs of fail
sfc-gh-fpawlowski Mar 26, 2025
f992afe
SNOW-1763673: Printing logs of fail
sfc-gh-fpawlowski Mar 26, 2025
6bf6074
SNOW-1763673: Dockerfile tools added to allow building numpy for 3.13
sfc-gh-fpawlowski Mar 26, 2025
a5872d1
SNOW-1763673: Dockerfile tools added to allow building numpy for 3.13
sfc-gh-fpawlowski Mar 26, 2025
2800609
SNOW-1763673: rust added cargo
sfc-gh-fpawlowski Mar 26, 2025
a132aab
SNOW-1763673: Cleaned up and added release notes
sfc-gh-fpawlowski Mar 26, 2025
33bd968
Merge branch 'refs/heads/main' into mkeller/SNOW-1763673/python3.13-s…
sfc-gh-fpawlowski Mar 26, 2025
9206d01
SNOW-1763673: Excluded windows builds fo 3.13
sfc-gh-fpawlowski Mar 26, 2025
3382f9f
SNOW-1763673: Jenkins Dockerfile fixed with cargo
sfc-gh-fpawlowski Mar 26, 2025
6119990
SNOW-1763673: Jenkins file update to use 3.9 for olddriver
sfc-gh-fpawlowski Mar 26, 2025
c445b8d
SNOW-1763673: Removed manual PATH setting in Dockerfiles
sfc-gh-fpawlowski Mar 26, 2025
0a07154
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 27, 2025
1f3e436
SNOW-1763673: Removed unnecessary changes
sfc-gh-fpawlowski Mar 27, 2025
605ffc7
SNOW-1763673: Reverted numpy NAN change
sfc-gh-fpawlowski Mar 27, 2025
2a5ecc4
Merge branch 'main' into mkeller/SNOW-1763673/python3.13-support
sfc-gh-fpawlowski Mar 27, 2025
d0c85ac
SNOW-1763673: Reverted nanoarrow
sfc-gh-fpawlowski Mar 27, 2025
fe335a7
SNOW-1763673: Reverted nanoarrow reversion
sfc-gh-fpawlowski Mar 27, 2025
892c036
SNOW-1763673: Added numpy bool_ type hint
sfc-gh-fpawlowski Mar 27, 2025
f3ccf8c
SNOW-1763673: Bump-up numpy
sfc-gh-fpawlowski Apr 16, 2025
0e2cfca
Merge branch 'main' into SNOW-1763673-python3.13-support
sfc-gh-fpawlowski Apr 16, 2025
fc4f3cb
SNOW-1763673: Numpy version
sfc-gh-fpawlowski Apr 16, 2025
8c0ffa7
SNOW-1763673: Test ubuntu bump for old driver
sfc-gh-fpawlowski Apr 16, 2025
81d880b
SNOW-1763673: [MAKE SURE IT SHOULD NOT BE REVERTED] Removed rust manu…
sfc-gh-fpawlowski Apr 16, 2025
5f82ee9
SNOW-1763673: Try fix test_abc
sfc-gh-fpawlowski Apr 16, 2025
cb4c176
Revert "SNOW-1763673: [MAKE SURE IT SHOULD NOT BE REVERTED] Removed r…
sfc-gh-fpawlowski Apr 16, 2025
ed756a4
SNOW-1763673: Added cargo install to darwin shell
sfc-gh-fpawlowski Apr 16, 2025
897aa51
SNOW-1763673: Nvmind
sfc-gh-fpawlowski Apr 16, 2025
dec4e68
Merge remote-tracking branch 'origin/main' into SNOW-2043523-windows-…
sfc-gh-fpawlowski Apr 16, 2025
3977689
SNOW-2043523: Isabs fix
sfc-gh-fpawlowski Apr 16, 2025
c238333
Merge branch 'main' into SNOW-2043523-windows-313-support
sfc-gh-fpawlowski May 7, 2025
2df4cb7
SNOW-2043523: Test original error
sfc-gh-fpawlowski May 7, 2025
9de2c7e
SNOW-2043523: Test original error
sfc-gh-fpawlowski May 7, 2025
6038ac5
SNOW-2043523: Test original error
sfc-gh-fpawlowski May 7, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
343 changes: 323 additions & 20 deletions .github/workflows/build_test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,8 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
# python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
python-version: ["3.12", "3.13"]
steps:
- uses: actions/checkout@v4
- name: Set up Python
Expand All @@ -75,15 +76,16 @@ jobs:
os:
- image: ubuntu-latest
id: manylinux_x86_64
- image: ubuntu-latest
id: manylinux_aarch64
# - image: ubuntu-latest
# id: manylinux_aarch64
- image: windows-latest
id: win_amd64
- image: macos-latest
id: macosx_x86_64
- image: macos-latest
id: macosx_arm64
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
# - image: macos-latest
# id: macosx_x86_64
# - image: macos-latest
# id: macosx_arm64
# python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
python-version: ["3.12", "3.13"]
name: Build ${{ matrix.os.id }}-py${{ matrix.python-version }}
runs-on: ${{ matrix.os.image }}
steps:
Expand Down Expand Up @@ -125,23 +127,24 @@ jobs:
fail-fast: false
matrix:
os:
- image_name: ubuntu-latest
download_name: manylinux_x86_64
- image_name: macos-latest
download_name: macosx_x86_64
# - image_name: ubuntu-latest
# download_name: manylinux_x86_64
# - image_name: macos-latest
# download_name: macosx_x86_64
- image_name: windows-latest
download_name: win_amd64
python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
# python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
python-version: ["3.12", "3.13"]
cloud-provider: [aws, azure, gcp]
# TODO: When there are prebuilt wheels accessible for our dependencies (i.e. numpy)
# TODO(SNOW-2043523): When there are prebuilt wheels accessible for our dependencies (i.e. numpy)
# for Python 3.13 windows runs can be re-enabled. Currently, according to numpy:
# "Numpy built with MINGW-W64 on Windows 64 bits is experimental, and only available for
# testing. You are advised not to use it for production."
exclude:
- os:
image_name: windows-latest
download_name: win_amd64
python-version: "3.13"
# exclude:
# - os:
# image_name: windows-latest
# download_name: win_amd64
# python-version: "3.13"
steps:
- uses: actions/checkout@v4
- name: Set up Python
Expand Down Expand Up @@ -179,7 +182,8 @@ jobs:
- name: Install tox
run: python -m pip install tox>=4
- name: Run tests
run: python -m tox run -e `echo py${PYTHON_VERSION/\./}-{extras,unit,integ,pandas,sso}-ci | sed 's/ /,/g'`
# run: python -m tox run -e `echo py${PYTHON_VERSION/\./}-{extras,unit,integ,pandas,sso}-ci | sed 's/ /,/g'`
run: python -m tox run -e `echo py${PYTHON_VERSION/\./}-{extras,integ,pandas,sso}-ci | sed 's/ /,/g'`
env:
PYTHON_VERSION: ${{ matrix.python-version }}
cloud_provider: ${{ matrix.cloud-provider }}
Expand Down Expand Up @@ -311,6 +315,201 @@ jobs:
.coverage
coverage.xml


# test:
# name: Test ${{ matrix.os.download_name }}-${{ matrix.python-version }}-${{ matrix.cloud-provider }}
# needs: build
# runs-on: ${{ matrix.os.image_name }}
# strategy:
# fail-fast: false
# matrix:
# os:
# - image_name: ubuntu-latest
# download_name: manylinux_x86_64
# - image_name: macos-latest
# download_name: macosx_x86_64
# - image_name: windows-latest
# download_name: win_amd64
# python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
# cloud-provider: [aws, azure, gcp]
# # TODO: When there are prebuilt wheels accessible for our dependencies (i.e. numpy)
# # for Python 3.13 windows runs can be re-enabled. Currently, according to numpy:
# # "Numpy built with MINGW-W64 on Windows 64 bits is experimental, and only available for
# # testing. You are advised not to use it for production."
# exclude:
# - os:
# image_name: windows-latest
# download_name: win_amd64
# python-version: "3.13"
# steps:
# - uses: actions/checkout@v4
# - name: Set up Python
# uses: actions/setup-python@v4
# with:
# python-version: ${{ matrix.python-version }}
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: Set up Java
# uses: actions/setup-java@v4 # for wiremock
# with:
# java-version: 11
# distribution: 'temurin'
# java-package: 'jre'
# - name: Fetch Wiremock
# shell: bash
# run: curl https://repo1.maven.org/maven2/org/wiremock/wiremock-standalone/3.11.0/wiremock-standalone-3.11.0.jar --output .wiremock/wiremock-standalone.jar
# - name: Setup parameters file
# shell: bash
# env:
# PARAMETERS_SECRET: ${{ secrets.PARAMETERS_SECRET }}
# run: |
# gpg --quiet --batch --yes --decrypt --passphrase="$PARAMETERS_SECRET" \
# .github/workflows/parameters/public/parameters_${{ matrix.cloud-provider }}.py.gpg > test/parameters.py
# - name: Download wheel(s)
# uses: actions/download-artifact@v4
# with:
# name: ${{ matrix.os.download_name }}_py${{ matrix.python-version }}
# path: dist
# - name: Show wheels downloaded
# run: ls -lh dist
# shell: bash
# - name: Upgrade setuptools, pip and wheel
# run: python -m pip install -U setuptools pip wheel
# - name: Install tox
# run: python -m pip install tox>=4
# - name: Run tests
# run: python -m tox run -e `echo py${PYTHON_VERSION/\./}-{extras,unit,integ,pandas,sso}-ci | sed 's/ /,/g'`
# env:
# PYTHON_VERSION: ${{ matrix.python-version }}
# cloud_provider: ${{ matrix.cloud-provider }}
# PYTEST_ADDOPTS: --color=yes --tb=short
# TOX_PARALLEL_NO_SPINNER: 1
# shell: bash
# - name: Combine coverages
# run: python -m tox run -e coverage --skip-missing-interpreters false
# shell: bash
# - uses: actions/upload-artifact@v4
# with:
# include-hidden-files: true
# name: coverage_${{ matrix.os.download_name }}-${{ matrix.python-version }}-${{ matrix.cloud-provider }}
# path: |
# .tox/.coverage
# .tox/coverage.xml
#
# test-olddriver:
# name: Old Driver Test ${{ matrix.os.download_name }}-${{ matrix.python-version }}-${{ matrix.cloud-provider }}
# needs: lint
# runs-on: ${{ matrix.os.image_name }}
# strategy:
# fail-fast: false
# matrix:
# os:
# # Because old the version 3.0.2 of snowflake-connector-python depends on oscrypto which causes conflicts with higher versions of libssl
# # TODO: It can be changed to ubuntu-latest, when python sf connector version in tox is above 3.4.0
# - image_name: ubuntu-22.04
# download_name: linux
# python-version: [3.9]
# cloud-provider: [aws]
# steps:
# - uses: actions/checkout@v4
# - name: Set up Python
# uses: actions/setup-python@v4
# with:
# python-version: ${{ matrix.python-version }}
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: Setup parameters file
# shell: bash
# env:
# PARAMETERS_SECRET: ${{ secrets.PARAMETERS_SECRET }}
# run: |
# gpg --quiet --batch --yes --decrypt --passphrase="$PARAMETERS_SECRET" \
# .github/workflows/parameters/public/parameters_${{ matrix.cloud-provider }}.py.gpg > test/parameters.py
# - name: Upgrade setuptools, pip and wheel
# run: python -m pip install -U setuptools pip wheel
# - name: Install tox
# run: python -m pip install tox>=4
# - name: Run tests
# run: python -m tox run -e olddriver
# env:
# PYTHON_VERSION: ${{ matrix.python-version }}
# cloud_provider: ${{ matrix.cloud-provider }}
# PYTEST_ADDOPTS: --color=yes --tb=short
# shell: bash
#
# test-noarrowextension:
# name: No Arrow Extension Test ${{ matrix.os.download_name }}-${{ matrix.python-version }}-${{ matrix.cloud-provider }}
# needs: lint
# runs-on: ${{ matrix.os.image_name }}
# strategy:
# fail-fast: false
# matrix:
# os:
# - image_name: ubuntu-latest
# download_name: linux
# python-version: [3.9]
# cloud-provider: [aws]
# steps:
# - uses: actions/checkout@v4
# - name: Set up Python
# uses: actions/setup-python@v4
# with:
# python-version: ${{ matrix.python-version }}
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: Upgrade setuptools, pip and wheel
# run: python -m pip install -U setuptools pip wheel
# - name: Install tox
# run: python -m pip install tox>=4
# - name: Run tests
# run: python -m tox run -e noarrowextension
# env:
# PYTHON_VERSION: ${{ matrix.python-version }}
# cloud_provider: ${{ matrix.cloud-provider }}
# PYTEST_ADDOPTS: --color=yes --tb=short
# shell: bash
#
# test-fips:
# name: Test FIPS linux-3.9-${{ matrix.cloud-provider }}
# needs: build
# runs-on: ubuntu-latest
# strategy:
# fail-fast: false
# matrix:
# cloud-provider: [aws]
# steps:
# - uses: actions/checkout@v4
# - name: Setup parameters file
# shell: bash
# env:
# PARAMETERS_SECRET: ${{ secrets.PARAMETERS_SECRET }}
# run: |
# gpg --quiet --batch --yes --decrypt --passphrase="$PARAMETERS_SECRET" \
# .github/workflows/parameters/public/parameters_${{ matrix.cloud-provider }}.py.gpg > test/parameters.py
# - name: Download wheel(s)
# uses: actions/download-artifact@v4
# with:
# name: manylinux_x86_64_py3.9
# path: dist
# - name: Show wheels downloaded
# run: ls -lh dist
# shell: bash
# - name: Run tests
# run: ./ci/test_fips_docker.sh
# env:
# PYTHON_VERSION: 3.9
# cloud_provider: ${{ matrix.cloud-provider }}
# PYTEST_ADDOPTS: --color=yes --tb=short
# TOX_PARALLEL_NO_SPINNER: 1
# shell: bash
# - uses: actions/upload-artifact@v4
# with:
# include-hidden-files: true
# name: coverage_linux-fips-3.9-${{ matrix.cloud-provider }}
# path: |
# .coverage
# coverage.xml

test-lambda:
name: Test Lambda linux-${{ matrix.python-version }}-${{ matrix.cloud-provider }}
needs: build
Expand Down Expand Up @@ -358,6 +557,110 @@ jobs:
.coverage.py${{ env.shortver }}-lambda-ci
junit.py${{ env.shortver }}-lambda-ci-dev.xml

# combine-coverage:
# if: ${{ success() || failure() }}
# name: Combine coverage
# needs: [lint, test, test-fips, test-lambda]
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# - uses: actions/download-artifact@v4
# with:
# path: artifacts
# - name: Set up Python
# uses: actions/setup-python@v4
# with:
# python-version: '3.9'
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: Upgrade setuptools and pip
# run: python -m pip install -U setuptools pip wheel
# - name: Install tox
# run: python -m pip install tox>=4
# - name: Collect all coverages to one dir
# run: |
# python -c '
# from pathlib import Path
# import shutil
#
# src_dir = Path("artifacts")
# dst_dir = Path(".") / ".tox"
# dst_dir.mkdir()
# for src_file in src_dir.glob("*/.coverage"):
# dst_file = dst_dir / ".coverage.{}".format(src_file.parent.name[9:])
# print("{} copy to {}".format(src_file, dst_file))
# shutil.copy(str(src_file), str(dst_file))'
# - name: Combine coverages
# run: python -m tox run -e coverage
# - name: Publish html coverage
# uses: actions/upload-artifact@v4
# with:
# include-hidden-files: true
# name: overall_cov_html
# path: .tox/htmlcov
# - name: Publish xml coverage
# uses: actions/upload-artifact@v4
# with:
# include-hidden-files: true
# name: overall_cov_xml
# path: .tox/coverage.xml
# - uses: codecov/codecov-action@v4
# with:
# files: .tox/coverage.xml
# token: ${{ secrets.CODECOV_TOKEN }}

# combine-coverage:
# if: ${{ success() || failure() }}
# name: Combine coverage
# needs: [lint, test, test-fips, test-lambda]
# runs-on: ubuntu-latest
# steps:
# - uses: actions/checkout@v4
# - uses: actions/download-artifact@v4
# with:
# path: artifacts
# - name: Set up Python
# uses: actions/setup-python@v4
# with:
# python-version: '3.9'
# - name: Display Python version
# run: python -c "import sys; print(sys.version)"
# - name: Upgrade setuptools and pip
# run: python -m pip install -U setuptools pip wheel
# - name: Install tox
# run: python -m pip install tox>=4
# - name: Collect all coverages to one dir
# run: |
# python -c '
# from pathlib import Path
# import shutil
#
# src_dir = Path("artifacts")
# dst_dir = Path(".") / ".tox"
# dst_dir.mkdir()
# for src_file in src_dir.glob("*/.coverage"):
# dst_file = dst_dir / ".coverage.{}".format(src_file.parent.name[9:])
# print("{} copy to {}".format(src_file, dst_file))
# shutil.copy(str(src_file), str(dst_file))'
# - name: Combine coverages
# run: python -m tox run -e coverage
# - name: Publish html coverage
# uses: actions/upload-artifact@v4
# with:
# include-hidden-files: true
# name: overall_cov_html
# path: .tox/htmlcov
# - name: Publish xml coverage
# uses: actions/upload-artifact@v4
# with:
# include-hidden-files: true
# name: overall_cov_xml
# path: .tox/coverage.xml
# - uses: codecov/codecov-action@v4
# with:
# files: .tox/coverage.xml
# token: ${{ secrets.CODECOV_TOKEN }}

combine-coverage:
if: ${{ success() || failure() }}
name: Combine coverage
Expand Down
Loading
Loading