Repository navigation
Conversation
…3.15 wheels zstandard publishes no cp315 wheel; Python 3.14 ships compression.zstd (PEP 784). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
CPython 3.15 is out (3.15.2), but PyIceberg cannot be installed on it from wheels:
zstandardis a core dependency, and it publishes no cp315 wheel: 0.25.0 (2025-09-14) is still its latest release. Its 3.15 PRs, Add Python 3.15 support indygreg/python-zstandard#357 and Build: Bump adlfs from 2024.1.0 to 2024.2.0 #372, are open. Without a C compiler,pip install pyicebergfails on 3.15.pyicebergitself ships no cp315 wheel either, becauseCIBW_PROJECT_REQUIRES_PYTHONstops at<3.15.Python 3.14 added a zstd module to the standard library:
compression.zstd, from PEP 784. PyIceberg only needs one-shot compress and decompress (the Avro codec and Puffin blobs), so on 3.14+ the third-party package is not needed at all.What changes
pyiceberg/table/puffin.pyusescompression.zstd.decompresson Python 3.14+. Below 3.14 it keepszstandard.ZstdDecompressor().decompress.pyiceberg/avro/codecs/zstandard_codec.pyusescompression.zstd.compressanddecompresson 3.14+. Below 3.14 the existingzstandardcode is unchanged, only indented under the version check (git diff -wshows the real change).pyproject.toml, the dependency becomeszstandard>=0.13.0; python_version < '3.14'.uv.lockis regenerated withuv lock.CIBW_PROJECT_REQUIRES_PYTHONbecomes>=3.11,<3.16, and 3.15 joins thesetup-pythonlist. cibuildwheel v4.3.4 already builds cp315 by default.CIBW_TEST_SKIP: "cp315-*"is set for now. The test step syncs the lockeddevgroup, which pins pyarrow 25.0.1 and numpy 2.4.6 throughpyspark[connect], and neither has a cp315 wheel. Moving those pins is a separate change, and the skip can go once it lands.The version check follows the existing
sys.version_info >= (3, 14)pattern (uuid7incatalog/rest/__init__.py), so mypy checks each branch for the interpreter it runs on.PEP 784 makes
compression.zstdan optional module, built when libzstd is available; python-build-standalone (what uv installs) ships it on 3.14 and 3.15. A CPython built without it fails at import time, as a missingzstandarddoes today.Are these changes tested?
The existing tests cover both paths: Puffin's
test_read_compressed_zstd, and thezstd/zstandardmanifest tests intests/utils/test_manifest.py. Locally, on macOS arm64:tests/avro tests/table/test_puffin.py tests/utils/test_manifest.py tests/table/test_metadata.pyruff (v0.14.3) and mypy (1.18.2) are clean on the changed files under 3.13 and 3.15.
Are there any user-facing changes?
On Python 3.14+,
pip install pyicebergno longer installszstandard. On 3.15, pyiceberg installs from a wheel.🤖 Generated with Claude Code