Build Windows wheels locally and document release builds

This commit is contained in:
Jan Petykiewicz 2026-11-17 15:07:51 -08:00
commit 58adf0cc99
4 changed files with 110 additions and 35 deletions

2
.gitignore vendored
View file

@ -7,4 +7,6 @@
__pycache__/ __pycache__/
/build/ /build/
/dist/ /dist/
/.cache/
/.venv/
/*.egg-info/ /*.egg-info/

View file

@ -0,0 +1 @@
quay.io/pypa/manylinux_2_28_x86_64@sha256:e0adf18521ad35192df733a6c62d35df7801957d24e0470238aacecfab2188a3

View file

@ -1,48 +1,108 @@
# Building releases # Building releases
This guide is for maintainers producing source archives, Python wheels, and Build packages locally for distribution. For installing and using Pheon, start
standalone CLI artifacts. See [development](development.md) for local checks. with the [CLI](cli.md) or [Python](python.md) guide.
## Build from the source archive ## Start a release
Use Python 3.11+ and Rust 1.88+. From the repository root, create a build
environment and put the source archive in an empty `dist/release/` directory.
```sh ```sh
python -m pip install build python3 -m venv .venv
python -m build . .venv/bin/activate
python -m pip install build twine
python -m build --sdist --outdir dist/release
``` ```
The default invocation builds the wheel from the source archive. Both use the Use `python -m build --wheel` to build a wheel for your own machine. For a
Cargo version, license, and locked dependencies. Source builds require Rust 1.88 portable Linux wheel, use the container below. See [development](development.md)
or newer; the checkout selects stable. Use `RUSTUP_TOOLCHAIN=1.88.0` to check the for compiler checks.
minimum compiler explicitly.
Python wheels contain the package and shared library. Cargo installation Python wheels bundle the native library. Install the CLI separately with Cargo.
provides the CLI. Neither installation discovers or depends on the other.
## Build Linux x86-64 wheels
On x86-64 Linux, use Podman and a Rust toolchain installed through `rustup`.
Download the [pinned manylinux image](../contrib/wheels/manylinux-image.txt)
once and save it for future releases.
```sh
pheon_wheel_cache="$PWD/.cache/wheels"
pheon_image=$(cat contrib/wheels/manylinux-image.txt)
pheon_cached_image=localhost/pheon-manylinux:2.28-2026.10.26-1
mkdir -p "$pheon_wheel_cache"
podman pull --platform linux/amd64 "$pheon_image"
podman tag "$pheon_image" "$pheon_cached_image"
podman save --format oci-archive \
--output "$pheon_wheel_cache/manylinux_2_28_x86_64.oci.tar" \
"$pheon_cached_image"
```
If the image is missing from Podman, restore it with
`podman load --input .cache/wheels/manylinux_2_28_x86_64.oci.tar`.
Build from the source archive using the local Rust toolchain and Cargo cache.
```sh
python -m pip install 'cibuildwheel==4.3.5'
cargo fetch --locked --target x86_64-unknown-linux-gnu
rust_toolchain=$(rustc --print sysroot)
cargo_registry="${CARGO_HOME:-$HOME/.cargo}/registry"
export CIBW_CONTAINER_ENGINE="podman; disable_host_mount: true; create_args: \
--volume '$PWD/.cache/wheels:/pheon-cache' \
--volume '$rust_toolchain:/rust-toolchain:ro' \
--volume '$cargo_registry:/pheon-cache/cargo/registry:ro'"
export CIBW_MANYLINUX_X86_64_IMAGE=localhost/pheon-manylinux:2.28-2026.10.26-1
export CIBW_ENVIRONMENT_LINUX='PATH=/rust-toolchain/bin:$PATH CARGO_HOME=/pheon-cache/cargo CARGO_NET_OFFLINE=true PIP_CACHE_DIR=/pheon-cache/pip'
export CIBW_BUILD='cp31{1,2,3,4}-manylinux_x86_64'
export CIBW_TEST_EXTRAS=arrow
export CIBW_TEST_COMMAND='python -I {project}/tools/check_installed.py'
unset PHEON_LIBRARY
python -m cibuildwheel --platform linux --archs x86_64 \
--output-dir dist/release dist/release/pheon-*.tar.gz
```
cibuildwheel repairs the wheel and tests the same file on Python 3.11–3.14.
The container downloads any Python dependencies that it needs.
## Build Windows x86-64 packages from Linux
Install MinGW-w64 with `x86_64-w64-mingw32-gcc` on `PATH`. From the project
environment, build the Windows wheel and the standalone CLI:
```sh
rustup target add x86_64-pc-windows-gnu
PHEON_BUILD_TARGET=x86_64-pc-windows-gnu python -m build --wheel --outdir dist/release
cargo build --locked --release --target x86_64-pc-windows-gnu --bin pheon
```
The `py3-none-win_amd64` wheel lands in `dist/release/`. The executable is
`target/x86_64-pc-windows-gnu/release/pheon.exe`. Test the wheel and CLI on Windows
before publishing.
## Validate artifacts ## Validate artifacts
The [package workflow](https://mpxd.net/code/jan/pheon/src/branch/master/.github/workflows/packages.yml) builds one source archive Check the Python package metadata and verify that the packaged crate builds.
and uses cibuildwheel to build and repair wheels for Linux and macOS x86-64/ARM64,
and Windows x86-64. Wheels use `py3-none` interpreter/ABI tags plus a platform tag.
Linux releases need an appropriate manylinux tag; unrepaired local `linux_*`
wheels are development artifacts.
The workflow tests each repaired wheel on Python 3.11–3.14 without rebuilding it.
Tests run in isolated mode against the installed package and bundled library.
To repeat the check after installing a wheel and PyArrow:
```sh ```sh
python -I /path/to/pheon/tools/check_installed.py python -m twine check --strict dist/release/*.tar.gz dist/release/*.whl
cargo publish --dry-run --locked
``` ```
Unset `PHEON_LIBRARY` for artifact checks. Native checks cover minimum and stable Wheel filenames use `py3-none` plus a platform tag, such as
Rust in debug and release modes. Standalone CLI artifacts contain the executable, `manylinux_2_28_x86_64` or `win_amd64`. Use repaired `manylinux_*` wheels for
license, README, documentation, and Git templates; each targets one OS and Linux releases.
architecture.
## Publish separately To test a wheel manually, install it with its `arrow` extra in a test
environment on the target OS. Clear `PHEON_LIBRARY` so the checks load the
bundled library. Replace the example path with the wheel you built.
Workflows upload artifacts; they do not publish to PyPI, crates.io, or a hosted ```sh
release. Review the artifacts and test results before publishing them. python -m pip install "path/to/pheon-wheel.whl[arrow]"
Coordinate companion package dependencies when changing Python interfaces. python -I tools/check_installed.py
C API, Arrow layout, JSON schema, and comparison-profile versions have separate ```
contracts; see the [native API](native-api.md) and [report reference](diff-reports.md).
Build standalone CLI artifacts for each OS and architecture. Include the
executable, license, README, documentation, and Git templates.
Publish the validated Python packages to PyPI and the Rust crate to crates.io.

View file

@ -1,5 +1,6 @@
"""Bundle Cargo's C ABI library in a platform wheel, without a Python ABI tie.""" """Bundle Cargo's C ABI library in a platform wheel, without a Python ABI tie."""
import json import json
import os
from pathlib import Path from pathlib import Path
import shutil import shutil
import subprocess import subprocess
@ -13,6 +14,11 @@ from wheel.bdist_wheel import bdist_wheel
ROOT = Path(__file__).resolve().parent ROOT = Path(__file__).resolve().parent
VERSION = tomllib.loads((ROOT / "Cargo.toml").read_text())["package"]["version"] VERSION = tomllib.loads((ROOT / "Cargo.toml").read_text())["package"]["version"]
BUILD_TARGET = os.environ.get("PHEON_BUILD_TARGET")
WINDOWS_TARGET = "x86_64-pc-windows-gnu"
if BUILD_TARGET is not None and BUILD_TARGET != WINDOWS_TARGET:
raise ValueError(f"PHEON_BUILD_TARGET must be {WINDOWS_TARGET}, got {BUILD_TARGET!r}")
if not (ROOT / "Cargo.lock").is_file(): if not (ROOT / "Cargo.lock").is_file():
@ -22,8 +28,11 @@ if not (ROOT / "Cargo.lock").is_file():
class CargoBuild(build_py): class CargoBuild(build_py):
def run(self): def run(self):
super().run() super().run()
command = ["cargo", "build", "--release", "--lib", "--locked", "--message-format=json"]
if BUILD_TARGET:
command.extend(["--target", BUILD_TARGET])
result = subprocess.run( result = subprocess.run(
["cargo", "build", "--release", "--lib", "--locked", "--message-format=json"], command,
cwd=ROOT, capture_output=True, text=True, cwd=ROOT, capture_output=True, text=True,
) )
sys.stderr.write(result.stderr) sys.stderr.write(result.stderr)
@ -43,8 +52,8 @@ class CargoBuild(build_py):
if Path(filename).suffix in (".so", ".dylib", ".dll"): if Path(filename).suffix in (".so", ".dylib", ".dll"):
library = Path(filename) library = Path(filename)
result.check_returncode() result.check_returncode()
if library is None: if library is None or (BUILD_TARGET and library.name != "pheon.dll"):
raise RuntimeError("Cargo did not produce the pheon shared library") raise RuntimeError("Cargo did not produce the expected Pheon shared library")
shutil.copy2(library, Path(self.build_lib) / "pheon" / library.name) shutil.copy2(library, Path(self.build_lib) / "pheon" / library.name)
@ -55,7 +64,9 @@ class NativeDistribution(Distribution):
class NativeWheel(bdist_wheel): class NativeWheel(bdist_wheel):
def get_tag(self): def get_tag(self):
# CFFI ABI mode needs the host platform, but no CPython extension ABI. # CFFI ABI mode needs a platform tag and no CPython extension ABI.
if BUILD_TARGET == WINDOWS_TARGET:
return "py3", "none", "win_amd64"
_, _, platform = super().get_tag() _, _, platform = super().get_tag()
return "py3", "none", platform return "py3", "none", platform
@ -64,4 +75,5 @@ setup(
version=VERSION, version=VERSION,
distclass=NativeDistribution, distclass=NativeDistribution,
cmdclass={"build_py": CargoBuild, "bdist_wheel": NativeWheel}, cmdclass={"build_py": CargoBuild, "bdist_wheel": NativeWheel},
options={"build": {"build_base": f"build/{BUILD_TARGET}"}} if BUILD_TARGET else {},
) )