Skip to content

Switch to uv build frontend and add top-level uv workspace#1615

Draft
kkraus14 wants to merge 31 commits intoNVIDIA:mainfrom
kkraus14:uv-build-frontend-and-workspace
Draft

Switch to uv build frontend and add top-level uv workspace#1615
kkraus14 wants to merge 31 commits intoNVIDIA:mainfrom
kkraus14:uv-build-frontend-and-workspace

Conversation

@kkraus14
Copy link
Collaborator

@kkraus14 kkraus14 commented Feb 12, 2026

Summary

  • Adds a root pyproject.toml as a virtual uv workspace covering cuda_bindings, cuda_core, cuda_pathfinder, cuda_python, and cuda_python_test_helpers.
  • Switches cibuildwheel to build[uv] build frontend for cuda_bindings and cuda_core on all platforms (Linux + Windows).
  • Adds uv.lock for the workspace with workspace-level cu12/cu13 conflict declarations.
  • Replaces pip with uv across all CI workflows:
    • build-wheel.yml: uv build --wheel for pathfinder/cuda-python, uv tool run twine for wheel checks, uv venv + uv pip install + uv sync --frozen --inexact for Cython test deps.
    • test-wheel-linux.yml / test-wheel-windows.yml: setup-uv with activate-environment, uv pip install for wheels, uv sync --frozen --inexact for dependency groups.
    • build-docs.yml: uv pip install --system into conda env.
    • coverage.yml: setup-uv with activate-environment, uv pip install + uv sync --frozen --inexact for test deps.
    • ci/tools/run-tests: uv pip install for wheels, uv sync --frozen --inexact for dependency groups.
  • Uses astral-sh/setup-uv@v7.3.0 (pinned to SHA) with activate-environment: true in test/coverage workflows.
  • Shortens Windows TEMP path to avoid cmd.exe ~8KB environment limit with build[uv] on py3.14t.
  • Uses uv run --no-project --with wheel for the cuda.core wheel merge step.
  • Adds pytest-randomly to cuda_bindings test dependency group.
  • Excludes uv.lock from SPDX and large-file pre-commit checks.
  • Adds uv.lock to .gitattributes (binary merge, linguist-generated).

Key patterns

  • Dependency groups: uv sync --frozen --inexact --package <pkg> --only-group <group> --no-install-project installs test deps from pyproject.toml without removing pre-installed wheels.
  • Pre-built wheels: uv pip install path/to/*.whl installs CI-built wheel artifacts into the venv.
  • One-shot tools: uv tool run twine check --strict *.whl and uv run --no-project --with wheel python script.py avoid polluting the environment.

Test plan

  • All build jobs pass (linux-64, linux-aarch64, win-64, all Python versions including py3.14t)
  • All test jobs pass (linux + windows, local + wheels, all GPU types)
  • Docs build passes
  • Coverage workflow passes
  • uv lock reproduces the same lockfile from a clean checkout with CUDA_HOME set

@copy-pr-bot
Copy link
Contributor

copy-pr-bot bot commented Feb 12, 2026

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@kkraus14 kkraus14 force-pushed the uv-build-frontend-and-workspace branch from 0e6820a to 3505cf1 Compare February 12, 2026 18:58
@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14 kkraus14 force-pushed the uv-build-frontend-and-workspace branch from 3e1070a to 36cf441 Compare February 13, 2026 03:18
@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14 kkraus14 force-pushed the uv-build-frontend-and-workspace branch from 36cf441 to 9c9757f Compare February 13, 2026 03:22
@kkraus14
Copy link
Collaborator Author

/ok to test

1 similar comment
@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14 kkraus14 force-pushed the uv-build-frontend-and-workspace branch from b5aaea0 to a736c3d Compare February 13, 2026 03:35
@kkraus14
Copy link
Collaborator Author

/ok to test

1 similar comment
@kkraus14
Copy link
Collaborator Author

/ok to test

@github-actions
Copy link

@kkraus14
Copy link
Collaborator Author

/ok to test

git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "cuda-core-v*[0-9]*"]

[tool.uv]
conflicts = [[{ extra = "cu12" }, { extra = "cu13" }]]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, it's already here, so this PR will close #1247

@kkraus14
Copy link
Collaborator Author

/ok to test

Comment on lines +44 to +50
# Version from git tags; setup.py still overrides install_requires at wheel build
# for exact version pinning (~= for release, == for dev).
[tool.setuptools_scm]
root = ".."
version_file = "cuda_python/_version.py"
tag_regex = "^(?P<version>v\\d+\\.\\d+\\.\\d+)"
git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "v*[0-9]*"]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mdboom to review this change

Comment on lines 20 to 22
# Workspace-level conflicts: cuda-toolkit can only be one major version at a
# time, so every cu12 extra/group is incompatible with every cu13 one.
conflicts = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Q: It seems we have to list all possible combinations of conflicts? Is this the only way at the workspace level? Maybe uv could honor the conflicts in each individual pyproject.toml? We currently only add a conflict for cuda-core.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is on my list to get cursor to see if it can simplify 😄. I agree this is quite annoying.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leofang see the updated comment. Unfortunately uv doesn't automatically propagate the information upwards so we need to keep this here in addition to defining it at the package level.

I'll open an issue upstream and link it here.

@kkraus14
Copy link
Collaborator Author

/ok to test

@cpcloud
Copy link
Contributor

cpcloud commented Feb 13, 2026

There are build frontends?

@kkraus14
Copy link
Collaborator Author

/ok to test

6 similar comments
@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14
Copy link
Collaborator Author

/ok to test

kkraus14 and others added 6 commits February 17, 2026 23:57
uv sync manages the full venv state and removes packages not in the
lockfile, including pre-built wheels installed via uv pip install.
Swap the order: run uv sync first (installs test deps), then
uv pip install the pre-built wheels.

Co-authored-by: Cursor <cursoragent@cursor.com>
Each uv sync call resets the venv state, removing previously installed
wheels. Consolidate all sync calls first, then install all pre-built
wheels after the last sync.

Co-authored-by: Cursor <cursoragent@cursor.com>
The merge_cuda_core_wheels.py script runs with the system Python
(3.12 from setup-python1), not the venv. Use --system so the wheel
package is available to the system Python.

Co-authored-by: Cursor <cursoragent@cursor.com>
uv sync manages the full venv state and removes packages not in the
lockfile, including pre-built wheels from uv pip install. Swap the
order everywhere: run uv sync first to install dependency groups,
then uv pip install the pre-built wheels after.

Affected: ci/tools/run-tests, test-wheel-linux.yml, test-wheel-windows.yml
Co-authored-by: Cursor <cursoragent@cursor.com>
--inexact tells uv sync not to remove extraneous packages, so
pre-built wheels installed via uv pip install are preserved. This
restores the natural ordering (install wheels first, then sync
dependency groups) and simplifies the logic.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14 kkraus14 force-pushed the uv-build-frontend-and-workspace branch from bf46dd7 to 10a23a6 Compare February 18, 2026 04:59
The --randomly-dont-reorganize flag was added to run-tests on main
but cuda-bindings test group was missing pytest-randomly. Regenerate
uv.lock.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

kkraus14 and others added 2 commits February 18, 2026 01:13
Add .venv/bin and .venv/Scripts to GITHUB_PATH in the Create venv
step so the venv is activated for all subsequent steps. Removes the
per-step PATH export hack from the Cython test build steps.

Co-authored-by: Cursor <cursoragent@cursor.com>
uv pip install --group reads dependency groups directly from
pyproject.toml. Combined with positional package args, a single call
installs both the wheel and its test deps:

  uv pip install ./artifact.whl --group pyproject.toml:test

This eliminates uv sync and its --frozen, --inexact, --package,
--only-group, --no-install-project flags entirely from CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

$CHOWN -R $(whoami) ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}
ls -lahR ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }}


Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change

Adding the venv to GITHUB_PATH makes it apply to ALL subsequent
steps, including the backport cibuildwheel build. The longer PATH
pushes Windows over the cmd.exe environment limit. Instead, activate
the venv inline within each Cython test build step using source.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

1 similar comment
@kkraus14
Copy link
Collaborator Author

/ok to test

- Use bare --group test syntax (reads [dependency-groups] from
  pyproject.toml without triggering the build backend).
- Combine wheel + group installs into single uv pip install calls.
- Add DISTUTILS_USE_SDK=1 to CIBW_ENVIRONMENT_WINDOWS to skip
  vcvarsall.bat invocation (avoids cmd.exe PATH length limit).
- Remove TEMP shortening hack (no longer needed with DISTUTILS_USE_SDK).
- Activate venv inline for Cython test build steps.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

@kkraus14 kkraus14 force-pushed the uv-build-frontend-and-workspace branch from d1ec616 to 530f00d Compare February 18, 2026 16:19
@kkraus14
Copy link
Collaborator Author

/ok to test

- Add missing trailing newline to .gitattributes (fixes end-of-file-fixer).
- Remove extra blank line after artifact listing step in build-wheel.yml.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

- Replace cuda-core[test] with {include-group = "test"} in cuda-core's
  test-cu* dependency groups. The self-reference triggered setuptools_scm
  via the build backend in CI containers without full git context.
- Pass VCToolsInstallDir through CIBW_ENVIRONMENT_PASS_WINDOWS and
  prepend its bin dir to PATH so MSVC's link.exe is found instead of
  Git's /usr/bin/link.exe when DISTUTILS_USE_SDK=1 skips vcvarsall.bat.
- Regenerate uv.lock.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

The $VCToolsInstallDir PATH prepend via CIBW_ENVIRONMENT_WINDOWS did
not take effect. Instead, rename Git's /usr/bin/link.exe (POSIX
hardlink tool) before each cibuildwheel build so MSVC's linker is
found on PATH when DISTUTILS_USE_SDK=1 skips vcvarsall.bat.

Co-authored-by: Cursor <cursoragent@cursor.com>
@kkraus14
Copy link
Collaborator Author

/ok to test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments