Switch to uv build frontend and add top-level uv workspace#1615
Switch to uv build frontend and add top-level uv workspace#1615kkraus14 wants to merge 31 commits intoNVIDIA:mainfrom
Conversation
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
0e6820a to
3505cf1
Compare
|
/ok to test |
3e1070a to
36cf441
Compare
|
/ok to test |
36cf441 to
9c9757f
Compare
|
/ok to test |
1 similar comment
|
/ok to test |
b5aaea0 to
a736c3d
Compare
|
/ok to test |
1 similar comment
|
/ok to test |
|
|
/ok to test |
| git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "cuda-core-v*[0-9]*"] | ||
|
|
||
| [tool.uv] | ||
| conflicts = [[{ extra = "cu12" }, { extra = "cu13" }]] |
|
/ok to test |
| # Version from git tags; setup.py still overrides install_requires at wheel build | ||
| # for exact version pinning (~= for release, == for dev). | ||
| [tool.setuptools_scm] | ||
| root = ".." | ||
| version_file = "cuda_python/_version.py" | ||
| tag_regex = "^(?P<version>v\\d+\\.\\d+\\.\\d+)" | ||
| git_describe_command = ["git", "describe", "--dirty", "--tags", "--long", "--match", "v*[0-9]*"] |
| # Workspace-level conflicts: cuda-toolkit can only be one major version at a | ||
| # time, so every cu12 extra/group is incompatible with every cu13 one. | ||
| conflicts = [ |
There was a problem hiding this comment.
Q: It seems we have to list all possible combinations of conflicts? Is this the only way at the workspace level? Maybe uv could honor the conflicts in each individual pyproject.toml? We currently only add a conflict for cuda-core.
There was a problem hiding this comment.
This is on my list to get cursor to see if it can simplify 😄. I agree this is quite annoying.
There was a problem hiding this comment.
@leofang see the updated comment. Unfortunately uv doesn't automatically propagate the information upwards so we need to keep this here in addition to defining it at the package level.
I'll open an issue upstream and link it here.
|
/ok to test |
|
There are build frontends? |
|
/ok to test |
6 similar comments
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
|
/ok to test |
uv sync manages the full venv state and removes packages not in the lockfile, including pre-built wheels installed via uv pip install. Swap the order: run uv sync first (installs test deps), then uv pip install the pre-built wheels. Co-authored-by: Cursor <cursoragent@cursor.com>
Each uv sync call resets the venv state, removing previously installed wheels. Consolidate all sync calls first, then install all pre-built wheels after the last sync. Co-authored-by: Cursor <cursoragent@cursor.com>
The merge_cuda_core_wheels.py script runs with the system Python (3.12 from setup-python1), not the venv. Use --system so the wheel package is available to the system Python. Co-authored-by: Cursor <cursoragent@cursor.com>
uv sync manages the full venv state and removes packages not in the lockfile, including pre-built wheels from uv pip install. Swap the order everywhere: run uv sync first to install dependency groups, then uv pip install the pre-built wheels after. Affected: ci/tools/run-tests, test-wheel-linux.yml, test-wheel-windows.yml Co-authored-by: Cursor <cursoragent@cursor.com>
--inexact tells uv sync not to remove extraneous packages, so pre-built wheels installed via uv pip install are preserved. This restores the natural ordering (install wheels first, then sync dependency groups) and simplifies the logic. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
bf46dd7 to
10a23a6
Compare
The --randomly-dont-reorganize flag was added to run-tests on main but cuda-bindings test group was missing pytest-randomly. Regenerate uv.lock. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
Add .venv/bin and .venv/Scripts to GITHUB_PATH in the Create venv step so the venv is activated for all subsequent steps. Removes the per-step PATH export hack from the Cython test build steps. Co-authored-by: Cursor <cursoragent@cursor.com>
uv pip install --group reads dependency groups directly from pyproject.toml. Combined with positional package args, a single call installs both the wheel and its test deps: uv pip install ./artifact.whl --group pyproject.toml:test This eliminates uv sync and its --frozen, --inexact, --package, --only-group, --no-install-project flags entirely from CI. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
.github/workflows/build-wheel.yml
Outdated
| $CHOWN -R $(whoami) ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }} | ||
| ls -lahR ${{ env.CUDA_BINDINGS_ARTIFACTS_DIR }} | ||
|
|
||
|
|
Adding the venv to GITHUB_PATH makes it apply to ALL subsequent steps, including the backport cibuildwheel build. The longer PATH pushes Windows over the cmd.exe environment limit. Instead, activate the venv inline within each Cython test build step using source. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
1 similar comment
|
/ok to test |
- Use bare --group test syntax (reads [dependency-groups] from pyproject.toml without triggering the build backend). - Combine wheel + group installs into single uv pip install calls. - Add DISTUTILS_USE_SDK=1 to CIBW_ENVIRONMENT_WINDOWS to skip vcvarsall.bat invocation (avoids cmd.exe PATH length limit). - Remove TEMP shortening hack (no longer needed with DISTUTILS_USE_SDK). - Activate venv inline for Cython test build steps. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
d1ec616 to
530f00d
Compare
|
/ok to test |
- Add missing trailing newline to .gitattributes (fixes end-of-file-fixer). - Remove extra blank line after artifact listing step in build-wheel.yml. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
- Replace cuda-core[test] with {include-group = "test"} in cuda-core's
test-cu* dependency groups. The self-reference triggered setuptools_scm
via the build backend in CI containers without full git context.
- Pass VCToolsInstallDir through CIBW_ENVIRONMENT_PASS_WINDOWS and
prepend its bin dir to PATH so MSVC's link.exe is found instead of
Git's /usr/bin/link.exe when DISTUTILS_USE_SDK=1 skips vcvarsall.bat.
- Regenerate uv.lock.
Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
The $VCToolsInstallDir PATH prepend via CIBW_ENVIRONMENT_WINDOWS did not take effect. Instead, rename Git's /usr/bin/link.exe (POSIX hardlink tool) before each cibuildwheel build so MSVC's linker is found on PATH when DISTUTILS_USE_SDK=1 skips vcvarsall.bat. Co-authored-by: Cursor <cursoragent@cursor.com>
|
/ok to test |
Summary
pyproject.tomlas a virtual uv workspace coveringcuda_bindings,cuda_core,cuda_pathfinder,cuda_python, andcuda_python_test_helpers.build[uv]build frontend forcuda_bindingsandcuda_coreon all platforms (Linux + Windows).uv.lockfor the workspace with workspace-level cu12/cu13 conflict declarations.pipwithuvacross all CI workflows:uv build --wheelfor pathfinder/cuda-python,uv tool run twinefor wheel checks,uv venv+uv pip install+uv sync --frozen --inexactfor Cython test deps.setup-uvwithactivate-environment,uv pip installfor wheels,uv sync --frozen --inexactfor dependency groups.uv pip install --systeminto conda env.setup-uvwithactivate-environment,uv pip install+uv sync --frozen --inexactfor test deps.uv pip installfor wheels,uv sync --frozen --inexactfor dependency groups.astral-sh/setup-uv@v7.3.0(pinned to SHA) withactivate-environment: truein test/coverage workflows.build[uv]on py3.14t.uv run --no-project --with wheelfor the cuda.core wheel merge step.pytest-randomlytocuda_bindingstest dependency group.uv.lockfrom SPDX and large-file pre-commit checks.uv.lockto.gitattributes(binary merge, linguist-generated).Key patterns
uv sync --frozen --inexact --package <pkg> --only-group <group> --no-install-projectinstalls test deps frompyproject.tomlwithout removing pre-installed wheels.uv pip install path/to/*.whlinstalls CI-built wheel artifacts into the venv.uv tool run twine check --strict *.whlanduv run --no-project --with wheel python script.pyavoid polluting the environment.Test plan
uv lockreproduces the same lockfile from a clean checkout withCUDA_HOMEset