How to skip copying artifacts?

[angelo-peronio]

I am trying to use hatch-build-scripts to compile Kaitai Struct format definitions into Python modules while building a wheel. In my case, the format definitions within the sdist are in fxx/jad/formats, and I would like to have the resulting Python modules in the same folder.

I managed to have hatch-build-scripts compile the Python modules, but it subsequently fails to copy them to the out_dir, since they already are where they should be.

I tried the following in my pyproject.toml:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.sdist]
sources = ["src"]
only-packages = true

[tool.hatch.build.targets.wheel]
exclude = ["**/formats/*.ksy"]
dependencies = ["hatch-build-scripts", "ruff"]

[[tool.hatch.build.targets.wheel.hooks.build-scripts.scripts]]
# These paths refer to the sdist, which removes the src folder.
work_dir = "fxx/jad/formats"
out_dir = "fxx/jad/formats"
commands = [
    "kaitai-struct-compiler.bat --verbose=file --target=python --python-package=. *.ksy",
    "ruff format --verbose --force-exclude --exclude=__init__.py *.py",
]
artifacts = ["*.py"]
clean_artifacts = false

python -m build generates the Python modules in fxx/jad/formats as expected, but fails to copy them, since they already are where they should be:

* Creating venv isolated environment...
* Installing packages in isolated environment... (hatchling)
* Getting build dependencies for sdist...
* Building sdist...
* Building wheel from sdist
* Creating venv isolated environment...
* Installing packages in isolated environment... (hatchling)
* Getting build dependencies for wheel...
* Installing packages in isolated environment... (hatch-build-scripts, ruff)
* Building wheel...
parsing vsx_image_stack.ksy...
reading vsx_image_stack.ksy...
... compiling it for python... 
.... writing vsx_image_stack.py
[2024-03-15][16:25:59][ruff::resolve][DEBUG] Using configuration file (via parent) at: C:\Users\peronian\AppData\Local\Temp\build-via-sdist-f_8i90n5\fxx-0.1.0\pyproject.toml
[2024-03-15][16:25:59][ruff_workspace::resolver][DEBUG] Ignored path via `exclude`: "C:\\Users\\peronian\\AppData\\Local\\Temp\\build-via-sdist-f_8i90n5\\fxx-0.1.0\\fxx\\jad\\formats\\__init__.py"
[2024-03-15][16:25:59][ruff::commands::format][DEBUG] format_path; path=C:\Users\peronian\AppData\Local\Temp\build-via-sdist-f_8i90n5\fxx-0.1.0\fxx\jad\formats\vsx_image_stack.py
[2024-03-15][16:25:59][tracing::span][DEBUG] Printer::print;
[2024-03-15][16:25:59][ruff::commands::format][DEBUG] Formatted 1 files in 2.84ms
1 file reformatted
Traceback (most recent call last):
  File "C:\venvs\2023-fxx-phase-unwrapping\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 353, in <module>
    main()
  File "C:\venvs\2023-fxx-phase-unwrapping\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 335, in main
    json_out['return_val'] = hook(**hook_input['kwargs'])
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\venvs\2023-fxx-phase-unwrapping\Lib\site-packages\pyproject_hooks\_in_process\_in_process.py", line 251, in build_wheel
    return _build_backend().build_wheel(wheel_directory, config_settings,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\peronian\AppData\Local\Temp\build-env-dyqb8313\Lib\site-packages\hatchling\build.py", line 58, in build_wheel
    return os.path.basename(next(builder.build(directory=wheel_directory, versions=['standard'])))
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\peronian\AppData\Local\Temp\build-env-dyqb8313\Lib\site-packages\hatchling\builders\plugin\interface.py", line 147, in build
    build_hook.initialize(version, build_data)
  File "C:\Users\peronian\AppData\Local\Temp\build-env-dyqb8313\Lib\site-packages\hatch_build_scripts\plugin.py", line 60, in initialize
    shutil.copyfile(src_file, out_file)
  File "C:\Users\peronian\AppData\Local\Programs\Python\Python312\Lib\shutil.py", line 240, in copyfile
    raise SameFileError("{!r} and {!r} are the same file".format(src, dst))
shutil.SameFileError: WindowsPath('C:/Users/peronian/AppData/Local/Temp/build-via-sdist-f_8i90n5/fxx-0.1.0/fxx/jad/formats/vsx_image_stack.py') and WindowsPath('C:/Users/peronian/AppData/Local/Temp/build-via-sdist-f_8i90n5/fxx-0.1.0/fxx/jad/formats/vsx_image_stack.py') are the same file

ERROR Backend subprocess exited when trying to invoke build_wheel

Is it possible to use hatch-build-scripts to do what I am aiming to? If so, how?

If not, I can think of different ways to implement it within hatch-build-scripts. For instance:

• by catching and ignoring this SameFileError exception, or
• by skipping the artifacts copy step if out_dir is an empty string, or
• by adding a copy_artifacts Boolean configuration parameter.

[jaxvanyang]

I also ran into this issue. How about making artifacts optional? I only expect this plugin to run some commands, and adding files into package can be done with Hatch's shared-data option: https://hatch.pypa.io/latest/plugins/builder/wheel/#options

[angelo-peronio]

I found an indirect way to have hatch-build-scripts do what I want:

• specify the build artifacts in tool.hatch.build.targets.wheel.artifacts, and
• have artifacts = [] under [[tool.hatch.build.targets.wheel.hooks.build-scripts.scripts]].

Then the copying step of hatch-build-scripts is effectively skipped.

Relevant excerpts from my pyproject.toml:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.hatch.build.targets.sdist]
only-packages = true

[tool.hatch.build.targets.wheel]
dependencies = ["hatch-build-scripts", "ruff"]
exclude = ["**/formats/*.ksy"]
artifacts = ["**/formats/*.py"]

# Compile the Kaitai Struct definitions for the wheel.
[[tool.hatch.build.targets.wheel.hooks.build-scripts.scripts]]
work_dir = "src/fxx/jad/formats"
commands = [
    "kaitai-struct-compiler.bat --verbose=file --target=python --python-package=. *.ksy",
    "ruff format --verbose --force-exclude --exclude=__init__.py *.py",
]
# Hack: we do not specify any artifact here to skip the copying step
# from work_dir to out_dir.
# Instead, we specified above tool.hatch.build.targets.wheel.artifacts
artifacts = []
clean_artifacts = false

[jaxvanyang]

Yes, this is a workaround.