List of issues with pre-commit

[Timmmm]

This is my collection of issues me and my colleagues have had with pre-commit's reliability (and the reliability of the linters). I'm collecting these so when people say "but it works for me; you must be doing something wrong" I can point them here.

[Timmmm]

I work on sail-riscv. We added a markdown link checker lint recently. For some reason it fails... even when I haven't edited any markdown files:

$ git push timmmm
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...............................................................Passed
check for added large files..............................................Passed
check that executables have shebangs.................(no files to check)Skipped
check that scripts with shebangs are executable..........................Passed
clang-format.........................................(no files to check)Skipped
No-tabs checker..........................................................Passed
prettier.................................................................Passed
codespell................................................................Passed
Markdown Link Check......................................................Failed
- hook id: markdown-link-check
- exit code: 1

/home/me/.cache/pre-commit/repolbur5l42/node_env-system/lib/node_modules/markdown-link-check/node_modules/undici/lib/web/webidl/index.js:509
webidl.is.ReadableStream = webidl.util.MakeTypeAssertion(ReadableStream)
                                                         ^

ReferenceError: ReadableStream is not defined
    at Object.<anonymous> (/home/me/.cache/pre-commit/repolbur5l42/node_env-system/lib/node_modules/markdown-link-check/node_modules/undici/lib/web/webidl/index.js:509:58)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)
    at Module.load (node:internal/modules/cjs/loader:1076:32)
    at Function.Module._load (node:internal/modules/cjs/loader:911:12)
    at Module.require (node:internal/modules/cjs/loader:1100:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/home/me/.cache/pre-commit/repolbur5l42/node_env-system/lib/node_modules/markdown-link-check/node_modules/undici/lib/web/fetch/util.js:12:20)
    at Module._compile (node:internal/modules/cjs/loader:1198:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1252:10)

error: failed to push some refs to 'github.com:Timmmm/sail-riscv.git'

When I accidentally pushed the exact same commit to my work's internal Gitlab it worked fine! Apparently then there are no files to check...

$ git push
trim trailing whitespace.................................................Passed
fix end of files.........................................................Passed
check yaml...........................................(no files to check)Skipped
check for added large files..............................................Passed
check that executables have shebangs.................(no files to check)Skipped
check that scripts with shebangs are executable..........................Passed
clang-format.........................................(no files to check)Skipped
No-tabs checker..........................................................Passed
prettier.................................................................Passed
codespell................................................................Passed
Markdown Link Check..................................(no files to check)Skipped

Same commit though. Apparently it matches markdown files and I definitely haven't edited any of those. No idea what is going on here.

I think the actual error is because it requires a newer version of Node than the one Rocky 9 comes with (16), but yeay what a nice error message, and why is it running that check anyway?

[Timmmm]

From a collegue:

Something weird going on - First I noticed my VDI session was terminated while having lunch, and now that it's back, it's using Rocky 8.10 instead of 8.8.

But now I can't get pre-commit to work. This is what I see:

$ pre-commit 
[WARNING] Unstaged files detected.
[INFO] Stashing unstaged files to /home/them/.cache/pre-commit/patch1751895089-16640.
[INFO] Installing environment for https://github.com/shellcheck-py/shellcheck-py.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Restored changes from /home/them/.cache/pre-commit/patch1751895089-16640.
An unexpected error has occurred: CalledProcessError: command: ('/home/them/.cache/pre-commit/repospt7hsau/py_env-python3.11/bin/python', '-mpip', 'install', '.')
return code: 1
stdout:
    Processing /home/them/.cache/pre-commit/repospt7hsau
      Installing build dependencies: started
      Installing build dependencies: finished with status 'done'
      Getting requirements to build wheel: started
      Getting requirements to build wheel: finished with status 'done'
      Preparing metadata (pyproject.toml): started
      Preparing metadata (pyproject.toml): finished with status 'done'
    Building wheels for collected packages: shellcheck_py
      Building wheel for shellcheck_py (pyproject.toml): started
      Building wheel for shellcheck_py (pyproject.toml): finished with status 'error'
    Failed to build shellcheck_py
stderr:
      error: subprocess-exited-with-error
      
      × Building wheel for shellcheck_py (pyproject.toml) did not run successfully.
      │ exit code: 1
      ╰─> [17 lines of output]
          /tmp/pip-build-env-vhedswfj/overlay/lib/python3.11/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
          !!
          
                  ********************************************************************************
                  Please consider removing the following classifiers in favor of a SPDX license expression:
          
                  License :: OSI Approved :: MIT License
          
                  See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
                  ********************************************************************************
          
          !!
            self._finalize_license_expression()
          running bdist_wheel
          running build
          running fetch_binaries
          error: [Errno 17] File exists: 'build/temp.linux-x86_64-cpython-311'
          [end of output]
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      ERROR: Failed building wheel for shellcheck_py
    ERROR: Failed to build installable wheels for some pyproject.toml based projects (shellcheck_py)
Check the log at /home/them/.cache/pre-commit/pre-commit.log

Someone linked to an earlier thread and...

Hmm, seems to be resolved by deleting ~/.cache

[Timmmm]

From a collegue:

An error has occurred: InvalidManifestError: 
==> File /home/them/.cache/pre-commit/repohvgcyhs9/.pre-commit-hooks.yaml
==> At Hook(id='clang-format')
==> At key: types_or
==> At index 9
=====> Type tag 'textproto' is not recognized.  Try upgrading identify and pre-commit?
Check the log at /home/them/.cache/pre-commit/pre-commit.log

"Just upgrade pre-commit" you might think. Well, it's not so easy, as the following conversation shows:

TW: Maybe try a pre-commit clean? Anything with .cache in the file-path makes me suspicious

O: Yeah I had this issue, I asked @tim Hutt and this re-install with UV worked for me.

"Hmm I think you probably need to upgrade pre-commit. I switch to installing it using uv which is waaaay more reliable:"

pre-commit uninstall
python3 -m pip uninstall pre-commit
uv tool install pre-commit
pre-commit install [--hook-type=pre-push]

"You'll have to pre-commit uninstall and then pre-commit install again on any other repos where you had it installed because it will point to the old version still."

AK: Try removing the cached files: ~/.cache/pre-commit

JW: I did. No change.

O: I think I tried clearing the cache files and re-installing with pip with no luck before doing the UV re-install. However I didn't really root cause it after I got it working so perhaps there is a better way.

DL: Imo using python without uv should be considered self harm

RA: Try a rm -r /home/them/.cache/pre-commit

JW: I'm fine with starting to use uv. How do I do that? @ra removing cache doesn't seem to help.

Using uv is definitely much better, but you do still run into weird issues like this all the time.

[Timmmm]

We often ran into this error with hadolint and eventually gave up and disabled it:

[INFO] Installing environment for https://github.com/AleksaC/hadolint-py.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
An unexpected error has occurred: CalledProcessError: command: ('/opt/svc-gitlabrunner/.cache/pre-commit/repoteqaym65/py_env-python3.11/bin/python', '-mpip', 'install', '.')
return code: 1
stdout:
    Processing /opt/svc-gitlabrunner/.cache/pre-commit/repoteqaym65
      Preparing metadata (setup.py): started
      Preparing metadata (setup.py): finished with status 'done'
    Building wheels for collected packages: hadolint_py
      Building wheel for hadolint_py (setup.py): started
      Building wheel for hadolint_py (setup.py): finished with status 'error'
      Running setup.py clean for hadolint_py
    Failed to build hadolint_py
stderr:
      error: subprocess-exited-with-error
      
      × python setup.py bdist_wheel did not run successfully.
      │ exit code: 1
      ╰─> [5 lines of output]
          running bdist_wheel
          running build
          running build_py
          running fetch_binaries
          error: [Errno 17] File exists: 'build/temp.linux-x86_64-cpython-311'
          [end of output]
      
      note: This error originates from a subprocess, and is likely not a problem with pip.
      ERROR: Failed building wheel for hadolint_py
    ERROR: Could not build wheels for hadolint_py, which is required to install pyproject.toml-based projects
Check the log at /opt/svc-gitlabrunner/.cache/pre-commit/pre-commit.log