Standardize our pre-commit hooks #71

karlhigley · 2022-04-13T16:20:02Z

This includes the super-set of commit hooks from NVTabular and Merlin Models, and applies them to the whole core library.

This includes the super-set of commit hooks from NVTabular and Merlin Models, and applies them to the whole `core` library.

nvidia-merlin-bot · 2022-04-13T16:21:12Z

Click to view CI Results

GitHub pull request #71 of commit f068b073e6acc47171c53193d1af316a08cba262, no merge conflicts.
Running as SYSTEM
Setting status of f068b073e6acc47171c53193d1af316a08cba262 to PENDING with url https://10.20.13.93:8080/job/merlin_core/30/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_core
using credential ce87ff3c-94f0-400a-8303-cb4acb4918b5
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/core # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/core
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems username and pass
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/core +refs/pull/71/*:refs/remotes/origin/pr/71/* # timeout=10
 > git rev-parse f068b073e6acc47171c53193d1af316a08cba262^{commit} # timeout=10
Checking out Revision f068b073e6acc47171c53193d1af316a08cba262 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f f068b073e6acc47171c53193d1af316a08cba262 # timeout=10
Commit message: "Standardize our pre-commit hooks"
 > git rev-list --no-walk c10b099f3a93a052769682f902ac4521fadccf85 # timeout=10
[merlin_core] $ /bin/bash /tmp/jenkins8902899518617978413.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_core/core, configfile: pyproject.toml
plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 338 items / 1 skipped
tests/unit/core/test_dispatch.py ..                                      [  0%]

tests/unit/dag/test_base_operator.py ....                                [  1%]

tests/unit/dag/test_column_selector.py ..........................        [  9%]

tests/unit/dag/test_tags.py ......                                       [ 11%]

tests/unit/dag/ops/test_selection.py ...                                 [ 12%]

tests/unit/io/test_io.py ............................................... [ 26%]

................................................................         [ 44%]

tests/unit/schema/test_column_schemas.py ............................... [ 54%]

........................................................................ [ 75%]

........................................................................ [ 96%]

.                                                                        [ 97%]

tests/unit/schema/test_schema_io.py ..                                   [ 97%]

tests/unit/utils/test_utils.py ........                                  [100%]
=============================== warnings summary ===============================

tests/unit/dag/test_base_operator.py: 4 warnings

tests/unit/io/test_io.py: 72 warnings

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/io/test_io.py::test_validate_and_regenerate_dataset

/var/jenkins_home/workspace/merlin_core/core/merlin/io/parquet.py:551: DeprecationWarning: 'ParquetDataset.pieces' attribute is deprecated as of pyarrow 5.0.0 and will be removed in a future version. Specify 'use_legacy_dataset=False' while constructing the ParquetDataset, and then use the '.fragments' attribute instead.

paths = [p.path for p in pa_dataset.pieces]
tests/unit/utils/test_utils.py::test_nvt_distributed[True-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 34679 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed[True-False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 35245 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed[False-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 43847 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed[False-False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 38593 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed_force[True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 33455 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed_force[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 42495 instead

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

================= 338 passed, 1 skipped, 83 warnings in 52.33s =================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/core/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_core] $ /bin/bash /tmp/jenkins6132097872713183884.sh

github-actions · 2022-04-13T16:22:59Z

Documentation preview

https://nvidia-merlin.github.io/core/review/pr-71

nvidia-merlin-bot · 2022-04-13T16:28:32Z

Click to view CI Results

GitHub pull request #71 of commit 4902f9fd982291334357b2388a9b51f48ac39afe, no merge conflicts.
Running as SYSTEM
Setting status of 4902f9fd982291334357b2388a9b51f48ac39afe to PENDING with url https://10.20.13.93:8080/job/merlin_core/31/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_core
using credential ce87ff3c-94f0-400a-8303-cb4acb4918b5
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/core # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/core
 > git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems username and pass
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/core +refs/pull/71/*:refs/remotes/origin/pr/71/* # timeout=10
 > git rev-parse 4902f9fd982291334357b2388a9b51f48ac39afe^{commit} # timeout=10
Checking out Revision 4902f9fd982291334357b2388a9b51f48ac39afe (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 4902f9fd982291334357b2388a9b51f48ac39afe # timeout=10
Commit message: "Remove stray unnecessary pylint disable"
 > git rev-list --no-walk f068b073e6acc47171c53193d1af316a08cba262 # timeout=10
[merlin_core] $ /bin/bash /tmp/jenkins6556794169656740009.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_core/core, configfile: pyproject.toml
plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 338 items / 1 skipped
tests/unit/core/test_dispatch.py ..                                      [  0%]

tests/unit/dag/test_base_operator.py ....                                [  1%]

tests/unit/dag/test_column_selector.py ..........................        [  9%]

tests/unit/dag/test_tags.py ......                                       [ 11%]

tests/unit/dag/ops/test_selection.py ...                                 [ 12%]

tests/unit/io/test_io.py ............................................... [ 26%]

................................................................         [ 44%]

tests/unit/schema/test_column_schemas.py ............................... [ 54%]

........................................................................ [ 75%]

........................................................................ [ 96%]

.                                                                        [ 97%]

tests/unit/schema/test_schema_io.py ..                                   [ 97%]

tests/unit/utils/test_utils.py ........                                  [100%]
=============================== warnings summary ===============================

tests/unit/dag/test_base_operator.py: 4 warnings

tests/unit/io/test_io.py: 72 warnings

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(
tests/unit/io/test_io.py::test_validate_and_regenerate_dataset

/var/jenkins_home/workspace/merlin_core/core/merlin/io/parquet.py:551: DeprecationWarning: 'ParquetDataset.pieces' attribute is deprecated as of pyarrow 5.0.0 and will be removed in a future version. Specify 'use_legacy_dataset=False' while constructing the ParquetDataset, and then use the '.fragments' attribute instead.

paths = [p.path for p in pa_dataset.pieces]
tests/unit/utils/test_utils.py::test_nvt_distributed[True-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 41691 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed[True-False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 42917 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed[False-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 40807 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed[False-False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 33519 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed_force[True]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 33501 instead

warnings.warn(
tests/unit/utils/test_utils.py::test_nvt_distributed_force[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:160: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 44699 instead

warnings.warn(
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

================= 338 passed, 1 skipped, 83 warnings in 52.54s =================

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/core/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_core] $ /bin/bash /tmp/jenkins1387899683211473498.sh

Standardize our pre-commit hooks

f068b07

This includes the super-set of commit hooks from NVTabular and Merlin Models, and applies them to the whole `core` library.

karlhigley added the clean up label Apr 13, 2022

karlhigley added this to the Merlin 22.05 milestone Apr 13, 2022

karlhigley requested review from benfred and marcromeyn April 13, 2022 16:20

karlhigley self-assigned this Apr 13, 2022

benfred approved these changes Apr 13, 2022

View reviewed changes

Remove stray unnecessary pylint disable

4902f9f

karlhigley merged commit 9f37373 into NVIDIA-Merlin:main Apr 13, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Standardize our pre-commit hooks #71

Standardize our pre-commit hooks #71

Uh oh!

karlhigley commented Apr 13, 2022

Uh oh!

nvidia-merlin-bot commented Apr 13, 2022

Uh oh!

github-actions bot commented Apr 13, 2022

Uh oh!

nvidia-merlin-bot commented Apr 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Standardize our pre-commit hooks #71

Standardize our pre-commit hooks #71

Uh oh!

Conversation

karlhigley commented Apr 13, 2022

Uh oh!

nvidia-merlin-bot commented Apr 13, 2022

Uh oh!

github-actions bot commented Apr 13, 2022

Documentation preview

Uh oh!

nvidia-merlin-bot commented Apr 13, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants