Skip to content

Commit f8d981e

Browse files
edknvedknv
committed
Merge branch 'ci/test-multi-gpu' of github.com:edknv/dataloader into ci/test-multi-gpu
2 parents 074a33e + 55d2491 commit f8d981e

File tree

7 files changed

+50
-40
lines changed

7 files changed

+50
-40
lines changed

‎.github/workflows/docs-sched-rebuild.yaml‎

Lines changed: 27 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,10 @@ jobs:
3232
- name: Report the versions to build
3333
run: |
3434
sphinx-multiversion --dump-metadata docs/source docs/build/html | jq "keys"
35+
- name: Setup local branches for docs build
36+
run: |
37+
git branch --track main origin/main || true
38+
git branch --track stable origin/stable || true
3539
- name: Building docs (multiversion)
3640
run: |
3741
sphinx-multiversion docs/source docs/build/html
@@ -89,27 +93,30 @@ jobs:
8993
exit 0
9094
fi
9195
# If any of these commands fail, fail the build.
92-
def_branch=$(gh api "repos/${GITHUB_REPOSITORY}" --jq ".default_branch")
96+
def_branch="stable"
9397
html_url=$(gh api "repos/${GITHUB_REPOSITORY}/pages" --jq ".html_url")
94-
# Beware ugly quotation mark avoidance in the foll lines.
95-
echo '<!DOCTYPE html>' > index.html
96-
echo '<html>' >> index.html
97-
echo ' <head>' >> index.html
98-
echo ' <title>Redirect to documentation</title>' >> index.html
99-
echo ' <meta charset="utf-8">' >> index.html
100-
echo ' <meta http=equiv="refresh" content="3; URL='${html_url}${def_branch}'/index.html">' >> index.html
101-
echo ' <link rel="canonical" href="'${html_url}${def_branch}'/index.html">' >> index.html
102-
echo ' <script language="javascript">' >> index.html
103-
echo ' function redirect() {' >> index.html
104-
echo ' window.location.assign("'${html_url}${def_branch}'/index.html")' >> index.html
105-
echo ' }' >> index.html
106-
echo ' </script>' >> index.html
107-
echo ' </head>' >> index.html
108-
echo ' <body onload="redirect()">' >> index.html
109-
echo ' <p>Please follow the link to the <a href="'${html_url}${def_branch}'/index.html">' >> index.html
110-
echo ${def_branch}'</a> branch documentation.</p>' >> index.html
111-
echo ' </body>' >> index.html
112-
echo '</html>' >> index.html
98+
99+
cat > index.html << EOF
100+
<!DOCTYPE html>
101+
<html>
102+
<head>
103+
<title>Redirect to documentation</title>
104+
<meta charset="utf-8">
105+
<meta http=equiv="refresh" content="3; URL="${html_url}${def_branch}/index.html"
106+
<link rel="canonical" href="'${html_url}${def_branch}/index.html">
107+
<script language="javascript">
108+
function redirect() {
109+
window.location.assign("${html_url}${def_branch}/index.html")
110+
}
111+
</script>
112+
</head>
113+
<body onload="redirect()">
114+
<p>Please follow the link to the <a href="${html_url}${def_branch}/index.html">
115+
${def_branch}'</a> branch documentation.</p>
116+
</body>
117+
</html>
118+
EOF
119+
113120
git add index.html
114121
- name: Commit changes to the GitHub Pages branch
115122
run: |

‎README.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
![PyPI - Python Version](https://img.shields.io/pypi/pyversions/merlin-dataloader)
44
[![PyPI version shields.io](https://img.shields.io/pypi/v/merlin-dataloader.svg)](https://pypi.python.org/pypi/merlin-dataloader/)
55
![GitHub License](https://img.shields.io/github/license/NVIDIA-Merlin/dataloader)
6-
[![Documentation](https://img.shields.io/badge/documentation-blue.svg)](https://nvidia-merlin.github.io/dataloader/main/README.html)
6+
[![Documentation](https://img.shields.io/badge/documentation-blue.svg)](https://nvidia-merlin.github.io/dataloader/stable/README.html)
77

88
The merlin-dataloader lets you quickly train recommender models for TensorFlow, PyTorch and JAX. It eliminates the biggest bottleneck in training recommender models, by providing GPU optimized dataloaders that read data directly into the GPU, and then do a 0-copy transfer to TensorFlow and PyTorch using [dlpack](https://github.com/dmlc/dlpack).
99

@@ -30,7 +30,7 @@ To install from PyPi:
3030
pip install merlin-dataloader
3131
```
3232

33-
There are also [docker containers on NGC](https://nvidia-merlin.github.io/Merlin/main/containers.html) with the merlin-dataloader and dependencies included on them
33+
There are also [docker containers on NGC](https://nvidia-merlin.github.io/Merlin/stable/containers.html) with the merlin-dataloader and dependencies included on them
3434

3535
## Basic Usage
3636

‎docs/source/conf.py‎

Lines changed: 11 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -4,17 +4,19 @@
44
# list see the documentation:
55
# https://www.sphinx-doc.org/en/master/usage/configuration.html
66

7-
# -- Path setup --------------------------------------------------------------
8-
9-
# If extensions (or modules to document with autodoc) are in another directory,
10-
# add these directories to sys.path here. If the directory is relative to the
11-
# documentation root, use os.path.abspath to make it absolute, like shown here.
127
import os
8+
import re
139
import subprocess
1410
import sys
1511

1612
from natsort import natsorted
1713

14+
# -- Path setup --------------------------------------------------------------
15+
16+
# If extensions (or modules to document with autodoc) are in another directory,
17+
# add these directories to sys.path here. If the directory is relative to the
18+
# documentation root, use os.path.abspath to make it absolute, like shown here.
19+
1820
sys.path.insert(0, os.path.abspath("../../"))
1921

2022
repodir = os.path.abspath(os.path.join(__file__, r"../../.."))
@@ -97,14 +99,15 @@
9799
# at a commit (not a Git repo).
98100
if os.path.exists(gitdir):
99101
tag_refs = subprocess.check_output(["git", "tag", "-l", "v*"]).decode("utf-8").split()
102+
tag_refs = [tag for tag in tag_refs if re.match(r"^v[0-9]+.[0-9]+.[0-9]+$", tag)]
100103
tag_refs = natsorted(tag_refs)[-6:]
101104
smv_tag_whitelist = r"^(" + r"|".join(tag_refs) + r")$"
102105
else:
103106
# SMV is reading conf.py from a Git archive of the repo at a specific commit.
104107
smv_tag_whitelist = r"^v.*$"
105108

106109
# Only include main branch for now
107-
smv_branch_whitelist = "^main$"
110+
smv_branch_whitelist = "^(main|stable)$"
108111

109112
smv_refs_override_suffix = r"-docs"
110113

@@ -113,11 +116,11 @@
113116
"cudf": ("https://docs.rapids.ai/api/cudf/stable/", None),
114117
"distributed": ("https://distributed.dask.org/en/latest/", None),
115118
"torch": ("https://pytorch.org/docs/stable/", None),
116-
"merlin-core": ("https://nvidia-merlin.github.io/core/main/", None),
119+
"merlin-core": ("https://nvidia-merlin.github.io/core/stable/", None),
117120
}
118121

119122
html_sidebars = {"**": ["versions.html"]}
120-
html_baseurl = "https://nvidia-merlin.github.io/dataloader/main"
123+
html_baseurl = "https://nvidia-merlin.github.io/dataloader/stable/"
121124

122125
# Add any paths that contain custom static files (such as style sheets) here,
123126
# relative to this directory. They are copied after the builtin static files,

‎docs/source/index.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ Merlin Dataloader GitHub Repository
1515

1616
About Merlin
1717
Merlin is the overarching project that brings together the Merlin projects.
18-
See the `documentation <https://nvidia-merlin.github.io/Merlin/main/README.html>`_
18+
See the `documentation <https://nvidia-merlin.github.io/Merlin/stable/README.html>`_
1919
or the `repository <https://github.com/NVIDIA-Merlin/Merlin>`_ on GitHub.
2020

2121
Developer website for Merlin
@@ -25,4 +25,4 @@ Developer website for Merlin
2525
Index
2626
-----
2727

28-
* :ref:`genindex`
28+
* :ref:`genindex`

‎examples/01a-Getting-started-Tensorflow.ipynb‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -572,10 +572,10 @@
572572
"\n",
573573
"The libraries are designed to work closely together. We recommend to check out our examples:\n",
574574
"\n",
575-
"* [Getting Started with NVTabular: Process Tabular Data On GPU](https://github.com/NVIDIA-Merlin/NVTabular/blob/main/examples/01-Getting-started.ipynb)\n",
576-
"* [Getting Started with Merlin Models: Develop a Model for MovieLens](https://github.com/NVIDIA-Merlin/models/blob/main/examples/01-Getting-started.ipynb)\n",
575+
"* [Getting Started with NVTabular: Process Tabular Data On GPU](https://github.com/NVIDIA-Merlin/NVTabular/blob/stable/examples/01-Getting-started.ipynb)\n",
576+
"* [Getting Started with Merlin Models: Develop a Model for MovieLens](https://github.com/NVIDIA-Merlin/models/blob/stable/examples/01-Getting-started.ipynb)\n",
577577
"\n",
578-
"In the example, [From ETL to Training RecSys models - NVTabular and Merlin Models integrated example](https://github.com/NVIDIA-Merlin/models/blob/main/examples/02-Merlin-Models-and-NVTabular-integration.ipynb), we explain how the close collaboration works."
578+
"In the example, [From ETL to Training RecSys models - NVTabular and Merlin Models integrated example](https://github.com/NVIDIA-Merlin/models/blob/stable/examples/02-Merlin-Models-and-NVTabular-integration.ipynb), we explain how the close collaboration works."
579579
]
580580
}
581581
],

‎examples/01b-Getting-started-Pytorch.ipynb‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -438,8 +438,8 @@
438438
"\n",
439439
"Our libraries are designed to work closely together. We recommend to check out our examples:\n",
440440
"\n",
441-
"* [Getting Started with NVTabular: Process Tabular Data On GPU](https://github.com/NVIDIA-Merlin/NVTabular/blob/main/examples/01-Getting-started.ipynb)\n",
442-
"* [Getting Started MovieLens: Training with PyTorch](https://github.com/NVIDIA-Merlin/Merlin/blob/main/examples/getting-started-movielens/03-Training-with-PyTorch.ipynb)"
441+
"* [Getting Started with NVTabular: Process Tabular Data On GPU](https://github.com/NVIDIA-Merlin/NVTabular/blob/stable/examples/01-Getting-started.ipynb)\n",
442+
"* [Getting Started MovieLens: Training with PyTorch](https://github.com/NVIDIA-Merlin/Merlin/blob/stable/examples/getting-started-movielens/03-Training-with-PyTorch.ipynb)"
443443
]
444444
}
445445
],

‎examples/02-Multi-GPU-Tensorflow-with-Horovod.ipynb‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -359,10 +359,10 @@
359359
"\n",
360360
"The libraries are designed to work closely together. We recommend to check out our examples:\n",
361361
"\n",
362-
"* [Getting Started with NVTabular: Process Tabular Data On GPU](https://github.com/NVIDIA-Merlin/NVTabular/blob/main/examples/01-Getting-started.ipynb)\n",
363-
"* [Getting Started with Merlin Models: Develop a Model for MovieLens](https://github.com/NVIDIA-Merlin/models/blob/main/examples/01-Getting-started.ipynb)\n",
362+
"* [Getting Started with NVTabular: Process Tabular Data On GPU](https://github.com/NVIDIA-Merlin/NVTabular/blob/stable/examples/01-Getting-started.ipynb)\n",
363+
"* [Getting Started with Merlin Models: Develop a Model for MovieLens](https://github.com/NVIDIA-Merlin/models/blob/stable/examples/01-Getting-started.ipynb)\n",
364364
"\n",
365-
"In the example, [From ETL to Training RecSys models - NVTabular and Merlin Models integrated example](https://github.com/NVIDIA-Merlin/models/blob/main/examples/02-Merlin-Models-and-NVTabular-integration.ipynb), we explain how the close collaboration works."
365+
"In the example, [From ETL to Training RecSys models - NVTabular and Merlin Models integrated example](https://github.com/NVIDIA-Merlin/models/blob/stable/examples/02-Merlin-Models-and-NVTabular-integration.ipynb), we explain how the close collaboration works."
366366
]
367367
}
368368
],

0 commit comments

Comments
 (0)