Skip to content

Commit d4f1951

Browse files
micwade-awsliangfuaws-wangwileshalakhotiaaws-sadaf
authored
Neuron SDK Release 2.18.2 (#877)
* Neuron SDK Release 2.18.2 Release notes for Neuron SDK Release 2.18.2 --------- Co-authored-by: Liangfu Chen <[email protected]> Co-authored-by: Willis Wang <[email protected]> Co-authored-by: Esha Lakhotia <[email protected]> Co-authored-by: aws-sadaf <[email protected]> Co-authored-by: Roopnath <[email protected]>
1 parent ce5bb89 commit d4f1951

File tree

9 files changed

+128
-18
lines changed

9 files changed

+128
-18
lines changed

‎conf.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@
142142
#top_banner_message="<span>&#9888;</span><a class='reference internal' style='color:white;' href='https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/setup/setup-troubleshooting.html#gpg-key-update'> Neuron repository GPG key for Ubuntu installation has expired, see instructions how to update! </a>"
143143

144144

145-
top_banner_message="Neuron 2.18.1 is released! check <a class='reference internal' style='color:white;' href='https://awsdocs-neuron.readthedocs-hosted.com/en/latest/release-notes/index.html#latest-neuron-release'> What's New </a> and <a class='reference internal' style='color:white;' href='https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/announcements/index.html'> Announcements </a>"
145+
top_banner_message="Neuron 2.18.2 is released! check <a class='reference internal' style='color:white;' href='https://awsdocs-neuron.readthedocs-hosted.com/en/latest/release-notes/index.html#latest-neuron-release'> What's New </a> and <a class='reference internal' style='color:white;' href='https://awsdocs-neuron.readthedocs-hosted.com/en/latest/general/announcements/index.html'> Announcements </a>"
146146

147147

148148
html_theme = "sphinx_book_theme"

‎general/announcements/neuron2.x/announce-eos-dlami.rst‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
.. post:: April 24, 2024
2+
:language: en
3+
:tags: announce-eos-dlami, neuron-dlami
4+
5+
.. _announce-eos-dlami:
6+
7+
Announcing end of support for Neuron Release 2.18.0 Deep Learning AMIs
8+
---------------------------------------------------------------------
9+
10+
We are announcing end of support for :ref:`Neuron release 2.18.0 <neuron-2.18.0-whatsnew>` Deep Learning AMIs. DLAMIs released between March 26,2024 (2024-03-26) and April 10, 2024 (2024-04-10) were shipped without the audit package. The following are the affected DLAMIs:
11+
12+
Deep Learning AMI Neuron (Ubuntu 22.04) 20240401
13+
Deep Learning AMI Neuron PyTorch 1.13 (Amazon Linux 2) 20240328
14+
Deep Learning AMI Neuron PyTorch 1.13 (Amazon Linux 2) 20240402
15+
Deep Learning AMI Neuron PyTorch 1.13 (Amazon Linux 2) 20240409
16+
Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 20240328
17+
Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 20240402
18+
Deep Learning AMI Neuron PyTorch 1.13 (Ubuntu 20.04) 20240409
19+
Deep Learning AMI Neuron TensorFlow 2.10 (Amazon Linux 2) 20240328
20+
Deep Learning AMI Neuron TensorFlow 2.10 (Amazon Linux 2) 20240402
21+
Deep Learning AMI Neuron TensorFlow 2.10 (Amazon Linux 2) 20240409
22+
Deep Learning AMI Neuron TensorFlow 2.10 (Ubuntu 20.04) 20240328
23+
Deep Learning AMI Neuron TensorFlow 2.10 (Ubuntu 20.04) 20240402
24+
Deep Learning AMI Neuron TensorFlow 2.10 (Ubuntu 20.04) 20240409
25+
Deep Learning Base Neuron AMI (Amazon Linux 2) 20240401
26+
Deep Learning Base Neuron AMI (Amazon Linux 2) 20240408
27+
Deep Learning Base Neuron AMI (Ubuntu 20.04) 20240401
28+
Deep Learning Base Neuron AMI (Ubuntu 20.04) 20240408
29+
30+
Current users of the above :ref:`Neuron release 2.18 <neuron-2.18.0-whatsnew>` Deep Learning AMIs are required to upgrade to the latest DLAMIs in order to consume those with the audit package installed. For instructions to upgrade to the latest AMI, see the :ref:`DLAMI User Guide <neuron-dlami-overview>` or find the specific DLAMI image id for the latest Neuron release with :ref:`SSM parameters <ssm-parameter-neuron-dlami>`.

‎general/announcements/neuron2.x/announce-eos-multiframework-dlamis-inf1.rst‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
.. post:: April 24, 2024
2+
:language: en
3+
:tags: announce-eos-dlamis-inf1, dlami-inf1
4+
5+
.. _announce-update-multiframework-dlami:
6+
7+
Announcing end of support for Neuron virtual environments in AWS Deep Learning AMI (Amazon Linux 2)
8+
----------------------------------------------------------------------------------------------------
9+
10+
:ref:`Neuron release 2.18.2 <neuron-2.18.2-whatsnew>` will be the last release that will include support for the following virtual environments in AWS Deep Learning AMI (Amazon Linux 2):
11+
12+
``aws_neuron_pytorch_p38: PyTorch 1.13, Python 3.8``
13+
``aws_neuron_tensorflow2_p38: TensorFlow 2.10, Python 3.8``
14+
15+
Future releases will not include Neuron support for these virtual environments.
16+
17+
Current users of Neuron virtual environments in `AWS Deep Learning AMI (Amazon Linux 2) <https://aws.amazon.com/releasenotes/aws-deep-learning-ami-amazon-linux-2/>`_ are required to migrate to the `Neuron multi framework DLAMI <https://aws.amazon.com/releasenotes/aws-deep-learning-ami-neuron-ubuntu-22-04/>`_.
18+
19+
To see a list of Neuron supported virtual environments, please refer to :ref:`Neuron Multi Framework DLAMI User Guide <neuron-dlami-overview>`.

‎libraries/transformers-neuronx/transformers-neuronx-developer-guide-for-continuous-batching.rst‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ In the following example we demonstrate how to perform continuous batching with
6262
6363
# Create an LLM.
6464
llm = LLM(
65-
model="openlm-research/open_llama_3b",
65+
model="TinyLlama/TinyLlama-1.1B-Chat-v1.0",
6666
max_num_seqs=8,
6767
# The max_model_len and block_size arguments are required to be same as max sequence length,
6868
# when targeting neuron device. Currently, this is a known limitation in continuous batching
@@ -71,7 +71,8 @@ In the following example we demonstrate how to perform continuous batching with
7171
block_size=128,
7272
# The device can be automatically detected when AWS Neuron SDK is installed.
7373
# The device argument can be either unspecified for automated detection, or explicitly assigned.
74-
device="neuron")
74+
device="neuron",
75+
tensor_parallel_size=2)
7576
# Generate texts from the prompts. The output is a list of RequestOutput objects
7677
# that contain the prompt, generated text, and other information.
7778
outputs = llm.generate(prompts, sampling_params)

‎neuron-runtime/nrt-configurable-parameters.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ configure NeuronX Runtime behavior.
4949
- Timeout for execution in seconds
5050
- Integer
5151
- 0 to INT_MAX
52-
- 30 on inf1, 600 on trn1/inf2
52+
- 30
5353
- 2.0+
5454
* - ``NEURON_RT_VALIDATE_HASH``
5555
- Validate NEFF contents before loading into accelerator

‎release-notes/compiler/neuronx-cc/index.rst‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,12 @@ Neuron Compiler (``neuronx-cc``) release notes
88
:depth: 2
99

1010

11+
Neuron Compiler [2.13.72.0]
12+
----------------------------
13+
Date: 04/25/2024
14+
15+
* Minor bug fixes and enhancements
16+
1117

1218
Neuron Compiler [2.13.68.0]
1319
----------------------------

‎release-notes/index.rst‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,12 @@ What's New
1111
.. _neuron-2.18.0-whatsnew:
1212

1313

14+
Neuron 2.18.2 (04/25/2024)
15+
--------------------------
16+
Patch release with minor Neuron Compiler bug fixes and enhancements. See more in :ref:`neuronx-cc-rn`
17+
18+
19+
1420
Neuron 2.18.1 (04/10/2024)
1521
--------------------------
1622

@@ -343,27 +349,27 @@ Release Artifacts
343349
Trn1 packages
344350
^^^^^^^^^^^^^^
345351

346-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=trn1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
352+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=trn1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.2
347353

348354
Inf2 packages
349355
^^^^^^^^^^^^^^
350356

351-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf2 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
357+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf2 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.2
352358

353359
Inf1 packages
354360
^^^^^^^^^^^^^^
355361

356-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
362+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.2
357363

358364
Supported Python Versions for Inf1 packages
359365
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
360366

361-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=pyversions --instance=inf1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
367+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=pyversions --instance=inf1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.2
362368

363369
Supported Python Versions for Inf2/Trn1 packages
364370
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
365371

366-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=pyversions --instance=inf2 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
372+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=pyversions --instance=inf2 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.2
367373

368374
Supported Numpy Versions
369375
^^^^^^^^^^^^^^^^^^^^^^^^

‎release-notes/prev/content.rst‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -7,20 +7,20 @@ Previous Releases Artifacts (Neuron 2.x)
77
:local:
88
:depth: 1
99

10-
Neuron 2.18.0 (04/01/2024)
10+
Neuron 2.18.1 (04/10/2024)
1111
---------------------------
1212

1313
Trn1 packages
1414
^^^^^^^^^^^^^
15-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=trn1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.0
15+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=trn1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
1616

1717
Inf2 packages
1818
^^^^^^^^^^^^^
19-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf2 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.0
19+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf2 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
2020

2121
Inf1 packages
2222
^^^^^^^^^^^^^
23-
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.0
23+
.. program-output:: python3 src/helperscripts/n2-helper.py --list=packages --instance=inf1 --file=src/helperscripts/n2-manifest.json --neuron-version=2.18.1
2424

2525

2626
Neuron 2.17.0 (02/13/2024)

‎src/helperscripts/n2-manifest.json‎

Lines changed: 53 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,13 +4,13 @@
44
{"repo_type":"rpm", "repo_url":"https://yum.repos.neuron.amazonaws.com/"},
55
{"repo_type":"deb", "repo_url":"https://apt.repos.neuron.amazonaws.com/"}
66
],
7-
"manifest_date": "04/10/2024",
8-
"manifest_version": "2.18.1",
7+
"manifest_date": "04/25/2024",
8+
"manifest_version": "2.18.2",
99
"latest_release": [
1010
{"instance":"inf1", "version":"2.18.0"},
11-
{"instance":"trn1", "version":"2.18.1"},
12-
{"instance":"inf2", "version":"2.18.1"},
13-
{"instance":"trn1n", "version":"2.18.1"}
11+
{"instance":"trn1", "version":"2.18.2"},
12+
{"instance":"inf2", "version":"2.18.2"},
13+
{"instance":"trn1n", "version":"2.18.2"}
1414
],
1515
"os_properties": [
1616
{"os":"ubuntu18", "default_python_version":"3.7"},
@@ -86,6 +86,54 @@
8686
{"name":"neuronx_distributed","component":"Neuron Distributed","category":"na","package_type":"os","use_cases":["inference","training"],"pin_major":"false"}
8787
],
8888
"neuron_releases": [
89+
{"neuron_version":"2.18.2", "packages": [
90+
{"name":"aws-neuronx-collectives","version":"2.20.22.0","supported_instances":["trn1","inf2"],"supported_python_versions":[]},
91+
{"name":"aws-neuronx-dkms","version":"2.16.7.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
92+
{"name":"aws-neuronx-gpsimd-customop-lib","version":"0.9.1.0","supported_instances":["trn1","inf2"],"supported_python_versions":[]},
93+
{"name":"aws-neuronx-gpsimd-tools","version":"0.9.0.0","supported_instances":["trn1","inf2"],"supported_python_versions":[]},
94+
{"name":"aws-neuronx-k8-plugin","version":"2.20.13.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
95+
{"name":"aws-neuronx-k8-scheduler","version":"2.20.13.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
96+
{"name":"aws-neuronx-oci-hook","version":"2.3.0.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
97+
{"name":"aws-neuronx-runtime-discovery","version":"2.9","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
98+
{"name":"aws-neuronx-runtime-lib","version":"2.20.22.0","supported_instances":["trn1","inf2"],"supported_python_versions":[]},
99+
{"name":"aws-neuronx-tools","version":"2.17.1.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
100+
{"name":"dmlc_nnvm","version":"1.19.0.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
101+
{"name":"dmlc_topi","version":"1.19.0.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
102+
{"name":"dmlc_tvm","version":"1.19.0.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
103+
{"name":"inferentia_hwm","version":"1.17.0.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
104+
{"name":"libneuronxla","version":"2.0.965","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
105+
{"name":"libneuronxla","version":"0.5.971","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
106+
{"name":"mx_neuron","version":"1.8.0.2.4.50.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
107+
{"name":"mxnet_neuron","version":"1.5.1.1.10.0.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
108+
{"name":"neuron-cc","version":"1.22.0.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
109+
{"name":"neuronperf","version":"1.8.55.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
110+
{"name":"neuronx-cc","version":"2.13.72.0","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
111+
{"name":"neuronx_distributed","version":"0.7.0","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
112+
{"name":"tensorboard-plugin-neuronx","version":"2.6.7.0","supported_instances":["trn1","inf2"],"supported_python_versions":[]},
113+
{"name":"tensorflow-model-server-neuronx","version":"2.10.1.2.10.19.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
114+
{"name":"tensorflow-model-server-neuronx","version":"2.7.4.2.10.19.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
115+
{"name":"tensorflow-model-server-neuronx","version":"2.8.4.2.10.19.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
116+
{"name":"tensorflow-model-server-neuronx","version":"2.9.3.2.10.19.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},
117+
{"name":"tensorflow-neuron","version":"2.10.1.2.10.19.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
118+
{"name":"tensorflow-neuron","version":"2.7.4.2.10.19.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
119+
{"name":"tensorflow-neuron","version":"2.8.4.2.10.19.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
120+
{"name":"tensorflow-neuron","version":"2.9.3.2.10.19.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
121+
{"name":"tensorflow-neuronx","version":"2.10.1.2.1.0","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
122+
{"name":"tensorflow-neuronx","version":"2.7.4.2.1.0","supported_instances":["trn1","inf."],"supported_python_versions":["3.8","3.9","3.10"]},
123+
{"name":"tensorflow-neuronx","version":"2.8.4.2.1.0","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
124+
{"name":"tensorflow-neuronx","version":"2.9.3.2.1.0","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
125+
{"name":"torch-neuron","version":"1.10.2.2.9.74.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
126+
{"name":"torch-neuron","version":"1.11.0.2.9.74.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
127+
{"name":"torch-neuron","version":"1.12.1.2.9.74.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
128+
{"name":"torch-neuron","version":"1.13.1.2.9.74.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
129+
{"name":"torch-neuron","version":"1.9.1.2.9.74.0","supported_instances":["inf1"],"supported_python_versions":["3.8","3.9","3.10"]},
130+
{"name":"torch-neuronx","version":"1.13.1.1.14.0","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
131+
{"name":"torch-neuronx","version":"2.1.2.2.1.0","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
132+
{"name":"torch_xla","version":"1.13.1+torchneurone","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
133+
{"name":"torch_xla","version":"2.1.2","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
134+
{"name":"transformers-neuronx","version":"0.10.0.360","supported_instances":["trn1","inf2"],"supported_python_versions":["3.8","3.9","3.10"]},
135+
{"name":"efa-installer","version":"na","supported_instances":["trn1"],"supported_python_versions":[]}
136+
]},
89137
{"neuron_version":"2.18.1", "packages": [
90138
{"name":"aws-neuronx-collectives","version":"2.20.22.0","supported_instances":["trn1","inf2"],"supported_python_versions":[]},
91139
{"name":"aws-neuronx-dkms","version":"2.16.7.0","supported_instances":["inf1","trn1","inf2"],"supported_python_versions":[]},

0 commit comments

Comments
 (0)