nova models, multi-modal

Author: aarora79

README.md

@@ -87,6 +87,10 @@ Llama3 is now available on SageMaker (read [blog post](https://aws.amazon.com/bl
 
 ## New in this release
 
+## 2.0.22
+1. Benchmarks for the [Amazon Nova](https://docs.aws.amazon.com/nova/latest/userguide/what-is-nova.html) family of models.
+1. Benchmarks for multi-modal models: LLama3.2-11B, Claude 3 Sonnet and Claude 3.5 Sonnet using the [ScienceQA](https://huggingface.co/datasets/derek-thomas/ScienceQA) dataset.
+
 ## 2.0.21
 
 1. Dynamically get EC2 pricing from Boto3 API.
@@ -95,10 +99,6 @@ Llama3 is now available on SageMaker (read [blog post](https://aws.amazon.com/bl
 ## 2.0.20
 1. Add `hf_tokenizer_model_id` parameter to automatically download tokenizers from Hugging Face.
 
-## 2.0.19
-1. Config files for `Llama3.1-1b` on AMD/Intel CPU instance types.
-1. Bug fixes for token counting for vLLM.
-
 
 [Release history](./release_history.md)
 

docs/releases.md

@@ -1,5 +1,9 @@
 # Releases
 
+## 2.0.22
+1. Benchmarks for the [Amazon Nova](https://docs.aws.amazon.com/nova/latest/userguide/what-is-nova.html) family of models.
+1. Benchmarks for multi-modal models: LLama3.2-11B, Claude 3 Sonnet and Claude 3.5 Sonnet using the [ScienceQA](https://huggingface.co/datasets/derek-thomas/ScienceQA) dataset.
+
 ## 2.0.21
 
 1. Dynamically get EC2 pricing from Boto3 API.

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "fmbench"
-version = "2.0.21"
+version = "2.0.22"
 description ="Benchmark performance of **any Foundation Model (FM)** deployed on **any AWS Generative AI service**, be it **Amazon SageMaker**, **Amazon Bedrock**, **Amazon EKS**, or **Amazon EC2**. The FMs could be deployed on these platforms either directly through `FMbench`, or, if they are already deployed then also they could be benchmarked through the **Bring your own endpoint** mode supported by `FMBench`."
 authors = ["Amit Arora <[email protected]>", "Madhur Prashant <[email protected]>"]
 readme = "README.md"

release_history.md

@@ -1,3 +1,7 @@
+## 2.0.19
+1. Config files for `Llama3.1-1b` on AMD/Intel CPU instance types.
+1. Bug fixes for token counting for vLLM.
+
 ## 2.0.18
 1. Delete SageMaker endpoint as soon as the run finishes.