Skip to content

Commit 3d0fcfa

Browse files
committed
readme updates
1 parent 92923ac commit 3d0fcfa

File tree

1 file changed

+5
-23
lines changed

1 file changed

+5
-23
lines changed

README.md

Lines changed: 5 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -149,7 +149,6 @@ Llama3 is now available on SageMaker (read [blog post](https://aws.amazon.com/bl
149149
1. Don't delete endpoints for the bring your own endpoint case.
150150
1. Fix bug with business summary chart.
151151

152-
153152
### v1.0.32
154153

155154
1. Report enhancements: New business summary chart, config file embedded in the report, version numbering and others.
@@ -305,27 +304,10 @@ For some enterprise scenarios it might be desirable to run `FMBench` directly on
305304
306305
1. All metrics are stored in the `/tmp/fmbench-write` directory created automatically by the `fmbench` package. Once the run completes all files are copied locally in a `results-*` folder as usual.
307306
308-
### Steps to run
309-
310-
1. `pip install` the `FMBench` package from PyPi.
311-
312-
1. Create a config file using one of the config files available [here](https://github.com/aws-samples/foundation-model-benchmarking-tool/tree/main/src/fmbench/configs).
313-
1. The configuration file is a YAML file containing configuration for all steps of the benchmarking process. It is recommended to create a copy of an existing config file and tweak it as necessary to create a new one for your experiment.
314-
315-
1. Create the read and write buckets as mentioned in the prerequisites section. Mention the respective directories for your read and write buckets within the config files.
316-
317-
1. Run the `FMBench` tool from the command line.
318-
319-
```{.bash}
320-
# the config file path could be an S3 path and https path
321-
# or even a path to a file on the local filesystem
322-
fmbench --config-file /path/to/config/file
323-
```
324-
325-
1. Depending upon the experiments in the config file, the `FMBench` run may take a few minutes to several hours. Once the run completes, you can find the report and metrics in the write S3 bucket set in the [config file](https://github.com/aws-samples/foundation-model-benchmarking-tool/blob/main/src/fmbench/configs/config-mistral-7b-tgi-g5.yml#L12). The report is generated as a markdown file called `report.md` and is available in the metrics directory in the write S3 bucket.
326-
327307
## Results
328308
309+
Depending upon the experiments in the config file, the `FMBench` run may take a few minutes to several hours. Once the run completes, you can find the report and metrics in the local `results-*` folder in the directory from where `FMBench` was run. The rpeort and metrics are also written to the write S3 bucket set in the [config file](https://github.com/aws-samples/foundation-model-benchmarking-tool/blob/main/src/fmbench/configs/config-mistral-7b-tgi-g5.yml#L12).
310+
329311
Here is a screenshot of the `report.md` file generated by `FMBench`.
330312
![Report](https://github.com/aws-samples/foundation-model-benchmarking-tool/blob/main/img/results.gif?raw=true)
331313
@@ -368,15 +350,15 @@ You can create an internal `FMBench` website to view results from multiple runs
368350
Choose any config file from the [`bedrock`](https://github.com/aws-samples/foundation-model-benchmarking-tool/tree/main/src/fmbench/configs/bedrock) folder and either run these directly or use them as templates for creating new config files specific to your use-case. Here is an example for benchmarking the `Llama3` models on Bedrock.
369351
370352
```{.bash}
371-
fmbench --config-file https://github.com/aws-samples/foundation-model-benchmarking-tool/blob/main/src/fmbench/configs/bedrock/config-bedrock-llama3.yml > fmbench.log 2>&1
353+
fmbench --config-file https://raw.githubusercontent.com/aws-samples/foundation-model-benchmarking-tool/main/src/fmbench/configs/bedrock/config-bedrock-llama3.yml > fmbench.log 2>&1
372354
```
373355

374356
### Benchmark models on SageMaker
375357

376358
Choose any config file from the model specific folders, for example the [`Llama3`](https://github.com/aws-samples/foundation-model-benchmarking-tool/tree/main/src/fmbench/configs/llama3) folder for `Llama3` family of models. These configuration files also include instructions for `FMBench` to first deploy the model on SageMaker using your configured instance type and inference parameters of choice and then run the benchmarking. Here is an example for benchmarking `Llama3-8b` model on an `ml.inf2.24xlarge` and `ml.g5.12xlarge` instance.
377359

378360
```{.bash}
379-
fmbench --config-file https://github.com/aws-samples/foundation-model-benchmarking-tool/blob/main/src/fmbench/configs/llama3/8b/config-llama3-8b-inf2-g5.yml > fmbench.log 2>&1
361+
fmbench --config-file https://raw.githubusercontent.com/aws-samples/foundation-model-benchmarking-tool/main/src/fmbench/configs/llama3/8b/config-llama3-8b-inf2-g5.yml > fmbench.log 2>&1
380362
```
381363

382364
### Benchmark models on EKS
@@ -408,7 +390,7 @@ The steps for deploying the model on your EKS cluster are described below.
408390
1. Run the `Llama3-8b` benchmarking using the command below (replace the config file as needed for a different model). This will first deploy the model on your EKS cluster and then run benchmarking on the deployed model.
409391

410392
```{.bash}
411-
fmbench --config-file https://github.com/aws-samples/foundation-model-benchmarking-tool/blob/main/src/fmbench/configs/llama3/8b/config-llama3-8b-eks-inf2.yml > fmbench.log 2>&1
393+
fmbench --config-file https://raw.githubusercontent.com/aws-samples/foundation-model-benchmarking-tool/main/src/fmbench/configs/llama3/8b/config-llama3-8b-eks-inf2.yml > fmbench.log 2>&1
412394
```
413395
414396
1. As the model is getting deployed you might want to run the following `kubectl` commands to monitor the deployment progress. Set the _model_namespace_ to `llama3` or `mistral` or a different model as appropriate.

0 commit comments

Comments
 (0)