Per token latency metrics - TTFT, TPOT

What's Changed

• rebaseline by @aarora79 in #140
• Add ttft tpot ttlt metrics by @aarora79 in #141
• updated manifest by @aarora79 in #142

Full Changelog: v1.0.48...v1.0.49

Phi-3, bug fixes

What's Changed

• update create combination with test count limit by @haozhx23 in #124
• min/max iterations per combination handling by @madhurprash in #133
• handling endpoint metrics by @madhurprash in #136
• add write multiple to s3 by @haozhx23 in #123
• tokenizer file handling fix by @madhurprash in #137
• Lllama3 70B on ec2 by @antara678 in #139
• adding phi config file by @madhurprash in #138

New Contributors

• @haozhx23 made their first contribution in #124

Full Changelog: v1.0.47...v1.0.48

FMBench in a Docker container

What's Changed

• Containerize FMBench by @dheerajoruganty in #127
• handling deployment return values from various scripts by @madhurprash in #128
• cloudformation template generalized for gov cloud by @madhurprash in #129
• Update eks_cluster-creation_steps.md by @antara678 in #132

Full Changelog: v1.0.46...v1.0.47

Native EC2, EKS support. FMbench now in GovCloud.

What's Changed

• adding EC2 djl rest prediction support for llama3 on fmbench by @madhurprash in #113
• Python Script to Plot Sagemaker Endpoint Utilization Metrics , Heatmap to visualize Cost for Various RPMs by @dheerajoruganty in #108
• Do eks by @bainskb in #110
• gemma support added to FMBench by @madhurprash in #116
• added the changes to get get do-eks to work up untill the metrics not… by @bainskb in #121
• Ec2 deployment by @antara678 in #125
• EKS integration with FMBench by @madhurprash in #126

New Contributors

• @bainskb made their first contribution in #110

Full Changelog: v1.0.45...v1.0.46

More analytics

What's Changed

• Update config-bedrock-llama3.yml by @antara678 in #109

Full Changelog: v1.0.44...v1.0.45

Llama3-8b on g6, SageMaker Endpoint metrics

Full Changelog: v1.0.43...v1.0.44

v1.0.42: llama3-8b all config, misc. fixes

Full Changelog: v1.0.41...v1.0.42

Easy to run in the Amazon EC2

What's Changed

• readme update by @madhurprash in #99

Full Changelog: v1.0.39...v1.0.40

FMBench internal website

What's Changed

• Add render_fmbench_website.py and Update README by @dheerajoruganty in #97

New Contributors

• @dheerajoruganty made their first contribution in #97

Full Changelog: v1.0.38...v1.0.39

FMBench on EC2 minus S3 dependency

What's Changed

• Issue 85 - Remove S3 depencies by @RajeshRamchander in #89

New Contributors

• @RajeshRamchander made their first contribution in #89

Full Changelog: v1.0.37...v1.0.38