Sync with main prior to v0.6.0 release

Signed-off-by: Abolfazl Shahbazi <[email protected]>

Author: ashahba

.gitignore

@@ -9,7 +9,6 @@ _autosummary
 **.log
 **/*.cache
 **/**.whl
-**/**/models/
 **/**venv
 **venv*
 build/

DATASETS.md

@@ -28,3 +28,6 @@ This is a comprehensive list of public datasets used by this repository.
 | [CDD-CESM](https://wiki.cancerimagingarchive.net/pages/viewpage.action?pageId=109379611) |  PyTorch & TensorFlow | Image & Text Classification |
 | [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/) |  PyTorch & TensorFlow | Text Classification |
 | [MVTec](https://www.mvtec.com/company/research/datasets/mvtec-ad) |  PyTorch | Anomaly Detection |
+| [Code Alpaca](https://github.com/sahil280114/codealpaca) |  PyTorch | Text Generation |
+| [Dolly-15k](https://huggingface.co/datasets/databricks/databricks-dolly-15k) |  PyTorch | Text Generation |
+| [RedPajama-Data](https://github.com/togethercomputer/RedPajama-Data) |  PyTorch | Text Generation |

GetStarted.md

@@ -83,14 +83,14 @@ approaches.
       install after each code change (preferred):
 
       ```
-      pip install --editable .
+      pip install .
       ```
 
       or build and install a wheel:
 
       ```
       python setup.py bdist_wheel
-      pip install dist/intel_transfer_learning_tool-0.5.0-py3-none-any.whl
+      pip install dist/intel_transfer_learning_tool-0.6.0-py3-none-any.whl
       ```
 
 
@@ -118,7 +118,7 @@ programming experience is required, and you'll need basic Python knowledge to
 use the API. Choose the approach that works best for you.
 
 
-### Run Using the No-Code CLI
+### a) Run Using the No-Code CLI
 
 Let's continue from the previous step where you prepared the dataset, and train
 a model using CLI commands.  This example uses the CLI to train an image
@@ -179,7 +179,7 @@ that includes evaluation, benchmarking, and quantization in the datasets.
 Read about all the CLI commands in the [CLI reference](/cli.md).
 Find more examples in our list of [Examples](examples/README.md).
 
-### Run Using the Low-Code API
+### b) Run Using the Low-Code API
 
 The following Python code example trains an image classification model with the TensorFlow
 flowers dataset using API calls from Python.  The model is
@@ -254,7 +254,7 @@ an Intel-optimized saved model for deployment. The sample CLI and API commands
 we've presented show how to execute end-to-end transfer learning workflows. 
 
 For the no-code CLI, you can follow a
-complete example that includes trainng, evaluation, benchmarking, and quantization
+complete example that includes training, evaluation, benchmarking, and quantization
 in the datasets, as well as some additional models in the [Beyond Get Started
 CLI example](examples/cli/README.md) documentation. You can also read about all the
 CLI commands in the [CLI reference](/cli.md).

Legal.md

@@ -6,29 +6,29 @@ Intel® Transfer Learning Tool scripts are not intended for benchmarking Intel®
 performance and/or benchmarking information on specific Intel platforms, visit
 https://www.intel.ai/blog.
 
-Intel is committed to the respect of human rights and avoiding complicity in
-human rights abuses, a policy reflected in the Intel Global Human Rights
-Principles. Accordingly, by accessing the Intel material on this platform you
-agree that you will not use the material in a product or application that causes
-or contributes to a violation of an internationally recognized human right.
+Intel is committed to respecting human rights and avoiding complicity in human rights abuses.
+See [Intel's Global Human Rights Principles](https://www.intel.com/content/www/us/en/policy/policy-human-rights.html).
+Intel's products and software are intended only to be used in applications that do not cause or
+contribute to a violation of an internationally recognized human right.
 
 ## License
 
 Intel® Transfer Learning Tool, documentation, and example code are all licensed
 under Apache License Version 2.0.
 
-## Datasets
+## Datasets and Models
 
-To the extent that any [public datasets](DATASETS.md) are referenced by Intel or accessed using
-tools or code on this site those datasets are provided by the third party
-indicated as the data source. Intel does not create the data, or datasets, and
-does not warrant their accuracy or quality. By accessing the public dataset(s)
-you agree to the terms associated with those datasets and that your use complies
-with the applicable license.
+To the extent that any [data, datasets]((DATASETS.md), or [models](Models.md) are referenced
+by Intel or accessed using tools or code on this site such data, datasets and models are provided
+by the third party indicated as the source of such content. Intel does not create the data, datasets,
+or models, provide a license to any third-party data, datasets, or models referenced, and does not
+warrant their accuracy or quality. By accessing such data, dataset(s) or model(s) you agree to the
+terms associated with that content and that your use complies with the applicable license.
 
-Intel expressly disclaims the accuracy, adequacy, or completeness of any public
-datasets, and is not liable for any errors, omissions, or defects in the data,
-or for any reliance on the data. Intel is not liable for any liability or
-damages relating to your use of public datasets.
+Intel expressly disclaims the accuracy, adequacy, or completeness of any data, datasets or models,
+and is not liable for any errors, omissions, or defects in such content, or for any reliance thereon.
+Intel also expressly disclaims any warranty of non-infringement with respect to such data, dataset(s),
+or model(s). Intel is not liable for any liability or damages relating to your use of such data,
+datasets, or models.
 
 \*Other names and brands may be claimed as the property of others. [Trademarks](http://www.intel.com/content/www/us/en/legal/trademarks.html)

Makefile

@@ -33,7 +33,7 @@ tlt_test_venv: $(CURDIR)/tests/requirements-test.txt
 	@test -d tlt_test_venv || virtualenv -p python3 tlt_test_venv
 
 	@echo "Building the TLT API in tlt_test_venv env..."
-	@. $(ACTIVATE_TEST_VENV) && pip install --editable .
+	@. $(ACTIVATE_TEST_VENV) && pip install --extra-index-url https://download.pytorch.org/whl/cpu .
 
 	@echo "Installing test dependencies..."
 	@. $(ACTIVATE_TEST_VENV) && pip install -r $(CURDIR)/tests/requirements-test.txt
@@ -43,7 +43,7 @@ tlt_notebook_venv: $(CURDIR)/notebooks/requirements.txt
 	@test -d tlt_notebook_venv || virtualenv -p python3 tlt_notebook_venv
 
 	@echo "Installing TF & PYT notebook dependencies..."
-	@. $(ACTIVATE_NOTEBOOK_VENV) && pip install -r $(CURDIR)/notebooks/requirements.txt
+	@. $(ACTIVATE_NOTEBOOK_VENV) && pip install --extra-index-url https://download.pytorch.org/whl/cpu -r $(CURDIR)/notebooks/requirements.txt
 
 test: unittest integration
 
@@ -57,7 +57,10 @@ integration: tlt_test_venv
 
 lint: tlt_test_venv
 	@echo "Style checks..."
-	@. $(ACTIVATE_TEST_VENV) && flake8 tlt tests downloader
+	@. $(ACTIVATE_TEST_VENV) && \
+        	flake8 tlt tests downloader && \
+		flake8 notebooks && \
+		flake8 docs
 
 clean:
 	rm -rf tlt_test_venv
@@ -76,7 +79,7 @@ test_docs: html
 
 tlt_notebook_venv: tlt_test_venv
 	@echo "Installing notebook dependencies..."
-	@. $(ACTIVATE_TEST_VENV) && pip install -r $(CURDIR)/notebooks/requirements.txt
+	@. $(ACTIVATE_TEST_VENV) && pip install --extra-index-url https://download.pytorch.org/whl/cpu -r $(CURDIR)/notebooks/requirements.txt
 
 test_notebook_custom: tlt_notebook_venv
 	@echo "Testing Jupyter notebooks with custom datasets..."

Models.md

@@ -141,38 +141,45 @@
 | bert-large-uncased | PyTorch | Hugging Face |
 | clinical-bert | PyTorch | Hugging Face |
 | distilbert-base-uncased | PyTorch | Hugging Face |
-| google/bert_uncased_L-10_H-128_A-2 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-10_H-256_A-4 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-10_H-512_A-8 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-10_H-768_A-12 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-12_H-128_A-2 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-12_H-256_A-4 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-12_H-512_A-8 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-12_H-768_A-12 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-2_H-128_A-2 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-2_H-256_A-4 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-2_H-512_A-8 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-2_H-768_A-12 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-4_H-128_A-2 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-4_H-256_A-4 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-4_H-512_A-8 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-4_H-768_A-12 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-6_H-128_A-2 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-6_H-256_A-4 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-6_H-512_A-8 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-6_H-768_A-12 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-8_H-128_A-2 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-8_H-256_A-4 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-8_H-512_A-8 | TensorFlow | Hugging Face |
-| google/bert_uncased_L-8_H-768_A-12 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-10_H-128_A-2 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-10_H-256_A-4 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-10_H-512_A-8 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-10_H-768_A-12 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-12_H-128_A-2 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-12_H-256_A-4 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-12_H-512_A-8 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-12_H-768_A-12 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-2_H-128_A-2 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-2_H-256_A-4 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-2_H-512_A-8 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-2_H-768_A-12 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-4_H-128_A-2 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-4_H-256_A-4 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-4_H-512_A-8 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-4_H-768_A-12 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-6_H-128_A-2 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-6_H-256_A-4 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-6_H-512_A-8 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-6_H-768_A-12 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-8_H-128_A-2 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-8_H-256_A-4 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-8_H-512_A-8 | TensorFlow | Hugging Face |
+| google_bert_uncased_L-8_H-768_A-12 | TensorFlow | Hugging Face |
 
 ## Image Anomaly Detection
 
 | Model name | Framework | Model Hub |
 |------------|-----------|-----------|
-| resnet101 | PyTorch | Torchvision |
-| resnet152 | PyTorch | Torchvision |
 | resnet18 | PyTorch | Torchvision |
-| resnet34 | PyTorch | Torchvision |
 | resnet50 | PyTorch | Torchvision |
 
+## Text Generation
+
+| Model name | Framework | Model Hub |
+|------------|-----------|-----------|
+| bloom-560m | PyTorch | Hugging Face |
+| bloomz-3b | PyTorch | Hugging Face |
+| bloomz-560m | PyTorch | Hugging Face |
+| distilgpt2 | PyTorch | Hugging Face |
+| gpt-j-6b | PyTorch | Hugging Face |
+| gpt2 | PyTorch | Hugging Face |

README.md

@@ -1,3 +1,9 @@
+![Style Checks](https://github.com/intel-innersource/frameworks.ai.transfer-learning/actions/workflows/style-test.yaml/badge.svg)
+![Doc Test](https://github.com/intel-innersource/frameworks.ai.transfer-learning/actions/workflows/docs-test.yaml/badge.svg)
+![Unit Tests](https://github.com/intel-innersource/frameworks.ai.transfer-learning/actions/workflows/unittest.yaml/badge.svg)
+![Integration Tests](https://github.com/intel-innersource/frameworks.ai.transfer-learning/actions/workflows/integration.yaml/badge.svg)
+![Notebook Test](https://github.com/intel-innersource/frameworks.ai.transfer-learning/actions/workflows/notebook-test.yaml/badge.svg)
+
 *Note: You may find it easier to read about Intel Transfer Learning tool, follow the Get
 Started guide, and browse the API material from our published documentation site
 https://intelai.github.io/transfer-learning.*
@@ -24,8 +30,13 @@ demonstrate its usage.
 
 **Features:**
 * Supports PyTorch\* and TensorFlow\*
-* Select from over [100 image classification and text classification models](Models.md) from
+* Select from over [100 computer vision and natural language processing models](Models.md) from
   Torchvision, PyTorch Hub, TensorFlow Hub, Keras, and Hugging Face
+* Use cases include:
+  * Image Classification
+  * Text Classification
+  * Text Generation
+  * Vision Anomaly Detection
 * Use your own custom dataset or get started quickly with built-in datasets
 * Automatically create a trainable classification layer customized for your dataset
 * Pre-process your dataset using scaling, cropping, batching, and splitting
@@ -37,7 +48,7 @@ demonstrate its usage.
 * Quantize to INT8 to reduce model size and speed up inference using Intel® Neural Compressor
 * Optimize model for FP32 inference using Intel Neural Compressor
 * Reduce training time with auto-mixed precision for select hardware platforms
-* Further reduce training time with multinode training for PyTorch
+* Further reduce training time with multinode training
 
 ## How the Intel Transfer Learning Tool Works
 
@@ -46,7 +57,7 @@ PyTorch using either no-code command line interface (CLI) commands at a bash
 prompt, or low-code application programming interface (API) calls from a Python
 script.
 
-Use your own dataset or select an existing image or text classification dataset listed in the
+Use your own dataset or select an existing image or text dataset listed in the
 [public datasets](DATASETS.md) documentation. Construct your own CLI or API commands for training, evaluation,
 and optimization using the TensorFlow or PyTorch framework, and finally export
 your saved model optimized for inference on Intel CPUs.
@@ -89,6 +100,10 @@ command can be found using, for example, `tlt train --help`.
 
 <!-- ExpandGetStarted-End -->
 
+## Note on Evaluation and Bias
+
+Intel Transfer Learning Tool provides standard evaluation metrics such as accuracy and loss for validation/test/train sets. While important, it's essential to acknowledge that these metrics may not explicitly capture biases. Users should be cautious and consider potential biases by analyzing disparities in the data and model prediction. Techniques such as confusion matrices, PR curves, ROC curves, local attribution-based and `gradCAM` explanations, can all be good indicators for bias. Clear documentation of model behavior and performance is also crucial for iterative bias mitigation. [Intel® Explainable AI Tools](https://github.com/IntelAI/intel-xai-tools/tree/main) provides components that demonstrate the aformentioned techniques with [Explainer](https://github.com/IntelAI/intel-xai-tools/tree/main/explainer), a simple API providing post-hoc model distillation and visualization methods, as well as The [Model Card Generator](https://github.com/IntelAI/intel-xai-tools/tree/main/model_card_gen) which provides an interactive HTML report that containing these workflows and demonstrations of model behavior.
+
 ## Support
 
 The Intel Transfer Learning Tool team tracks bugs and enhancement requests using