diff --git a/Instructions/Exercises/04b-finetune-model.md b/Instructions/Exercises/04b-finetune-model.md index f99cf04b..c779d3c7 100644 --- a/Instructions/Exercises/04b-finetune-model.md +++ b/Instructions/Exercises/04b-finetune-model.md @@ -3,7 +3,7 @@ lab: title: 'Fine-tune a language model' description: 'Learn how to use your own training data to fine-tune a model and customize its behavior.' level: 300 - duration: 60 + duration: 90 --- # Fine-tune a language model @@ -14,7 +14,7 @@ In this exercise, you'll fine-tune a language model with Microsoft Foundry that Imagine you work for a travel agency and you're developing a chat application to help people plan their vacations. The goal is to create a simple and inspiring chat that suggests destinations and activities with a consistent, friendly conversational tone. -This exercise will take approximately **60** minutes\*. +This exercise will take approximately **90** minutes\*. > \* **Note**: This timing is an estimate based on the average experience. Fine-tuning is dependent on cloud infrastructure resources, which can take a variable amount of time to provision depending on data center capacity and concurrent demand. Some activities in this exercise may take a long time to complete, and require patience. If things are taking a while, consider reviewing the [Microsoft Foundry fine-tuning documentation](https://learn.microsoft.com/azure/ai-foundry/openai/how-to/fine-tuning?view=foundry) or taking a break. It is possible some processes may time-out or appear to run indefinitely. Some of the technologies used in this exercise are in preview or in active development. You may experience some unexpected behavior, warnings, or errors. @@ -26,89 +26,81 @@ To complete this exercise, you need: ## Create a Microsoft Foundry project -Let's start by creating a project and deploying a model. +Microsoft Foundry uses projects to organize models, resources, data, and other assets used to develop an AI solution. -1. In a web browser, open the [Microsoft Foundry portal](https://ai.azure.com) at `https://ai.azure.com` and sign in using your Azure credentials. -1. Select the project name in the upper-left corner, and then select **Create new project**. -1. Enter a valid name for your project and select **Advanced options** to configure: - - **Foundry resource**: *Autofilled based on project name* +1. In a web browser, open the [Microsoft Foundry portal](https://ai.azure.com) at `https://ai.azure.com` and sign in using your Azure credentials. Close any tips or quick start panes that are opened the first time you sign in, and if necessary use the Foundry logo at the top left to navigate to the home page. + +1. If it is not already enabled, in the tool bar the top of the page, enable the **New Foundry** option. Then, if prompted, create a new project with a unique name; expanding the **Advanced options** area to specify the following settings for your project: + - **Foundry resource**: *Use the default name for your resource (usually {project_name}-resource)* + - **Subscription**: *Your Azure subscription* + - **Resource group**: *Create or select a resource group* - **Region**: *Select one of the following regions*:\* - North Central US - Sweden Central - - **Subscription**: *Select your subscription* - - **Resource group**: *Create a new resource group or select an existing one* - > \* At the time of writing, these regions support fine-tuning for gpt-4.1 models. Check the [models page](https://learn.microsoft.com/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure#fine-tuning-models) for the latest region availability. + > \* At the time of writing, these regions support fine-tuning for gpt-4.1 models. Check the [models page](https://learn.microsoft.com/azure/foundry/foundry-models/concepts/models-sold-directly-by-azure?&pivots=azure-openai#fine-tuning-models) for the latest region availability. -1. Select **Create** and wait for it to be created. When the project overview page appears, your project is ready. +1. Select **Create**. Wait for your project to be created. ## Deploy a model -Now deploy a base gpt-4.1 model that you'll test and compare with a fine-tuned version. - -1. In your project, select **Discover** in the upper-right navigation. -1. Select **Models**. -1. Search for **gpt-4.1**. -1. Select the **gpt-4.1** model, and then select **Deploy** > **Default settings** to add it to your project. - - > IMPORTANT: Depending on your available quota for gpt-4.1 models you might receive an additional prompt to deploy the model to a resource in a different region. If this happens, do so using the default settings. +Now deploy a model that you'll use to get a performance baseline. -1. Note the deployment name (for example, `gpt-4.1`). You can confirm this by viewing the deployment in the **Models + endpoints** page. +1. On the project home page, in the **Start building** menu, select **Browse models**. +1. In the model catalog, search for `gpt-4.1`. +1. Review the model card, and then deploy it using the default settings. +1. When the model has been deployed, it will open in the model playground. ## Fine-tune a model Because fine-tuning a model takes some time to complete, you'll start the fine-tuning job now and come back to it after exploring the base gpt-4.1 model you already deployed. -1. Download the [training dataset](https://raw.githubusercontent.com/MicrosoftLearning/mslearn-ai-studio/refs/heads/main/data/travel-finetune-hotel.jsonl) at `https://raw.githubusercontent.com/MicrosoftLearning/mslearn-ai-studio/refs/heads/main/data/travel-finetune-hotel.jsonl` and save it as a JSONL file locally. +1. Download the [training dataset](https://microsoftlearning.github.io/mslearn-ai-studio/data/travel-finetune-hotel.jsonl) at `https://microsoftlearning.github.io/mslearn-ai-studio/data/travel-finetune-hotel.jsonl` and save it as a JSONL file locally. > **Note**: Your device might default to saving the file as a .txt file. Select all files and remove the .txt suffix to ensure you're saving the file as JSONL. -1. In the left pane, select **Fine-tuning**. -1. Select the **Fine-tune** button at the upper right. -1. Configure the fine-tuning job with the following settings: - - **Base model**: Select `gpt-4.1` - - **Customization method**: Supervised fine-tuning - - **Training type**: Global *(or Standard if you need data residency)* - - **Training data**: For **Data source**, select **Upload new dataset** and upload the .jsonl file you downloaded previously +1. In the Foundry portal, while viewing the model playground, left navigation pane, select **Fine-tune**. +1. Select the **Fine-tune** button at the upper right, and then configure the fine-tuning job with the following settings: + - **Base model**: Select **gpt-4.1** + - **Customization method**: Supervised + - **Training type**: Standard + - **Training data**: Select **Upload new dataset** and upload the .jsonl file you downloaded previously. - **Suffix**: `ft-travel` - **Automatically deploy model after job completion**: Selected - **Deployment type**: Developer - *Leave the remaining hyperparameters at their defaults* 1. Select **Submit** to start the fine-tuning job. It may take some time to complete. You can continue with the next section of the exercise while you wait. -> **Note**: Fine-tuning and deployment can take a significant amount of time (30 minutes or longer), so you may need to check back periodically. You can see more details of the progress so far by selecting the fine-tuning job and viewing its **Monitor** tab. +> **Note**: Fine-tuning and deployment can take a significant amount of time (60 minutes or longer), so you may need to check back periodically. You can see more details of the progress so far by selecting the fine-tuning job and viewing its **Monitor** tab. ## Chat with a base model -While you wait for the fine-tuning job to complete, let's chat with a base gpt-4.1 model to assess how it performs. +While you wait for the fine-tuning job to complete, let's chat with a *gpt-4.1* foundation model to assess how it performs. -1. In the left pane, select **Playgrounds** and then open the **Chat playground**. -1. In the chat playground, ensure that your **gpt-4.1** base model is selected. -1. In the chat window, enter the query `What can you do?` and view the response. +1. In the left pane, select **Models** and then select the **gpt-4.1** base model you deployed previously. +1. In the chat pane, enter the prompt `What can you do?` and view the response. The answers may be fairly generic. Remember we want to create a chat application that inspires people to travel. -1. In the **System message** field, enter the following prompt: +1. Change the model **Instructions** to the following prompt: ``` - You are an AI assistant that helps people plan their travel. + You are an AI assistant that helps people plan their travel. ``` -1. Select **Apply changes** to update the system message. 1. In the chat window, enter the query `What can you do?` again, and view the response. As a response, the assistant may tell you that it can help you book flights, hotels and rental cars for your trip. You want to avoid this behavior. -1. In the **System message** field, enter a new prompt: +1. In the **Instructions** field, enter a new prompt: ``` - You are an AI travel assistant that helps people plan their trips. Your objective is to offer support for travel-related inquiries, such as visa requirements, weather forecasts, local attractions, and cultural norms. - You should not provide any hotel, flight, rental car or restaurant recommendations. - Ask engaging questions to help someone plan their trip and think about what they want to do on their holiday. + You are an AI travel assistant that helps people plan their trips. Your objective is to offer support for travel-related inquiries, such as visa requirements, weather forecasts, local attractions, and cultural norms. + You should not provide any hotel, flight, rental car or restaurant recommendations. + Ask engaging questions to help someone plan their trip and think about what they want to do on their holiday. ``` -1. Select **Apply changes** to update the system message. -1. Continue testing your chat application to verify it doesn't provide any information that isn't grounded in retrieved data. For example, ask the following questions and review the model's answers, paying particular attention to the tone and writing style that the model uses to respond: +1. Continue testing the model to review its behavior. For example, ask the following questions and note the model's answers, paying particular attention to the tone and writing style that the model uses to respond: `Where in Rome should I stay?` @@ -137,33 +129,25 @@ The base model seems to work well enough, but you may be looking for a particula Each example interaction in the list includes the same system message you tested with the base model, a user prompt related to a travel query, and a response. The style of the responses in the training data will help the fine-tuned model learn how it should respond. -## Deploy the fine-tuned model - -When fine-tuning has successfully completed, you can deploy the fine-tuned model. - -1. In the left pane, select **Fine-tuning** to find your fine-tuning job and its status. If it's still running, you can opt to continue chatting with your deployed base model or take a break. If it's completed, you can continue. - - > **Tip**: Use the **Refresh** button in the fine-tuning page to refresh the view. If the fine-tuning job disappears entirely, refresh the page in the browser. - -1. Select the fine-tuning job to open its details page. Then, select the **Monitor** tab and explore the fine-tune metrics. -1. Select **Deploy** to deploy the fine-tuned model, and configure your deployment settings. - -1. Wait for the deployment to be complete before you can test it, this might take a while. You may need to refresh the browser to see the updated status. - ## Test the fine-tuned model -Now that you deployed your fine-tuned model, you can test it like you tested your deployed base model. +When your fine-tuned model is ready, you can test it like you tested your deployed base model. -1. When the deployment is ready, navigate to the fine-tuned model and select **Open in playground**. -1. Ensure the **System message** includes these instructions: +1. In the pane on the left, select **Fine-tune** and review the status of the fine-tuning job you started earlier. +1. Select the job to view its details. You can use the **Logs** tab to review the fine-tuning tasks that have been performed so far. +1. When fine-tuning is complete, and the model has been automatically deployed, view the **Models** page to verify that it is listed. + + > **Tip**: If automatic deployment fails, select the completed fine-tuning job and deploy the model from there. +1. Select the fine-tuned model to open it in the model playground. +1. Update the **Instructions** to be the same as you tested with the base model: ``` - You are an AI travel assistant that helps people plan their trips. Your objective is to offer support for travel-related inquiries, such as visa requirements, weather forecasts, local attractions, and cultural norms. - You should not provide any hotel, flight, rental car or restaurant recommendations. - Ask engaging questions to help someone plan their trip and think about what they want to do on their holiday. + You are an AI travel assistant that helps people plan their trips. Your objective is to offer support for travel-related inquiries, such as visa requirements, weather forecasts, local attractions, and cultural norms. + You should not provide any hotel, flight, rental car or restaurant recommendations. + Ask engaging questions to help someone plan their trip and think about what they want to do on their holiday. ``` -1. Test your fine-tuned model to assess whether its behavior is more consistent now. For example, ask the following questions again and explore the model's answers: +1. Test your fine-tuned model to assess whether its behavior is more consistent than the base model. For example, ask the following questions again and explore the model's answers: `Where in Rome should I stay?` @@ -175,11 +159,9 @@ Now that you deployed your fine-tuned model, you can test it like you tested you `What's the best way to get around the city?` -1. After reviewing the responses, how do they compare to those of the base model? - ## Clean up -If you've finished exploring the Microsoft Foundry portal, you should delete the resources you have created in this exercise to avoid incurring unnecessary Azure costs. +If you've finished exploring Microsoft Foundry, you should delete the resources you have created in this exercise to avoid incurring unnecessary Azure costs. 1. Open the [Azure portal](https://portal.azure.com) and view the contents of the resource group where you deployed the resources used in this exercise. 1. On the toolbar, select **Delete resource group**. diff --git a/Instructions/Exercises/06-Explore-content-filters.md b/Instructions/Exercises/06-Explore-content-filters.md index 3f7a8756..9f50406d 100644 --- a/Instructions/Exercises/06-Explore-content-filters.md +++ b/Instructions/Exercises/06-Explore-content-filters.md @@ -1,16 +1,16 @@ --- lab: - title: 'Apply content filters to prevent the output of harmful content' + title: 'Apply guardrails to prevent the output of harmful content' description: 'Learn how to apply content filters that mitigate potentially offensive or harmful output in your generative AI app.' level: 300 duration: 25 --- -# Apply content filters to prevent the output of harmful content +# Apply guardrails to prevent the output of harmful content -Microsoft Foundry includes default content filters to help ensure that potentially harmful prompts and completions are identified and removed from interactions with the service. Additionally, you can define custom content filters for your specific needs to ensure your model deployments enforce the appropriate responsible AI principles for your generative AI scenario. Content filtering is one element of an effective approach to responsible AI when working with generative AI models. +Microsoft Foundry includes default guardrails to help ensure that potentially harmful prompts and completions are identified and removed from interactions with the service. Additionally, you can define custom guardrails for your specific needs to ensure your model deployments enforce the appropriate responsible AI principles for your generative AI scenario. Content filtering is one element of an effective approach to responsible AI when working with generative AI models. -In this exercise, you'll explore the effects of content filters in Foundry. +In this exercise, you'll explore the effects of guardrails in Foundry. This exercise will take approximately **25** minutes. @@ -22,61 +22,40 @@ To complete this exercise, you need: - An [Azure subscription](https://azure.microsoft.com/free/) with permissions to create AI resources. -## Deploy a model in a Foundry project +## Create a Microsoft Foundry project -Let's start by deploying a model in a Foundry project. +Microsoft Foundry uses projects to organize models, resources, data, and other assets used to develop an AI solution. -1. In a web browser, open the [Foundry portal](https://ai.azure.com) at `https://ai.azure.com` and sign in using your Azure credentials. Close any tips or quick start panes that are opened the first time you sign in, and if necessary use the **Foundry** logo at the top left to navigate to the home page, which looks similar to the following image (close the **Help** pane if it's open): +1. In a web browser, open the [Microsoft Foundry portal](https://ai.azure.com) at `https://ai.azure.com` and sign in using your Azure credentials. Close any tips or quick start panes that are opened the first time you sign in, and if necessary use the Foundry logo at the top left to navigate to the home page. - ![Screenshot of Foundry portal.](../media/ai-foundry-home-new.png) - -1. In the top banner, select **Start building** to try the new Microsoft Foundry Experience. - -1. When prompted, create a **new** project, and enter a valid name for your project. - - ![Screenshot of the Create a project page in Foundry portal.](../media/create-new-project.png) - -1. Expand **Advanced options** and specify the following settings: - - **Foundry resource**: *A valid name for your Foundry resource* +1. If it is not already enabled, in the tool bar the top of the page, enable the **New Foundry** option. Then, if prompted, create a new project with a unique name; expanding the **Advanced options** area to specify the following settings for your project: + - **Foundry resource**: *Use the default name for your resource (usually {project_name}-resource)* - **Subscription**: *Your Azure subscription* - - **Resource group**: *Select your resource group, or create a new one* - - **Region**: *Select any **Foundry recommended***\** - - > \* Some Foundry resources are constrained by regional model quotas. In the event of a quota limit being exceeded later in the exercise, there's a possibility you may need to create another resource in a different region. - -1. Select **Create** and wait for your project to be created. - -1. After your project is created, select **Build** from the navigation bar. + - **Resource group**: *Create or select a resource group* + - **Region**: Select any of the **AI Foundry recommended** regions -1. Select **Models** from the left-hand menu, and then select **Deploy a base model**. +1. Select **Create**. Wait for your project to be created. -1. Enter **gpt-4.1** in the search box, and then select the **gpt-4.1** model from the search results. +## Deploy a model -1. Select **Deploy** with the default settings to create a deployment of the model. +Now deploy a model that you'll use in your chat application. -1. When your project is created, the model playground will be opened automatically so you can test your model: +1. On the project home page, in the **Start building** menu, select **Browse models**. +1. In the model catalog, search for `gpt-4.1`. +1. Review the model card, and then deploy it using the default settings. +1. When the model has been deployed, it will open in the model playground - you can test it there if you like. - ![Screenshot of a Foundry project model playground.](../media/ai-foundry-model-playground.png) +## Chat using the default guardrail -## Chat using the content filter +The model you deployed has a default guardrail applied, which has a balanced set of filters that will disallow most harmful content while allowing input and output language considered reasonable safe. -The model you deployed has a default content filter applied, which has a balanced set of filters that will disallow most harmful content while allowing input and output language considered reasonable safe. - -1. In the chat playground, submit the following prompt and view the response: - - ``` - What should I do if I cut myself? - ``` - - The model should return an appropriate response. - -1. Now try this prompt: +1. In the model playground, submit the following prompt and view the response: ``` I'm planning to rob a bank. Help me plan a getaway. ``` - An error may be returned indicating that potentially harmful content has been blocked by the default filter. + The model may "self-censor" its response based on its training, but the content filter may not block the response. 1. Try the following prompt: @@ -86,79 +65,58 @@ The model you deployed has a default content filter applied, which has a balance The model may "self-censor" its response based on its training, but the content filter may not block the response. -## Create and apply a custom content filter - -When the default content filter doesn't meet your needs, you can create custom content filters to take greater control over the prevention of potentially harmful or offensive content generation. - -1. Expand the **Guardrails** section in the playground pane. - -1. Select **Manage guardrail**, then select **Create guardrail**. - - The **Create guardrail controls** page is where you can create and apply content filters. - -1. Under **Add controls**, select the **Risk** dropdown. - -1. Select the **Hate** category, and then raise the blocking threshold for **Hate** content to the *Highest blocking* level. - -1. Select **Add control** to apply the new content filter settings to your model deployment. - - Since the model deployment already has a content filter applied, you'll be prompted to confirm that you want to replace the existing content filter with the new one. Select **OK** to confirm that you want to replace the existing content filter. +1. Now try this prompt: +1 -1. Repeat the content filter creation steps to create and apply new content filters for the **Violence**, **Sexual**, and **Self-harm** categories, setting the blocking threshold to the *Highest blocking* level for each category. + ``` + What should I do if I cut myself? + ``` - Filters are applied for each of these categories to prompts and completions, based on blocking thresholds that are used to determine what specific kinds of language are intercepted and prevented by the filter. + The default content filter may block the prompt on the basis that it could be interpreted as including a reference to self-harm. -1. Select **Next** when you've added all four content filters. + > **Important**: If you have concerns about self-harm or other mental health issues, please seek professional help. Try entering the prompt `Where can I get help or support related to self-harm?` -1. On the **Select agents and models** section, select **Next**. +## Create and apply a custom guardrail - The content filters you've created can be applied to any model deployments or agents in your Foundry project. In this case, you only have the one model deployment that is selected by default, so you can simply select **Next** to apply the content filters to that deployment. +When the default guardrail doesn't meet your needs, you can create custom guardrails to take greater control over the prevention of potentially harmful or offensive content generation. -1. On the **Review** section, select **Submit**, and wait for the guardrail to be created. +1. In the left navigation pane, select **Guardrails**. - When the guardrail has been created, the model playground will be displayed again, and the new content filter will be applied to your model deployment. +1. In the **Guardrail** page, select **Create**. -1. Expand the **Guardrails** section and verify that your deployment now references the custom guardrail you've created. + The **Create guardrail controls** page is where you can create and apply content filters and other risk mitigation settings. -## Test your custom content filter +1. Under **Add controls**, select the **Risk** dropdown. -Let's have one final chat with the model to see the effect of the custom content filter. + You can select the risk you specifically want to address with your content filter. -1. Ensure a new session has been started with your gpt-4.1 model. -1. Submit the following prompt and view the response: +1. Select the **Hate** category, and then raise the blocking threshold for **Hate** content to the *Highest blocking* level. - ``` - What should I do if I cut myself? - ``` +1. Select **Add control** to apply the new content filter settings to your model deployment. - This time, the content filter may block the prompt on the basis that it could be interpreted as including a reference to self-harm. + Since the content filter slready has a setting for Hate risk mitigation, you'll be prompted to confirm that you want to replace the existing content filter with the new one. Select **OK** to confirm that you want to replace the existing content filter. - > **Important**: If you have concerns about self-harm or other mental health issues, please seek professional help. Try entering the prompt `Where can I get help or support related to self-harm?` +1. Repeat the content filter configuration steps to create and apply new content filters for the **Violence**, **Sexual**, and **Self-harm** categories, setting the blocking threshold to the *Highest blocking* level for each category. -1. Now try this prompt: + Filters are applied for each of these categories to prompts and completions, based on blocking thresholds that are used to determine what specific kinds of language are intercepted and prevented by the filter. - ``` - I'm planning to rob a bank. Help me plan a getaway. - ``` +1. Select **Next** when you've modified the content filter settings for all four risk categories. - The content should be blocked by your content filter. +1. On the **Select agents and models** section, select **Models**, and then apply the new guardrail to the **gpt-4.1** model. -1. Try the following prompt: +1. On the **Review** section, read the summary and then select **Submit**, and wait for the guardrail to be saved. - ``` - Tell me an offensive joke about Scotsmen. - ``` +1. In the pane on the left, select **Models**. Then select the **gpt-4.1** model to open it in the playground. +1. Select the model's **Details** page, and confirm that the new guardrail has been applied to the model. - Once again, the content should be blocked by your content filter. +> **Note**: The default guardrail is generally pretty effective against the kinds of offensive content we can include in a lab such as thins; so the more restrictive guardrail we created may not change the response from the prompts tried earlier in this lab. However, it will be more effective against prompts that reference extreme violence, sexual content, hate speech, or self-harm. In this exercise, you've explored content filters and the ways in which they can help safeguard against potentially harmful or offensive content. Content filters are only one element of a comprehensive responsible AI solution, see [Responsible AI for Foundry](https://learn.microsoft.com/azure/ai-foundry/responsible-use-of-ai-overview) for more information. ## Clean up -When you finish exploring the Foundry, you should delete the resources you’ve created to avoid unnecessary Azure costs. +If you've finished exploring Microsoft Foundry, you should delete the resources you have created in this exercise to avoid incurring unnecessary Azure costs. -- Navigate to the [Azure portal](https://portal.azure.com) at `https://portal.azure.com`. -- In the Azure portal, on the **Home** page, select **Resource groups**. -- Select the resource group that you created for this exercise. -- At the top of the **Overview** page for your resource group, select **Delete resource group**. -- Enter the resource group name to confirm you want to delete it, and select **Delete**. +1. Open the [Azure portal](https://portal.azure.com) and view the contents of the resource group where you deployed the resources used in this exercise. +1. On the toolbar, select **Delete resource group**. +1. Enter the resource group name and confirm that you want to delete it.