Add Function Calling Fine-tuning LLMs on xLAM Dataset notebook #321

behroozazarkhalili · 2025-08-01T19:41:44Z

Summary

This notebook demonstrates how to fine-tune language models for function calling capabilities using the xLAM dataset from Salesforce and QLoRA technique.

Key Features

Universal Model Support: Works with Llama, Qwen, Mistral, Gemma, Phi, and more
Memory Efficient: QLoRA training on consumer GPUs (16-24GB VRAM)
Automatic Configuration: Smart token detection and model setup
Production Ready: Comprehensive documentation and error handling
Complete Pipeline: From training to Hugging Face Hub deployment

Technical Details

Uses QLoRA (Quantized Low-Rank Adaptation) for efficient fine-tuning
Supports multiple model architectures with automatic pad token detection
Includes comprehensive testing and evaluation functions
Modular design with proper type hints and documentation

Contribution Guidelines Compliance

Notebook filename in lowercase: function_calling_fine_tuning_llms_on_xlam.ipynb
Author information added with GitHub profile link
Added to _toctree.yml in LLM Recipes section
Added to index.md in Latest notebooks section
Non-informative outputs removed from pip install cells
No empty code cells
Comprehensive documentation and markdown explanations

Test Plan

Notebook structure and organization verified
All cells contain proper documentation
Code quality and error handling implemented
Ready for community use and contribution

✅ All contribution guidelines followed according to the README

@merveenoyan , @stevhliu

review-notebook-app · 2025-08-01T19:41:49Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

stevhliu · 2025-08-04T16:43:24Z

Thanks for your contribution!

My first impression is that it is very code-heavy without really any supporting text that explains what is happening and the rationale behind certain decisions. Breaking up these code blocks will make it easier for users to digest.

Also pinging @sergiopaniego, our recipe chef, for any other additional suggestions ❤️

behroozazarkhalili · 2025-08-04T17:57:13Z

Thanks for your contribution!

My first impression is that it is very code-heavy without really any supporting text that explains what is happening and the rationale behind certain decisions. Breaking up these code blocks will make it easier for users to digest.

Also pinging @sergiopaniego, our recipe chef, for any other additional suggestions ❤️

Hi @stevhliu, Thank you for the feedback.
Are you asking if I should include an explanation for each step and the reasons for selecting specific parameters, etc.? I would be grateful if you could add any more clarification.

stevhliu · 2025-08-05T15:26:17Z

Sorry I wasn't clear!

Yes, a general explanation for each step would be nice. You don't have to go too in-depth explaining why you selected specific parameters (unless its important), but the user should be able to read a paragraph and have a good idea of what is happening at a step.

behroozazarkhalili · 2025-08-05T17:55:25Z

Sorry I wasn't clear!

Yes, a general explanation for each step would be nice. You don't have to go too in-depth explaining why you selected specific parameters (unless its important), but the user should be able to read a paragraph and have a good idea of what is happening at a step.

No worries. I'll make the updates based on your comments and submit the pull request soon. :)

notebooks/en/function_calling_fine_tuning_llms_on_xlam.ipynb

sergiopaniego

Thanks for the effort!! 😃 Following the same ideas suggested by @stevhliu and similar to #319:
Code blocks should be divided into smaller sections and explained. We don’t need an in-depth breakdown of every parameter, but rather an explanation of the problem we’re trying to solve and why each function or block of code is necessary.

A recipe should be aimed at readers who want to learn more about a specific technique or package, so the focus should be more educational rather than simply presenting a complete project with a lot of code. You can also reference other recipes to provide additional context and insights.

- Dense code blocks that need breaking up - Missing explanatory text between sections - Large import block needs splitting - ModelConfig/TrainingConfig needs simplification - Indentation issues in process_xlam_sample function - Need to remove <small> tags and subsections

- Change max_seq_length to max_length parameter in SFTConfig instantiation - Resolves TypeError when running train_qlora_model function - Maintains compatibility with TRL library API requirements

notebooks/en/function_calling_fine_tuning_llms_on_xlam.ipynb

sergiopaniego

Thanks for the iteration! We'd need to update the toctree and index with the new notebook too

…tputs - Add detailed explanations for dataset processing functions (process_xlam_sample, load_and_process_xlam_dataset, preview_dataset_sample) - Document rationale for Llama 3-8B-Instruct model selection with performance/resource balance reasoning - Include execution outputs showing successful environment setup and model testing - Add .env to gitignore for environment variable security - Update package installation commands to use uv pip for faster dependency management - Demonstrate complete workflow from setup through testing with comprehensive function calling examples The notebook now provides clearer guidance on the xLAM dataset processing pipeline and model selection rationale while maintaining full functionality for QLoRA fine-tuning.

sergiopaniego

Thanks for the update!

Could you update the toctree and index files?

- Add function calling fine-tuning notebook to _toctree.yml under LLM Recipes section - Feature notebook in index.md latest notebooks section for discoverability - Enables users to find the xLAM dataset function calling tutorial through cookbook navigation The notebook is now properly integrated into the cookbook structure and discoverable through standard navigation paths.

behroozazarkhalili · 2025-09-09T15:57:22Z

Thanks for the update!

Could you update the toctree and index files?

Done. :)

notebooks/en/index.md

Clean up latest notebooks list to highlight most recent additions while maintaining focus on current relevant content.

Ensure both function calling notebook and existing T5 PEFT notebook are properly listed in latest notebooks section to maintain compatibility with main branch while adding new content.

Remove redundant entries and improve formatting consistency in the latest notebooks list.

Include both function calling notebook and T5 PEFT notebook in the latest notebooks section for complete coverage of recent additions.

Keep function calling notebook and existing structure for clean merge compatibility.

Maintain current version without T5 PEFT entry as requested.

HuggingFaceDocBuilderDev · 2025-09-10T17:59:54Z

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

sergiopaniego

Thanks! 🚀

sergiopaniego reviewed Aug 11, 2025

View reviewed changes

behroozazarkhalili added 2 commits August 28, 2025 11:38

Fix SFTConfig parameter error in function calling notebook

a74ce9a

- Change max_seq_length to max_length parameter in SFTConfig instantiation - Resolves TypeError when running train_qlora_model function - Maintains compatibility with TRL library API requirements

behroozazarkhalili force-pushed the add-function-calling-xlam-notebook branch from 26ee1bc to a74ce9a Compare August 28, 2025 19:27

sergiopaniego reviewed Sep 2, 2025

View reviewed changes

sergiopaniego reviewed Sep 9, 2025

View reviewed changes

sergiopaniego reviewed Sep 10, 2025

View reviewed changes

notebooks/en/index.md Show resolved Hide resolved

behroozazarkhalili added 6 commits September 10, 2025 09:01

Update latest notebooks section in index

5336485

Clean up latest notebooks list to highlight most recent additions while maintaining focus on current relevant content.

Resolve merge conflicts in index.md

45dcbae

Ensure both function calling notebook and existing T5 PEFT notebook are properly listed in latest notebooks section to maintain compatibility with main branch while adding new content.

Clean up latest notebooks section formatting

10b8c10

Remove redundant entries and improve formatting consistency in the latest notebooks list.

Add missing T5 PEFT notebook to latest notebooks list

43a517b

Include both function calling notebook and T5 PEFT notebook in the latest notebooks section for complete coverage of recent additions.

Resolve index.md conflicts maintaining both notebooks

6d62370

Keep function calling notebook and existing structure for clean merge compatibility.

Resolve conflict keeping function calling notebook only

186b679

Maintain current version without T5 PEFT entry as requested.

sergiopaniego approved these changes Sep 11, 2025

View reviewed changes

sergiopaniego merged commit 8af62ca into huggingface:main Sep 11, 2025
1 check passed

Add Function Calling Fine-tuning LLMs on xLAM Dataset notebook #321

Add Function Calling Fine-tuning LLMs on xLAM Dataset notebook #321

Uh oh!

Conversation

behroozazarkhalili commented Aug 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Key Features

Technical Details

Contribution Guidelines Compliance

Test Plan

Uh oh!

review-notebook-app bot commented Aug 1, 2025

Uh oh!

stevhliu commented Aug 4, 2025

Uh oh!

behroozazarkhalili commented Aug 4, 2025

Uh oh!

stevhliu commented Aug 5, 2025

Uh oh!

behroozazarkhalili commented Aug 5, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sergiopaniego left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sergiopaniego left a comment

Choose a reason for hiding this comment

Uh oh!

sergiopaniego left a comment

Choose a reason for hiding this comment

Uh oh!

behroozazarkhalili commented Sep 9, 2025

Uh oh!

Uh oh!

HuggingFaceDocBuilderDev commented Sep 10, 2025

Uh oh!

sergiopaniego left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

behroozazarkhalili commented Aug 1, 2025 •

edited

Loading