Feature/ifbench #113
Merged
Feature/ifbench #113
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 2 commits
July 12, 2025 17:40
BlankCheng
reviewed
Jul 17, 2025
| """Split the fixed IFBench data into 90% train and 10% test.""" | ||
|
|
||
| # Input file (fixed data) | ||
| input_file = "/mnt/sharefs/users/jianshu.she/ood__ifbench_95.1k_fixed.parquet" |
There was a problem hiding this comment.
There are absolute paths here. Is this file necessary for production?
| """Split the fixed IFBench data into 90% train and 10% test.""" | ||
|
|
||
| # Input file (fixed data) | ||
| input_file = "/mnt/sharefs/users/jianshu.she/ood__ifbench_95.1k_fixed.parquet" |
|
|
||
| # Fix the data | ||
| print("\n=== Fixing data ===") | ||
| fix_ifbench_data(original_file, "/mnt/sharefs/users/jianshu.she/ood__ifbench_95.1k_fixed.parquet") |
|
Hi @Jianshu1only , this looks awesome! One issue is some files are not directly used in RL training, which don't need to be included, e.g., |
|
Hi @BlankCheng , I’ve deleted all unnecessary files in the IFbench folder. Please review the PR again when you have a chance. Thanks! |
|
Thanks @Jianshu1only! I delete the bash files as we only keep the template there. Let's merge. |
BlankCheng
approved these changes
Jul 25, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Checklist Before Starting
What does this PR do?
Create IFBench dataset verification function and data preprocess code
High-Level Design
Add ifbench in reward_score
Specific Changes
Add ifbench verification function in reward_score, add ood_ifbench in util, add data preprocess function
API
NA
Usage Example
NA
Test
python verl/utils/reward_score/ifbench/test_ifbench.py
sbatch verl/scripts/train/ifbench_test.sh
Additional Info.
NA
Checklist Before Submitting
[BREAKING]to the PR title if it breaks any API.