Skip to content

Conversation

Copy link

Copilot AI commented Aug 7, 2025

Created a comprehensive .github/copilot-instructions.md file to guide GitHub Copilot in working effectively with the rank_llm codebase. This addresses the need for structured guidance when working with this complex ML/IR repository that involves multiple backends, model types, and dependencies.

Key Features

Environment & Setup

  • Python 3.11+ and JDK 21 requirements with macOS incompatibility warnings
  • Network timeout workarounds for pip installations (common issue in this environment)
  • PYTHONPATH configuration for development without full installation

Development Workflow

  • Pre-commit hook setup with black, isort, and flake8 formatting
  • Unit testing commands with proper timeout values (tests run in ~0.08s)
  • Regression testing guidance for the 5-test suite with 60+ minute timeouts

Application Usage Examples

  • Complete CLI examples for different model types (RankZephyr, MonoT5, DuoT5, FirstMistral)
  • Proper parameter configurations for listwise, pointwise, and pairwise rerankers
  • Demo script locations and usage patterns

Critical Timing Documentation

  • Explicit "NEVER CANCEL" warnings for long-running operations
  • Measured timeouts: pip install (15+ min), tests (2+ min), regression (60+ min)
  • Performance expectations for first-time setup vs. regular development

Validation Requirements

  • Mandatory test execution after changes
  • Import verification steps
  • Linting requirements before commits

Troubleshooting & Navigation

  • Common error patterns and diagnostic commands
  • Key directory structure and important files (247-line main CLI script)
  • Workflow patterns for adding rerankers, retrieval methods, and tests

The instructions emphasize reliability and proper timeout handling, which is critical for this codebase that involves downloading large ML models and running compute-intensive inference operations.

Fixes #295.


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI changed the title [WIP] ✨Set up Copilot instructions Add comprehensive GitHub Copilot instructions for rank_llm development Aug 7, 2025
Copilot AI requested a review from ronakice August 7, 2025 22:52
Copilot finished work on behalf of ronakice August 7, 2025 22:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

✨Set up Copilot instructions

2 participants