-
Notifications
You must be signed in to change notification settings - Fork 307
ChatQnA Example with OpenAI-Compatible Endpoint #2091
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Ed Lee <[email protected]>
…w null json. Also improved exception handling and logging Signed-off-by: Ed Lee <[email protected]>
…yaml Co-authored-by: Copilot <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Integrate MultimodalQnA set_env to ut scripts. Add README.md for UT scripts. Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
…nt (opea-project#1996) Signed-off-by: Mustafa <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Yi Yao <[email protected]> Co-authored-by: Copilot <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
…2030) Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
…d HybridRAG (opea-project#2037) Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
…archQnA and Translation (opea-project#2038) update secrets token name for ProductivitySuite, RerankFinetuning, SearchQnA and Translation Fix shellcheck issue Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
…rkflowExecAgent (opea-project#2039) update secrets token name for InstructionTuning, MultimodalQnA and WorkflowExecAgent Fix shellcheck issue Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Yongbozzz <[email protected]> Signed-off-by: Ed Lee <[email protected]>
…pea-project#1981) Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: ZePan110 <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Yongbozzz <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Wang, Xigui <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: chensuyue <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Wang, Xigui <[email protected]> Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Ed Lee <[email protected]>
…es into chatqna_w_endpoints
Dependency Review✅ No vulnerabilities or license issues found.Scanned FilesNone |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This pull request introduces an OpenAI-compatible endpoint for ChatQnA, updates the deployment documentation, and includes improvements in error handling and logging.
- Added new Docker Compose file (compose_endpoint_openai.yaml) to support OpenAI-like endpoints.
- Updated README files for clearer deployment instructions and configuration details.
- Fixed the align_generator function in chatqna.py to better handle and filter null content chunks.
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.
File | Description |
---|---|
CodeGen/docker_compose/intel/cpu/xeon/README.md | Updated docker compose command and environment variable documentation; note a markdown table formatting issue. |
ChatQnA/docker_compose/intel/cpu/xeon/compose_endpoint_openai.yaml | Added new compose file for OpenAI-compatible endpoint integration. |
ChatQnA/docker_compose/intel/cpu/xeon/README_endpoint_openai.md | New documentation with detailed instructions for deploying ChatQnA using the new endpoint. |
ChatQnA/chatqna.py | Improved logging and error handling in input/output alignment and generator functions. |
Comments suppressed due to low confidence (1)
CodeGen/docker_compose/intel/cpu/xeon/README.md:111
- The table row for LLM_ENDPOINT appears to be broken into two columns due to an unintended pipe character. Please merge the content into a single cell to ensure the URL displays correctly.
| `LLM_ENDPOINT` | Internal URL for the LLM serving endpoint (used by `codegen-llm-server`). Configured in `compose.yaml`. | `http://codegen-vllm | tgi-server:9000/v1/chat/completions` |
Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Ed Lee <[email protected]>
Signed-off-by: Ed Lee <[email protected]>
…es into chatqna_w_endpoints
for more information, see https://pre-commit.ci
Hi @yao531441 @letonghan if either of you can, I'm looking for one more reviewer please :) |
…erence:cpu-1.7 from 1.5 Signed-off-by: Ed Lee <[email protected]>
…es into chatqna_w_endpoints
Description
Allows ChatQnA to be used with thousands of OpenAI-like endpoints e.g. OpenRouter.ai, Hugging Face, Denvr, and improve the developer experience to use OPEA quickly even on low resource environments.
Key Changes Made:
Also:
Issues
N/A
Type of change
List the type of change like below. Please delete options that are not relevant.
Dependencies
N/A
Tests
OpenRouter.ai: anthropic/claude-3.7-sonnet
Denvr: meta-llama/Llama-3.1-70B-Instruct
Hugging Face Inference Endpoint: microsoft/phi-4