Skip to content

Commit 0147d2f

Browse files
committed
Add documentation for adding examples to the vector database
Signed-off-by: Acuspeedster <[email protected]>
1 parent c2f78a2 commit 0147d2f

File tree

1 file changed

+72
-0
lines changed

1 file changed

+72
-0
lines changed

README.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -436,6 +436,75 @@ Process repeats until successful or max attempts reached
436436

437437
---
438438

439+
## 📊 Adding to the Vector Database
440+
441+
The system uses vector embeddings to find similar projects and error examples, which helps improve code generation quality. Here's how to add your own examples:
442+
443+
### Method 1: Using Python API Directly
444+
445+
```python
446+
from app.llm_client import LlamaEdgeClient
447+
from app.vector_store import QdrantStore
448+
449+
# Initialize the components
450+
llm_client = LlamaEdgeClient()
451+
vector_store = QdrantStore()
452+
453+
# Ensure collections exist
454+
vector_store.create_collection("project_examples") # or "error_examples"
455+
456+
# 1. Prepare your data
457+
project_data = {
458+
"query": "A command-line calculator in Rust",
459+
"example": "Your full project example with code here..."
460+
}
461+
462+
# 2. Get embedding for the query text
463+
embedding = llm_client.get_embeddings([project_data["query"]])[0]
464+
465+
# 3. Add to vector database
466+
vector_store.add_item(
467+
collection_name="project_examples",
468+
vector=embedding,
469+
item=project_data
470+
)
471+
```
472+
473+
### Method 2: Adding Multiple Examples from JSON Files
474+
Place JSON files in the appropriate directories:
475+
476+
Project examples: ```project_examples```
477+
Error examples: ```error_examples```
478+
Format for project examples:
479+
```
480+
{
481+
"query": "Description of the project",
482+
"example": "Full example code or description"
483+
}
484+
```
485+
Format for error examples:
486+
```
487+
{
488+
"error": "Rust compiler error message",
489+
"solution": "How to fix the error",
490+
"context": "Additional explanation (optional)"
491+
}
492+
```
493+
Then run the data loading script:
494+
```
495+
python -c "from app.load_data import load_project_examples, load_error_examples; load_project_examples(); load_error_examples()"
496+
```
497+
498+
### Method 3: Using the ```parse_and_save_qna.py``` Script
499+
For bulk importing from a Q&A format text file:
500+
501+
Place your Q&A pairs in a text file with format similar to ```QnA_pair.txt```
502+
Modify the ```parse_and_save_qna.py``` script to point to your file
503+
Run the script:
504+
```
505+
python parse_and_save_qna.py
506+
```
507+
439508
## 🤝 Contributing
440509
Contributions are welcome! This project uses the Developer Certificate of Origin (DCO) to certify that contributors have the right to submit their code. Follow these steps:
441510

@@ -458,3 +527,6 @@ This certifies that you wrote or have the right to submit the code you're contri
458527
## 📜 License
459528
Licensed under [GPLv3](https://www.gnu.org/licenses/gpl-3.0.en.html).
460529

530+
531+
532+

0 commit comments

Comments
 (0)