Intellectual Property RAG Pipeline

This repository contains a retrieval-augmented generation (RAG) prototype for answering questions about intellectual property (IP) and patent documents. The pipeline ingests PDF or plain-text sources, chunks and enriches them, embeds each chunk, stores the vectors in Weaviate, and exposes a lightweight CLI for retrieval + generation. Use this guide to set up the environment, ingest a corpus, and run queries end-to-end.

1. Prerequisites

• Python: 3.10 or newer is recommended.
• Weaviate: an accessible Weaviate instance (self-hosted Docker container or the Weaviate Cloud Service). The ingestion scripts expect the database to be available before indexing runs.
• GPU (optional): the embedding/generation models run faster on GPUs, but CPU-only environments work for small experiments.

2. Installation

1. Clone the repo and create a virtual environment:

   python -m venv .venv
   source .venv/bin/activate

2. Install the dependencies from either requirements.txt or the programmatic helper:

   pip install -r requirements.txt
   # or
   python requirements.py | pip install -r /dev/stdin

3 Start Weaviate (Docker Compose)

A docker-compose.yml file is provided for quick setup of a local Weaviate instance.

Requirements

• Docker installed and running
• Docker Compose plugin available (docker compose, not legacy docker-compose)

Verify your installation:

docker --version
docker-compose version

3. Configuration

Keys and constants live in config.py:

Variable	Default	Purpose
DATA_DIR	data/	Directory containing PDFs/TXT files to ingest.
WEAVIATE_URL	http://localhost:8080	Base URL of the Weaviate instance.
WEAVIATE_CLASS_NAME	PatentChunk	Target class name for stored chunks.
CHUNK_STRATEGY	fixed_tokens	Choose between fixed_tokens, markdown_sections, semantic_sentences, sentence_level.
CHUNK_SIZE, CHUNK_OVERLAP	1200, 200	Default character window and overlap for helper functions.
FIXED_TOKEN_CHUNK_SIZE, FIXED_TOKEN_CHUNK_OVERLAP	600, 80	Token window and overlap for the baseline chunker.
MARKDOWN_MAX_SECTION_CHARS	1600	Maximum size for any Markdown section before splitting.
SEMANTIC_CHUNK_TOKEN_SIZE, SEMANTIC_CHUNK_TOKEN_OVERLAP	420, 60	Budgets for semantic sentence grouping.
SEMANTIC_SIMILARITY_THRESHOLD	0.6	Split when consecutive sentences fall below this cosine similarity.
INGEST_BATCH_SIZE	128	Number of chunks processed per streaming batch (controls RAM).
TOP_K_RESULTS	8	Number of nearest neighbours to fetch per query.
EMBED_MODEL_NAME	mixedbread-ai/mxbai-embed-large-v1	Sentence-transformer used for embeddings.
GENERATION_MODEL_NAME	google/flan-t5-base	Model used to generate grounded answers.

Override any value via environment variables before running the scripts, for example:

export WEAVIATE_URL="https://localhost:portnumber"

4. Preparing the Corpus

1. Source files (pdf) are located in the data/ directory. You may create nested folders, however they will nonetheless be placed into the global extracted-data/ directory after processing.
2. Supported formats: .pdf and .txt. The ingestion utilities will normalise whitespace automatically.
3. See below for methods to directly download file from openAlex.

/ ! \ Failure on some files

Some files have names that contain specific characters that makes them undetectable by the pdf extraction pipeline. To handle extraction issues we implemented a blacklist to skip those files and prevent the process from crashing again (see BLACKLIST_PATTERNS in docling_preprocessing.py). Another option is to simply name these files after the worker thread (our initial method).

Retrieving files from openAlex

A substantial part of our data files comes from openAlex, which is an archive for scientific papers. Simply running openalex_retrieval.py will search for new files in PDF format and download them, up to 50 at once.

python openalex_retrieval.py

PDF Processing

We process PDF files using docling, which uses OCR. We directly export from it as text (Markdown). Below is an example usage, it is a long process so we recommend that you try smaller at the beginning. Max extract parameter can be omitted and will result in using 10 as default value (recommended for the first time running the project or for testing purposes).

The target files are located in the data/ directory, and after processing will be stored in extracted-data/. We have added a parameter extract-openalex which specifies that only files within data/openalex should be processed, and that they should be stored inside extracted-data/openalex/ specifically.

python main.py --only-extract

python main.py --only-extract --max-extract 50

For more informaion, see docling_preprocessing

5. Running the Ingestion Pipeline

Important: Ingestion is disabled by default.
You must explicitly pass --ingest for documents to be chunked, embedded, and indexed into Weaviate.

Full ingestion

To stream documents through the full pipeline (chunk → embed → index):

python main.py --ingest

This uses a streaming design so that only a small batch of chunks lives in memory at any time.

Key behaviours

When --ingest is enabled: Documents are read one at a time from EXTRACTED_DATA_DIR. Each document is chunked with overlap to provide contextual windows. Lightweight section tags (e.g. introduction, methods, results) are inferred. The embedding model is loaded once and reused across all chunk batches.

The Weaviate schema (PatentChunk by default) is created if it does not exist. Chunk metadata and vectors are indexed into Weaviate in batches controlled by INGEST_BATCH_SIZE or --batch-size.

Common variants

Extract + ingest in one run:

python main.py --only-extract --ingest

Fresh reindex (drop + recreate schema):

python main.py --force-reindex --ingest

Debug / development run:

python main.py --ingest --max-docs 2 --max-chunks 200

If --ingest is omitted, the pipeline will skip ingestion and no data will be written to Weaviate.

Chunking strategies

Three chunkers are available and can be selected with CHUNK_STRATEGY:

• fixed_tokens: simple sliding window in token space (default). Cheap and reliable when documents are unstructured or you just need a strong baseline.
• markdown_sections: honours Docling-style Markdown headings so chunks line up with sections like Introduction/Model/Results/Conclusion. Use when PDFs render with clear headings.
• semantic_sentences: groups sentences until semantic similarity drops or the token budget is reached, mimicking "semantic chunking" techniques for long, topic-dense papers.
• sentence_level: splits the text into individual sentences, treating each sentence as a self-contained chunk.

Example: CHUNK_STRATEGY=semantic_sentences python main.py --max-docs 100

There are additional arguments you can add :

python main.py \
  --ingest \
  --batch-size 16 \
  --max-docs 2 \
  --max-chunks 200

You can re-run the script whenever you add or update documents. Existing vectors in Weaviate are left untouched, so consider clearing the class if you want a full refresh.

6. Resource Usage & Memory Footprint

• RAM during ingestion: streaming keeps roughly batch_size * (chunk_size text + embedding) resident. With the defaults (128 chunks, 1200 characters each, ~768–1024 float32 embedding dimensions) this is comfortably under 10 MB of working memory on top of the model weights, so laptops no longer exhaust RAM.
• Model weights: the embedding model occupies ~1–1.5 GB of RAM once loaded. Ensure your machine has headroom for that plus the Python process itself.
• Scaling up: if you have plentiful RAM, increase INGEST_BATCH_SIZE (or pass --batch-size) for better throughput. If you're tight on memory, drop it to 32 or 64.

7. Querying the Corpus

For ad-hoc CLI queries use query.py:

python query.py

This will:

1. Load the same embedding model used during ingestion.
2. Preprocess and embed your question, then run a nearest-neighbour search in Weaviate.
3. Assemble a context window from the top-K chunks and print the text used.
4. Generate a grounded answer via the configured text-to-text model.

PS: You can change or add your queries in the query.file.

Inspect the console output to understand which chunks supported each answer. You can adjust TOP_K_RESULTS or MAX_CONTEXT_CHARS (in config.py) to trade off context size versus latency.

Running the UI interface

We provide UI that is accessible through the browser. It requires already having computed the ingestion (weaviate DB).

Inside is a chat with the LLM, please note that it only answers based on the latest question (we do not provide more context due to a bottleneck with context windows in the current generative model). Another use for the UI is that it shows the most relevant chunks regarding the question (best chunks among the top_k specification, see settings for more info).

python chat_server.py

The default url for access is http://0.0.0.0:8000.

UI settings

You can specify which retrieval method to use in the settings section (vector, bm25, hybrid).

In the case of hybrid retrieval method, you can specify which alpha to use (threshold between vector and bm25).

Lastly, you can directly specify which top_k to use. However, please note that with the current generative model response are limited to the best chunk only. It will still show the best chunk of every file within the specified top_k ratings.

Changing the generative models

We have a configuration file in which GENERATION_MODEL_NAME specifies which generative model to use through our inside libraries. Currently we are using google/flan-t5-base, which is a lightweight model that we are able to run in order to perform tests on the pipeline.

We have recognized a bottleneck in response quality due to small context window : token budget is too small to give more than the first chunk of the best file. To solve that issue we have tried adding other models with a greater context window, but came across RAM and incompatibility issues.

While we only garantee proper function with the current provided model, please note that this project has a simple but important potential considering further updates by coding an interface for better generative models.

The next step would be trying to use yeirr/phi3-mini-128k-instruct-awq-g128-4bit, which is a 4-bit quantization with minimal RAM requirements that allows up to 128K tokens. Having a large enough context window would allow both for multiple sources to be taken into account as context for generating responses, but also to have context-aware chat that remembers previous questions and responses. The ideal result, for this project, would be obtaining real-time answers that are based on the top 3-5 files using multiple chunks of the best sections to compose a complex answer, and for the generative model to be able to correct his answer based on user feedback (for instance we currently have some prompts with added "be as precise as possible" that affect the retrieved chunks, and because we are limited to one, it leads to incorrectness in answers for prompts that are more "chat-like" than a pure question). General explanation within a specific context is also affected, as we cannot provide both general sources and specific sources at the same time.

As a last note, it is also possible to change the ingestion model (feature extraction), however it requires re-running the whole ingestion in case of different vector size support.

8. Benchmarking Chunking & Retrieval Strategies

We include a lightweight benchmarking suite to compare retrieval quality across different chunking and retrieval strategies, with the goal of identifying which configuration is best adapted to our specific corpus (IP policy papers, patent-related academic work).

Because there is no single “correct” answer for most policy-style questions, the benchmarks are not meant to produce absolute scores, but rather to support relative comparisons between strategies.

Why benchmarking is useful in this project

• Our documents are long, dense, and heterogeneous (law, economics, policy, patents).
• Different chunking strategies may preserve context differently depending on structure.
• Retrieval quality depends strongly on how information is segmented before embedding.
• Benchmarks help answer practical questions such as:
  • Do semantic chunks outperform fixed-size chunks on our data?
  • Does sentence-level chunking hurt recall for policy questions?
  • At which k does retrieval saturate (no new relevant documents)?

The benchmark results are therefore mainly used to select the most appropriate chunking and retrieval strategy for our dataset, not to claim universal optimality.

---

8.1 Chunking benchmark (local, no Weaviate)

The script benchmark_chunking.py compares chunking strategies in isolation:

• Documents are re-chunked using each strategy.
• Chunks are embedded locally with the same sentence-transformer used in ingestion.
• Nearest-neighbour search is performed in-memory.
• Performance is measured using Hit@k (i.e., whether at least one expected document appears in the top-k retrieved chunks).

This avoids any dependency on Weaviate and isolates the effect of chunking alone.

Run:

python benchmark/benchmark_chunking.py --max-docs 25 --top-k 10 \
  --questions-file benchmark/benchmark_questions.json \
  --data-dir extracted-data \
  --openalex-dir extracted-data/openAlex

8.2 Retrieval benchmark (vector / keyword / hybrid)

In addition to chunking, we provide a retrieval-level benchmark (retrieval_benchmark.py) that evaluates full retrieval strategies on top of the indexed Weaviate corpus.

Supported strategies include:

• Vector (semantic) search
• Keyword (BM25) search
• Hybrid retrieval (client-side fusion of vector + BM25)

Metrics computed include:

• HitRate@k
• MRR@k (Mean Reciprocal Rank with cutoff)
• Stagnation k: The smallest k after which the scores become constant.

This helps understand not only whether a strategy works, but how quickly it converges in terms of relevant sources.

Run:

python benchmark/retrieval_benchmark.py \
  --questions-file benchmark/benchmark_questions.json \
  --k-values 1 3 5 10 20 30 \
  --output-prefix benchmark_results

8.3 Question dataset construction

The benchmark questions are defined in benchmark/benchmark_questions.json. They were constructed manually with the following principles:

• Questions are document-grounded: Each question is associated with one or more expected source documents (identified by doc_id).
• Questions are policy-style and open-ended: They reflect realistic user queries (e.g., “How do stronger IPRs affect pharmaceutical exports?”).
• Semantic robustness: Many questions are paraphrases or reformulations of paper contributions to test semantic understanding rather than keyword overlap.
• Many-to-one mapping: Multiple questions may map to the same document to reduce sensitivity to phrasing.
• Permissive ground truth: Because several documents may legitimately answer a question, the ground truth is intentionally permissive, and evaluation focuses on retrieval plausibility rather than exact matching.

Interpretation note

Benchmark results should be read comparatively:

• A higher Hit@k indicates better recall for relevant documents.
• MRR highlights how early relevant documents appear in the ranking.
• Stagnation analysis helps choose a reasonable TOP_K_RESULTS for RAG.

There is no single “best” score—the benchmark is a decision-support tool used to guide engineering choices for this specific corpus and task.

With these steps you can iterate on the RAG pipeline, refresh the vector store as your corpus changes, and run retrieval-augmented Q&A tailored to IP and patent data.