[Question]: Technical Specifications of Ragflow's Data Capacity and Key Considerations

[ipa2800]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (Language Policy).
• Non-english title submitions will be closed directly ( 非英文标题的提交将会被直接关闭 ) (Language Policy).
• Please do not modify this template :) and fill in all the required fields.

Describe your problem

What is the maximum data volume that Ragflow can handle, and what are the critical considerations?
For example, in a RAG project, can the single-machine version of Ragflow support queries on 200GB of vector data?

[dosubot]

Hi @ipa2800! I'm Dosu and I’m helping the ragflow team.

Ragflow does not impose a software-level limit on total data volume—it can handle very large datasets (even 100TB+) by scaling backend components like Elasticsearch and MinIO nodes. For a single-machine setup, the main constraints are your hardware: CPU, RAM, disk space, and the configuration of your vector store backend (usually Elasticsearch). There are no explicit limits documented for handling 200GB of vector data on a single machine; if your hardware (especially RAM and disk) and Elasticsearch are tuned appropriately (e.g., setting vm.max_map_count), Ragflow can support this scale in principle. The default per-file analysis limit is 128MB, but you can increase this via the MAX_CONTENT_LENGTH environment variable if needed. For best performance, ensure your machine has sufficient resources and that Docker container memory and Elasticsearch settings are properly configured. See more details in the quickstart guide and configuration docs.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[ipa2800]

I understand that infrastructure components like Elasticsearch and MinIO are the bottlenecks of the entire system's processing capacity. I'd like to know if there are any relevant case studies or performance test reports under large data volumes. This would significantly boost our confidence in using RagFlow for large-scale scenarios and help us mitigate risks.

[qassiel]

@dosu, following up on @ipa2800 question, can RAGFlow open source version handle like a million documents? Can a knowledge base (dataset) handle that amount of documents? Also, does the RAG quality decrease when having so many documents?

[dosubot]

Yes, the open source version of RAGFlow can handle knowledge bases with around a million documents, as long as your backend (Elasticsearch/Infinity, MinIO) and hardware are sized appropriately—there are no hard software limits at the application level for dataset size or document count. For enterprise deployments, it's noted that up to 1 million document chunks can be managed with two nodes if the embedding service and storage are properly configured (source).

RAG quality does not inherently degrade just because of scale; RAGFlow uses hybrid search, reranking, and clustering (RAPTOR) to maintain retrieval quality as datasets grow. However, as your dataset increases, you may need to tune parameters like similarity thresholds, top-K, and rerank limits, and monitor backend performance to keep latency and quality in check (source). Features like RAPTOR and reranker can improve quality but may slow things down at large scale, so you might want to disable them for speed if needed (source).

There are no official public benchmarks or case studies for RAGFlow at the million-document scale, but the architecture and code are designed for this use case. Retrieval quality and system throughput will depend on your hardware, backend tuning, and retrieval configuration. If you run into performance issues, try adjusting retrieval parameters, using GPU for embedding, and scaling your backend storage (source).

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[qassiel]

Yes, the open source version of RAGFlow can handle knowledge bases with around a million documents, as long as your backend (Elasticsearch/Infinity, MinIO) and hardware are sized appropriately—there are no hard software limits at the application level for dataset size or document count. For enterprise deployments, it's noted that up to 1 million document chunks can be managed with two nodes if the embedding service and storage are properly configured (source).

@dosu, what nodes are you referring to? Minio nodes? Elasticsearch nodes? And how can I scale to two of these nodes in the open source version?

[qassiel]

@dosu reply pls.