Skip to content

fix: store multimodal_processed in separate KV namespace to prevent DocProcessingStatus errors#253

Open
peterCheng123321 wants to merge 1 commit intoHKUDS:mainfrom
peterCheng123321:fix/91-multimodal-status-separate-storage
Open

fix: store multimodal_processed in separate KV namespace to prevent DocProcessingStatus errors#253
peterCheng123321 wants to merge 1 commit intoHKUDS:mainfrom
peterCheng123321:fix/91-multimodal-status-separate-storage

Conversation

@peterCheng123321
Copy link
Copy Markdown

@peterCheng123321 peterCheng123321 commented Apr 21, 2026

Summary

Fixes #91, fixes #119.

RAGAnything was injecting a multimodal_processed field directly into LightRAG's doc_status KV records. The LightRAG Server API deserializes those records into DocProcessingStatus dataclass objects. Any LightRAG version whose DocProcessingStatus does not declare multimodal_processed raised:

DocProcessingStatus.__init__() got an unexpected keyword argument 'multimodal_processed'
500 Internal Server Error on /documents/paginated

Fix: introduce a dedicated raganything_multimodal_status KV namespace (same storage class as parse_cache) to hold per-document multimodal processing state. LightRAG's own doc_status records are no longer modified with RAGAnything-specific fields, so DocProcessingStatus deserialization always succeeds regardless of LightRAG version.

Changes:

  • raganything/raganything.py — add multimodal_status field, initialize in both pre-provided and newly-created LightRAG paths, finalize in finalize_storages
  • raganything/processor.py — all multimodal_processed reads/writes now go through self.multimodal_status; doc_status upserts no longer contain this field

Test plan

  • Process a document, then call /documents/paginated on LightRAG Server — confirm no 500 error
  • Verify adelete_by_doc_id still works after processing
  • Confirm is_document_fully_processed() and get_document_processing_status() return correct values

LightRAG's Server API deserializes doc_status records into DocProcessingStatus
dataclass objects. Because RAGAnything was injecting a 'multimodal_processed'
key directly into those records, any version of the dataclass that did not
declare that field raised:

  DocProcessingStatus.__init__() got an unexpected keyword argument 'multimodal_processed'

causing 500 errors on /documents/paginated and similar endpoints.

Fix: introduce a dedicated 'raganything_multimodal_status' KV namespace
(same storage class as parse_cache) to hold per-document multimodal
processing state. LightRAG's own doc_status records are no longer modified
with extra fields, so DocProcessingStatus deserialization always succeeds.
All read/write paths in processor.py are updated accordingly.

Fixes HKUDS#91
Fixes HKUDS#119

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@LarFii
Copy link
Copy Markdown
Collaborator

LarFii commented Apr 25, 2026

Thanks for working on this. I reviewed the separate raganything_multimodal_status KV namespace approach against the current LightRAG DocProcessingStatus model.

I don't think this should be merged as an isolated small fix yet. Moving multimodal_processed out of doc_status avoids older LightRAG deserialization issues, but it also changes the status semantics for newer LightRAG versions where DocProcessingStatus.multimodal_processed is now a first-class field and can drive PROCESSED -> PREPROCESSED presentation while multimodal processing is incomplete.

Before this can land, I think we need at least:

  • a migration or read-fallback plan for existing doc_status records that already contain multimodal_processed, otherwise is_document_fully_processed() can regress until documents are reprocessed;
  • tests covering get_document_processing_status() / is_document_fully_processed() and the LightRAG server pagination failure this is meant to avoid;
  • an explicit plan for how this coordinates with fix: create doc_status even when LightRAG lacks multimodal insert args #255, because both PRs touch the same doc_status / multimodal completion area and will conflict semantically;
  • a decision on whether LightRAG server APIs should continue to reflect multimodal completion state, or whether that state becomes RAG-Anything-only metadata.

So I would hold this for a design pass rather than merging it directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

3 participants