Skip to content

Latest commit

 

History

History
16 lines (9 loc) · 1.2 KB

File metadata and controls

16 lines (9 loc) · 1.2 KB

MultiModal-RAG

Overview:

With the growing number of research papers being published it is becoming more and more important to continue reading and understnading them to understand the AI landscape. The following project aims to reduce the gap between the research paper and an engineering undergrad. The following project outlines two approaches to build a chatbot to have a conversation with a chatbot about a research paper(Attention is all you need in this case). Since research paper consists of unstructured data in the form of tables,images and texts it becomes increasing difficult to extract each bit of information.

Approach 1: in MM-RAG

image image

Approach 2: in Multimodal-RAG

the following approach utillisees CLIP and an ensemble retriever using Similarity search and MMR(rereanker)

ChatGPT Image Aug 17, 2025, 07_25_28 PM