-
Notifications
You must be signed in to change notification settings - Fork 19
Add websearch #86
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add websearch #86
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work so far, sounds promising. Some changes needed.
I have to go so there will be a follow-up review later today
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Additional comments
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will make soon the changes related to the few additional comments I added
src/mmore/websearchRAG/config.py
Outdated
n_loops: int = 2 | ||
max_searches: int = 10 | ||
llm_config: Dict[str, Any] = field( | ||
default_factory=lambda: {"llm_name": "gpt-4", "max_new_tokens": 1200} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Make it a field of type LLMConfig
Websearch on RAG output
Input:
Output file from the RAG model
Output:
How to Run
Key Parameters
n_loops
: Number of search loopsmax_searches
: Maximum number of sources retrieved per web searchPipeline Overview
Load Input Data
Generate Initial Summary
Iterative Search and Analysis (Repeated
n_loops
times)Formulate a refined search query by combining the original query, current knowledge, and previous findings (if any).
Retrieve relevant web results using DuckDuckGo.
Use the language model (LLM) to integrate new web information with existing knowledge, updating the summary accordingly.
Save Final Results