Issue with ColPali: Incorrect Page Retrieval for Fund-specific Queries

[cnkoralli]

I am using vidore/colpali-v1.3 in my project to work with PDF documents of funds from a company. However, I have noticed that when I search for a specific fund or ask a question related to a particular fund, the system often retrieves the wrong page.
Instead of identifying and returning the page that contains relevant information for that exact fund, ColPali sometimes:
Returns a page with the fund name that appears most frequently across the document, or
Matches based on individual high-frequency words from the query rather than the actual context or meaning of the question.
This results in irrelevant or less relevant pages being retrieved, especially when multiple funds share common words in their names or descriptions.
Could you please advise on how to improve the semantic understanding in ColPali so it retrieves the most contextually relevant page embeddings for a given fund-specific query?

[ManuelFay]

No need to cross post ;)
Try colqwen first!

[ManuelFay]

It's a much better model

[cnkoralli]

I have tried copali and colqwen both, On a set of 10 specific queries, they both were able to retrieve top pages for 8 queries. However i considered top 5 pages to solve the remaining 2 queries. colpali was able to retrieve the page required in the top 5 and Colqwen did not.
Didnt intent to cross post, I later realised that there was an option to raise an issue :