Hey everyone! I'm currently working on an AI-based grant writing system that includes two main features:

1. Main AI: Uses LLMs to generate grant-specific suggestions based on user-uploaded documents.
2. Copilot Feature: Allows document-specific Q&A by utilizing a query format like /{filename} {query} to fetch information from the specified document.

Currently, we use FAISS for vector storage and retrieval, with metadata managed through .pkl files. This setup works for similarity-based retrieval of relevant content. However, I’m considering introducing a reranking model to further enhance retrieval accuracy, especially for our Copilot feature.

Challenges with Current Setup:

• Document-Specific Retrieval: We're storing document-specific embeddings and metadata in .pkl files, and retrieval works by first querying FAISS.
• Objective: Improve the precision of the results retrieved by Copilot when the user requests data from a specific document (e.g., /example.pdf summarize content).

Questions for the Community:

1. Is using a reranking model (e.g., BERT-based reranker, MiniLM) a good idea to add another layer of precision for document retrieval, especially when handling specific document requests?
2. If I implement a reranking model, do I still need the structured .pkl files, or can I rely solely on the embeddings and reranking for retrieval?
3. How can I effectively integrate a reranking model into my current FAISS + Langchain setup?

I’d love to hear your thoughts, and if you have experience in using reranking models with FAISS or similar, any advice would be highly appreciated. Thank you!