Guidance Needed for Project

Hi everyone**,**

I’m working on a project to build an optimized GPT platform that takes a prompt as input and makes API calls to multiple LLMs (e.g., ChatGPT, Perplexity, etc.). The goal is to evaluate the responses from these models and return the best output to the user.

As a student, I’m on a tight budget, and I need your guidance on a few key points:

1. Cost-Effective API Calls:
   • How can I design the API calls so they don’t turn out to be overly expensive?
   • Are there any best practices or techniques to optimize token usage for APIs like OpenAI and Perplexity?
2. Reducing Latency:
   • How can I handle API calls efficiently to avoid long wait times for the user?
   • Are there strategies for managing multiple API calls in parallel without increasing the response time significantly?

Additionally, I want to experiment with building this system using open-source models like LLaMA or Mistral. However, I’m limited by my hardware: I have a Dell Inspiron 5000 laptop without a GPU.

• What are some lightweight open-source models I could use that run well on a CPU?
• Are there cloud-based solutions (preferably free or low-cost) where I could experiment with running these models?

Any advice, resources, or tips would be incredibly helpful.