adaptive-classifier

Author:

🔗 asankhs

Real discussions and feedbacks of adaptive-classifier

🔗 Join the Discussion on Forums

App Description

Hey Folks! I’m excited to share a new open-source library that can help optimize your LLM deployment costs. The adaptive-classifier library learns to route queries between your models based on complexity, continuously improving through real-world usage.

We tested it on the arena-hard-auto dataset, routing between a high-cost and low-cost model (2x cost difference). The results were impressive:

• 32.4% cost savings with adaptation enabled
• Same overall success rate (22%) as baseline
• System automatically learned from 110 new examples during evaluation
• Successfully routed 80.4% of queries to the cheaper model

Perfect for setups where you’re running multiple LLama models (like Llama-3.1-70B alongside Llama-3.1-8B) and want to optimize costs without sacrificing capability. The library integrates easily with any transformer-based models and includes built-in state persistence.

Check out the repo for implementation details and benchmarks. Would love to hear your experiences if you try it out!

Repo – https://github.com/codelion/adaptive-classifier

Project Overview

The adaptive-classifier is an open-source library designed to optimize LLM deployment costs by intelligently routing queries between models based on complexity. It has demonstrated significant cost savings (32.4%) while maintaining the same success rate as baseline models. The library is particularly suited for setups using multiple LLama models and integrates easily with any transformer-based models, featuring built-in state persistence.

Links

🌐 Website: https://github.com/codelion/adaptive-classifier

Features & Benefits

✅ Significant cost savings (32.4%)
✅ Maintains success rate
✅ Automatically learns from real-world usage
✅ Easily integrates with transformer-based models
✅ Built-in state persistence