How do we cut AI spend without sending personal data to the cloud?
Multi-Model Router
Every request is scored for difficulty and checked for personal data, then sent to the cheapest model allowed to see it. You see the decision, the reasons, every candidate it weighed, the fallback chain and what routing saved.
Personal data such as card numbers, IBANs, national ID references (Emirates ID, passport), emails and phone numbers keeps a request on a local model.
- Who it is for: teams paying for a cloud model on every request, including the easy ones and the ones that should not leave the building.
- The outcome: lower cost per request, sensitive data kept local, and a fallback chain when a model is slow or down.
- How it works: a policy layer detects personal data, scores difficulty, and picks between local Ollama models and cloud models, logging cost against a cloud-only baseline.
- Costs shown are calculated from stated list prices; the assumptions are on the page.
Built and run by Deepak Kumar Jha, Technical Lead and Solution Architect in Dubai. Talk to Deepak · LinkedIn · GitHub
Questions people ask
What is an LLM router?
A layer in front of several language models that decides, per request, which model should handle it, based on rules such as data sensitivity, difficulty, cost and latency, with a fallback when a model fails or is slow.
How does routing cut AI costs?
Most requests do not need the largest cloud model. Sending easy requests to a small local model and only hard, safe ones to a paid model lowers the cost per request. The lab compares each request with a cloud-only baseline using stated list prices.
How is personal data kept out of cloud models?
Each prompt is checked for personal data such as card numbers, IBANs, national ID references, emails and phone numbers. A request that contains any of them is only allowed to run on a local model.
Has Deepak used model routing in production?
Yes. At SmartSitting he introduced Gemini-based support with request routing by data sensitivity, cost and latency, which cut customer support load by 40-60%.
Who built this?
Deepak Kumar Jha, a Technical Lead and Solution Architect based in Dubai, UAE, with 14 years of building and running production systems. He designed, built, deployed and operates every part of the lab himself: the Angular console, the NestJS APIs, the model serving and the gateway.