mcp-server

semantic-router: Smart Routing

Name: semantic-router: Smart Routing
Author: vllm-project

Get intelligent routing for mixture-of-models with semantic-router, ideal for startup founders using AI at scale, backed by 4.7k+ GitHub stars

4,975 stars754 forksGoQuality 8/10Updated 7/15/2026100% free · open source

What it does

Semantic-router intelligently routes traffic for mixture-of-models deployments across cloud, data center, and edge environments, optimizing performance and efficiency

Install / run

git clone https://github.com/vllm-project/semantic-router.git

When to use it

•When deploying AI models at scale and needing efficient traffic routing
•In environments with diverse model types requiring smart routing decisions
•For optimizing resource utilization in cloud, data center, or edge computing

Quick start

1Navigate into the cloned repository with 'cd semantic-router'
2Build the semantic-router with 'go build -o semantic-router main.go'
3Configure the routing logic by editing the 'config.yaml' file
4Run the semantic-router using './semantic-router -config config.yaml'
5Verify the routing decisions by checking the logs or using a tool like 'curl' to test the API

Ready-to-paste prompt

Run './semantic-router -config config.yaml -model-dir /path/to/models' to start the router with a specific model directory

Heads up: Ensure you have Go (version 1.17 or later) installed on your system, as semantic-router is built using Go and requires it to compile and run

Saves to your device