/// about
Open models on dedicated GPUs, without the DevOps.
GPU Router exists to close the gap between an open-weights model on Hugging Face and a production inference endpoint. We handle GPU selection, driver setup, the serving runtime, and cost controls — so shipping a frontier model feels like shipping a serverless function.
Deploy in a few clicks
Pick a model, pick a GPU tier, and go. No Dockerfiles, no CUDA drivers, no Kubernetes YAML.
The right GPU, every time
We map each model to a validated instance — BF16 models to Ampere, FP8 MoEs to Ada and Hopper — so it just boots.
Dedicated, not shared
Every deployment runs on its own dedicated GPU. Predictable latency, no noisy neighbors, no token metering.
Cost guards built in
TTL auto-shutdown and per-hour pricing are first-class. Frontier deployments always ship with a safety timer.
How it works under the hood
A curated model registry
Each model in the catalog carries its precision, weight size, attention backend, and the exact GPU topology that serves it. GLM-5.2, for example, is pinned to an 8× H200 node because its sparse MLA attention needs a Hopper-class FlashMLA backend.
vLLM under the hood
Every endpoint is served by vLLM with continuous batching and paged attention. FP8 models use fp8 KV-cache to fit long context on a single card, and weights stream from same-region S3 for fast, free in-region transfer.
An OpenAI-compatible endpoint
Once a deployment is running you get a standard /v1/chat/completions URL and an API key. Point your existing OpenAI client at it and you are done — no SDK rewrite required.
Built for the whole spectrum
From a 4B vision model on a single A10G to a 756GB MoE across 8× H200 — one workflow, every scale.