Deploy AI Models in Minutes. Not Hours.
GPU Router removes infrastructure complexity and lets developers deploy AI models to GPU infrastructure with just a few clicks.
gpurouter deploy glm-4-9b-chat --size smallReal-time AWS Spot capacity
AWS Spot Placement Score — 0 (unlikely) to 10 (very likely to get capacity right now).
Live score history is still building — check back soon.
Everything you need to serve a model
No EC2 wrangling, no security groups, no server flags. Just the parts that matter.
One-click deploy
Pick a model and a GPU tier — we boot the instance, load the model, and hand you a URL and API key.
GPU that fits
We match each model to the smallest GPU that can serve it, from a single A10G to an 8× H200 node.
OpenAI-compatible API
Every endpoint speaks the standard chat-completions API. Point any HTTP client at the URL.
TTL cost guards
Set an auto-terminate window so dev instances never quietly rack up a bill overnight.
Private by default
Endpoints are protected by an API key and can be locked to a CIDR range you control.
Built-in load testing
Measure TTFT and throughput from both client and server before you ship to production.
How it works
From zero to a running endpoint in four steps.
Connect your AWS account
One CloudFormation stack installs a small agent. It only ever talks outward, so nothing is opened up to us.
Pick a model
Each model ships in one tuned configuration, so there is no instance type, disk size, or server flag to choose.
Deploy
The agent launches it in your account. We watch it boot and health-check until it is genuinely answering.
Call your endpoint
Get a URL + API key. Point any OpenAI-compatible client at it and start shipping.
Supported models
Four production-ready foundation models, each pre-tuned for the GPUs it runs on.
Why GPU Router
Built for people who want a private GPU-backed endpoint without hand-rolling infrastructure.
- No EC2, security groups, drivers, or vLLM flags to hand-roll
- GPU auto-matched to each model — pay only for what fits
- State lives in AWS tags, so endpoint keys are never lost
- One command up, one command down — billing stops instantly
- Server-side metrics scraped for real throughput numbers
- Restrict access to a CIDR range and rotate keys anytime
“We went from a blank AWS account to a live inference endpoint before our standup finished. It is genuinely a few clicks.”
“The GPU-fit sizing alone saved us from over-provisioning. The load-test report is exactly the data I needed.”
“The CLI was already great. The dashboard makes it something I can hand to the rest of the team without a walkthrough.”
Frequently asked questions
Everything you might want to know before deploying.
No. GPU Router picks a GPU that fits your model, boots the instance, loads the model, and returns a ready-to-call URL and API key. When you are done, one action tears it down and stops billing.
Ship your first endpoint today
Deploy a foundation model to your own GPU in minutes. No infra code, no console clicking.