Testing LLM endpoints is becoming a single command

Setting up a private, OpenAI-compatible endpoint used to mean wrestling with Kubernetes or provisioning servers. It’s a high-friction barrier for anyone just trying to run a quick evaluation or a batch of tests.

Hugging Face is removing that friction by bringing vLLM directly into HF Jobs. Instead of managing infrastructure, you can now spin up a server with a single command—essentially treating HF's hardware like a local `docker run` instance.

You pick your GPU flavor—ranging from a budget-friendly `a10g-large` to beefy `h200x2` setups for massive models like Qwen3.5-122B—and use an `--expose` flag to get a reachable URL immediately. Because the system is billed per second, it's designed for high-velocity experimentation rather than permanent hosting.

It’s not meant to replace managed production services like Inference Endpoints. Instead, it fills the gap for developers who need maximum flexibility, the ability to SSH into a running container for debugging, or a quick way to back a coding agent without the overhead of a long-lived service.