We use cookies to enhance your experience, serve personalized ads or content, and analyze our traffic. By clicking "Accept", you consent to our use of cookies. For more information, please see our privacy policy.
    Skip to content
    vLLM logo

    vLLM

    Run an OpenAI & Claude inference endpoint with vLLM
    LocalAIInferenceLlmOpenAIVllm

    About vLLM

    Deploy an OpenAI & Claude compatible REST API endpoint using vLLM with GPU acceleration — running natively under systemd instead of Docker. This template runs Qwen/Qwen3-Coder-Next across 2 GPUs with tensor parallelism, tool calling support, and HuggingFace model caching.
    The endpoint is fully compatible with the OpenAI /v1/chat/completions and Claude /v1/messages, making it a drop-in replacement for any OpenAI compatible SDK client.
    Use a GPU server with NVIDIA drivers already installed (e.g. Verda 2xA100, 2xH100, or similar). Dependencies are managed with uv from a native pyproject.toml, and uv auto-selects the PyTorch CUDA wheels matching the host driver (CUDA 12.9 / 13.0), so no CUDA toolkit install is required. Set HF_TOKEN to your HuggingFace token to download gated models, and adjust the tensor-parallel size in the start command to match your GPU count.
    This template is experimental — verify the deployment on your own GPU server before relying on it.
    DollarDeploy

    About DollarDeploy

    DollarDeploy deploys and manages apps on your own VPS — no SSH, no YAML, no lock-in. Launch vLLM in a few clicks, then get HTTPS, monitoring, logs and backups handled for you.