inference server for AMD GPUs
radiance
Language models, and image, video, speech and music generation, in C++ and HIP. Its HTTP API speaks OpenAI's, Anthropic's and llama-server's, so their clients and SDKs work unchanged.
Models
Every container holds the weights, the tokeniser, the chat template and any drafter, and each download's repository has the recipe that made it. A compose file in deploy/compose serves it; VRAM is what that file uses, in 32 GB cards.
Kind
Fits in VRAM