GPU
GPU machines for AI, inference or compute. Where to find them, how to choose them, and how to have a service use them.
Where to find GPUs
Section titled “Where to find GPUs”Only some hosting providers and some regions offer them, and their stock changes often. When you create a cluster, answer Yes to Need GPUs (AI, compute)?: only regions offering GPU machines remain, with the price of the cheapest GPU machine.
If no region appears, no GPU machine is available right now at these providers. Try again later, try Anywhere for where your data may go, or answer No.
Choose the machine
Section titled “Choose the machine”At the How many machines? step, pick the GPU machine in a machine group. GPU machine types are listed under With GPU, with their CPUs, memory and monthly price.
Have a service use the GPU
Section titled “Have a service use the GPU”In the application’s panel, Resources tab, set GPU to NVIDIA. The service will be placed on one of the cluster’s GPU machines.
To serve an AI model, an image such as Ollama or vLLM is added as an Application.
What it costs
Section titled “What it costs”The machine at the hosting provider’s price, plus €100 per GPU per month for Fransys. See Pricing. A GPU machine is expensive: take the cluster offline when you’re not using it.