First ask whether one complete replica fits.
vLLM's deployment guidance starts with a single GPU when the model fits, then considers tensor parallelism within a node and combined strategies across nodes for larger models. Fit should cover the intended serving workload, not just the weight files. Our practical starting point is to establish one successful replica with a bounded context and request load, then use that measurement to decide what additional GPUs should accomplish.
Splitting a model is different from duplicating it.
Tensor parallelism distributes parts of model computation across devices that must cooperate on requests. Independent replicas each keep a complete model and handle different requests. The former can address per-replica memory constraints; the latter increases the number of available serving workers. Replicas do not combine their memory into one larger checkpoint budget. Record the GPUs per replica and the replica count separately in any report.
Total VRAM is not a complete hardware description.
Two deployments with the same aggregate memory can have different communication paths and runtime behavior. Record the GPU model, node count, interconnect, and parallelism settings with the result. A comparison that simply says four GPUs leaves too much unspecified. Our proposed test matrix changes one topology or parallelism setting at a time and includes the single-replica baseline whenever it is feasible.
Test the reason you added the GPU.
If the goal is fit, verify long requests and peak concurrency without allocation failures. If the goal is capacity, measure completed work within a latency target. If the goal is lower latency, compare the same request distribution at controlled load. For example, two replicas may be useful for independent batch jobs even when splitting each request brings little benefit. That is a hypothesis to measure, not a universal scaling claim. Publish failures and the workload alongside any speedup.
Put the context to work.
Explore the models, compare the fundamentals, and start with a clearer picture.
Compare models