Capacity planning - Poolside
Estimate capacity
Use the planner to estimate the maximum number of active agent tasks your deployment can sustain under the selected assumptions. Set the inputs to match the deployment you are planning:
- Hardware: Select the GPU type you want to model.
- Model: Select the Laguna model you plan to deploy.
- Number of GPUs: Select the number of GPUs assigned to the model-serving node.
- Average context per task: Select how large the agent’s context window grows by the end of a typical task. Use a higher value for longer tasks, larger codebases, or workflows that read many files. Use a lower value for short, focused tasks.
- Step-latency SLO: Select the p50 latency target per agent turn. A stricter SLO lowers the number of concurrent agents the deployment can serve.
The planner reports:
- Concurrent agents: The estimated number of active agent tasks that can occupy model-serving slots at the same time.
- Seats at 40%: A conservative developer-seat estimate for first-time sizing or agent-heavy usage.
- Seats at 25%: A lighter-concurrency estimate for pilots, mixed workloads, or deployments with telemetry that shows lower peak activity.
Capacity planner
Estimate concurrent agents and supported developer seats for a Laguna deployment, based on Poolside's analytical inference simulator calibrated against measured benchmarks.
Hardware
- H200 (141 GB · Hopper)
- H100 (80 GB · Hopper)
- RTX PRO 6000 Blackwell (96 GB)
- A100 80 GB (Ampere · BF16)
- A100 40 GB (Ampere · BF16)
- L40S (48 GB · Ada · PCIe)
- NVIDIA DGX Spark (128 GB unified)
Model
- Laguna M.1 (222B MoE)
- Laguna XS.2 (33B MoE+SWA)
Number of GPUs
1248
Average context per task
- 8K
- 16K
- 32K
- 64K
- 128K
- 256K
Step-latency SLO
- 5s
- 10s
- 20s
- 30s
Capacity
- Concurrent agents: 96 active model slots
- Seats at 40%: ~240 conservative default
- Seats at 25%: ~384 light concurrency
If the planner reports that a configuration cannot serve a single agent, increase the GPU count, choose a smaller model, reduce the average context size, or relax the step-latency SLO.
Interpret the estimate
Concurrent-agent capacity is not the same as the number of developers a deployment supports. A developer consumes a model-serving slot only while an agent task is actively running. Outside of an active task, the developer does not consume agent capacity. To estimate supported seats, divide concurrent-agent capacity by the fraction of seats actively running an agent at peak:
Seat estimate
seats = concurrent-agent capacity / active-concurrency ratio
Use a planning range of 25 to 40 percent, with 40 percent as the conservative default for initial sizing. Laguna agent tasks typically take two to three minutes to complete. Each active agent occupies a slot for that full duration, so the instantaneous concurrency ratio for agent workloads runs higher than for chat-style models. Without real-world telemetry from your deployment, plan against the higher end of the range.
Understand calibration confidence
The planner uses an analytical inference simulator calibrated against measured Poolside benchmarks. The confidence badge in the planner indicates how closely the selected configuration matches measured data:
- Calibrated: Direct measurement exists for the selected model, GPU, GPU count, and precision.
- Same arch: Measurement exists for the same model on the same GPU architecture.
- Factorized, partial signal, or arch median: The estimate depends more heavily on extrapolation.
Use extrapolated estimates for comparison and early planning. For production sizing, validate the selected configuration with Poolside against your expected workload.
What affects capacity
Real-world capacity depends on your deployment shape and workload:
- Model choice
- GPU type and GPU count
- Weight and key-value cache precision
- Average context size per trajectory
- Step-latency target
- Number of steps each agent task takes
- Mix of agent and chat workloads
- Peak-time concurrency and burst behavior
Use the planner output as a conservative starting point. As your team builds usage history, replace the default planning assumptions with your own observed values.
Choose a model for your deployment
For full model details, see Supported models.
| Model | When to choose it |
|---|---|
| Laguna XS.2 | Use when concurrent-agent throughput is the priority, you have limited GPU availability, or you need a strong default for most agent workloads. |
| Laguna M.1 | Use when agent quality matters more than raw throughput. It is the best fit on 8× H200 hardware and can serve smaller teams on RTX 6000 Blackwell when you need lower concurrency. |
Malibu 2.2 remains available for existing deployments and dense-model preferences, but the planner focuses on Laguna capacity. For Malibu sizing, contact your Poolside account team.
Scale beyond a single node
Both on-premises and cloud deployments support multi-node GPU inference within a single Kubernetes cluster. Multi-node configurations distribute independent inference replicas across nodes to add throughput. Cross-node tensor parallelism is not supported, and multi-node configurations do not provide high availability against node failures. For deployment-specific configuration, see Supported configurations for on-premises, Manage models on OpenShift, or Manage models on upstream Kubernetes.
Keep estimates current
Capacity changes as Poolside improves quantization formats, key-value cache handling, request scheduling, and inference-engine performance. Revisit this page when you change model versions, hardware, workload mix, or latency targets.