Upgrade on Amazon EKS - Poolside

Overview

This guide describes how to upgrade an existing model inference deployment on Amazon EKS to a new Helm bundle. The upgrade updates the inference Helm release. The upgrade process includes the following phases:

  1. Prepare the new bundle: Extract the bundle and reuse the values file from the previous deployment. Add any new values the new chart requires.
  2. Upload new container images: Push the new bundle’s container images into Amazon ECR.
  3. Upgrade the inference release: Run helm upgrade against the inference chart.
  4. Verify: Confirm that the new revision is deployed and the pods are healthy.

Prerequisites

Downtime

The upgrade rolls model pods one Deployment at a time. The chart sets maxSurge to 0 so a rolled model does not request additional GPUs during the rollout, which means that model goes down briefly while its new pod starts. Each model server also re-downloads its checkpoint from S3 on restart, so expect a delay before a rolled model becomes ready. Plan a maintenance window if you run single-replica models.

Step 1: Extract the new bundle

Poolside provides the new bundle as a tarball. Extract it to a directory of your choice, then set a shell variable for the new bundle root:

export NEW_BUNDLE=<path-to-new-bundle>

Step 2: Review the values file

Reuse the inference_values.yaml file from your previous deployment. Poolside notes any required values changes in the release notes. The new bundle contains the reference values.yaml for the inference chart at charts/inference/values.yaml. Use it as a reference while reviewing your existing file.

Step 3: Upload the new container images

The new bundle ships updated container images in ./containers/. Authenticate skopeo to your ECR registry, then push the images to the same repositories that the inference release uses:

aws ecr get-login-password --region <aws-region> \
  | skopeo login --username AWS --password-stdin <account-id>.dkr.ecr.<aws-region>.amazonaws.com

cd $NEW_BUNDLE
./scripts/upload_images.sh <account-id>.dkr.ecr.<aws-region>.amazonaws.com

The image tags are specific to the new bundle, not fixed values. After the upload completes, confirm the atlas tag that was pushed before you continue:

aws ecr describe-images --repository-name atlas --region <aws-region> \
  --query 'sort_by(imageDetails,&imagePushedAt)[-1].imageTags' --output text

Step 4: Dry-run the upgrade (optional)

Preview the changes before you apply them:

helm upgrade inference \
  $NEW_BUNDLE/charts/inference \
  -f <path-to-inference-values.yaml> \
  -n poolside-models --dry-run --debug | less

Step 5: Apply the upgrade

Run the upgrade and watch the pods roll. The pods should return to a Running state when the upgrade completes:

helm upgrade inference \
  $NEW_BUNDLE/charts/inference \
  -f <path-to-inference-values.yaml> \
  -n poolside-models

kubectl get pods -n poolside-models -w

Step 6: Update models (optional)

You can add, update, or remove model checkpoints as part of this upgrade rather than as a separate operation. Make the model edits in the same inference_values.yaml file you reviewed in Step 2, before you run the helm upgrade in Step 5. The single helm upgrade then reconciles both the new chart and the model changes. For the full procedure to add, update, or remove models, see Manage models on Amazon EKS. You can also run those changes separately at any time after the upgrade.

Verification

Confirm the release is deployed:

helm history inference -n poolside-models

Verify that all pods are healthy:

kubectl get pods -n poolside-models

Confirm that the inference endpoints still serve traffic, where <model-hostname> is the ingressHost of a model under models:

curl -s https://<model-hostname>/v1/models \
  -H "Authorization: Bearer <vllm-api-key>"

If API key authentication is off, omit the Authorization header.

Troubleshooting