Autoscaling

This guide covers configuring node pool and cluster autoscaling for HostedClusters. Autoscaling automatically adjusts the number of worker nodes based on workload demands.

Node Pool Autoscaling

Node pool autoscaling enables individual NodePools to automatically scale between a minimum and maximum number of nodes based on pending pod resource requests.

Enable Autoscaling on a NodePool

To enable autoscaling, set spec.autoScaling and remove spec.replicas:

oc patch nodepool -n <HOSTED_CLUSTER_NAMESPACE> <NODEPOOL_NAME> --type merge -p '{
  "spec": {
    "replicas": null,
    "autoScaling": {
      "min": 1,
      "max": 5
    }
  }
}'

Labels and Taints When Scaling from Zero

Without a live worker, the autoscaler needs scheduling labels and taints in addition to resource capacity. For AWS and Azure autoscaling NodePools, HyperShift reconciles spec.nodeLabels and spec.taints into the capacity.cluster-autoscaler.kubernetes.io/labels and capacity.cluster-autoscaler.kubernetes.io/taints annotations on the MachineDeployment (Replace) or MachineSet (InPlace). Native provider capacity replaces CPU/memory/GPU workaround annotations, not labels or taints; their reconciliation does not require the optional capacity-discovery provider.

Topology labels are user-configured, not discovered automatically. If workloads use zone/region affinity, selectors, or volume topology requirements, supply the corresponding labels on the NodePool before relying on scale-from-zero. For example, for an AWS pool in eu-central-1b:

spec:
  nodeLabels:
    topology.kubernetes.io/region: eu-central-1
    topology.kubernetes.io/zone: eu-central-1b
    topology.ebs.csi.aws.com/zone: eu-central-1b
    topology.k8s.aws/zone-id: euc1-az3
    failure-domain.beta.kubernetes.io/region: eu-central-1
    failure-domain.beta.kubernetes.io/zone: eu-central-1b

Use the actual region, zone name, and zone ID for the pool's subnet in the tenant AWS account; these example values are not a universal zone-name/zone-ID mapping. Include the label keys required by your workloads, along with any custom labels. Label propagation does not validate placement against AWS. Incorrect values can misrepresent the pool to the autoscaler's simulation and result in unsuitable scale-up decisions or workloads remaining pending.

Verify the generated annotation before testing scale-up:

oc -n <CONTROL_PLANE_NAMESPACE> get machinedeployment <NODEPOOL_NAME> \
  -o go-template='{{index .metadata.annotations "capacity.cluster-autoscaler.kubernetes.io/labels"}}{{"\n"}}'

For InPlace pools, use machineset instead. Configuring only these labels or taints does not change the NodePool rollout hash or replace workers. Validate actual node labels and workload scheduling on a disposable test pool; this configuration is not a guarantee of successful scheduling.

Verify Autoscaling is Enabled

Check the AutoscalingEnabled condition on the NodePool:

oc get nodepool -n <HOSTED_CLUSTER_NAMESPACE> <NODEPOOL_NAME> -o jsonpath='{.status.conditions[?(@.type=="AutoscalingEnabled")].status}'

The output should be True.

Disable Autoscaling

To disable autoscaling and return to a fixed replica count:

oc patch nodepool -n <HOSTED_CLUSTER_NAMESPACE> <NODEPOOL_NAME> --type merge -p '{
  "spec": {
    "autoScaling": null,
    "replicas": 2
  }
}'

Cluster Autoscaling

Cluster autoscaling configures global autoscaling behavior that applies to all NodePools in a HostedCluster. This includes scale-down policies, node group balancing, and expander strategies.

Configure Cluster Autoscaling

Set spec.autoscaling on the HostedCluster:

oc patch hostedcluster -n <HOSTED_CLUSTER_NAMESPACE> <HOSTED_CLUSTER_NAME> --type merge -p '{
  "spec": {
    "autoscaling": {
      "scaling": "ScaleUpAndScaleDown",
      "maxNodesTotal": 10,
      "expanders": ["LeastWaste"],
      "scaleDown": {
        "delayAfterAddSeconds": 300,
        "unneededDurationSeconds": 600,
        "utilizationThresholdPercent": 50
      }
    }
  }
}'

Configuration Reference

Scaling Behavior

Expanders

Expanders control how the autoscaler selects which NodePool to scale when multiple NodePools can satisfy pending pods. Set via spec.autoscaling.expanders:

Default: [Priority, LeastWaste]. Up to 3 expanders can be specified in priority order.

Scale Down Configuration

The scaleDown field is only valid when scaling is set to ScaleUpAndScaleDown:

Node Group Balancing

How It Works

1. When a pod cannot be scheduled due to insufficient resources, the cluster autoscaler identifies NodePools with autoscaling enabled that can satisfy the pod's requirements.
2. The autoscaler selects a NodePool based on the configured expander strategy and triggers a scale-up by increasing the NodePool's replica count.
3. HyperShift provisions new platform-specific machine instances for the NodePool and the new nodes join the guest cluster.
4. When scaling is set to ScaleUpAndScaleDown, the autoscaler monitors node utilization. Nodes that remain underutilized (below utilizationThresholdPercent) for longer than unneededDurationSeconds are removed.

Monitoring

Check the current state of autoscaling:

# View NodePool autoscaling status
oc get nodepools -n <HOSTED_CLUSTER_NAMESPACE> -o wide

# Check autoscaler deployment in the control plane namespace
oc get deployment cluster-autoscaler -n <HOSTED_CLUSTER_NAMESPACE>-<HOSTED_CLUSTER_NAME>

# View autoscaler logs
oc logs deployment/cluster-autoscaler -n <HOSTED_CLUSTER_NAMESPACE>-<HOSTED_CLUSTER_NAME>