Mastering Cloud Capacity Autoscaling for Cost-Efficient Kubernetes Operations

Scenario

After migrating to the cloud, a company often faces either over-provisioning or under-provisioning, leading to high costs or performance degradation. For example, an e-commerce platform did not enable autoscaling during a flash sale, causing nodes to be fully loaded and pods to stay pending, while idle nodes wasted resources during off-peak hours.

Symptoms

• Cloud bills spike unexpectedly, especially compute costs.
• kubectl get pods shows many pods in Pending state.
• Node resource utilization is imbalanced: some nodes >80% CPU/memory, some <20%.
• Cluster autoscaler logs contain errors.

Diagnosis

1. Check pod status: kubectl describe pod <pod-name> to confirm insufficient resources.
2. View node resources: kubectl top nodes or use metrics-server.
3. Monitor scaling events: kubectl get events --field-selector reason=FailedScaleUp.
4. Analyze cloud costs via provider console or tools like Kubecost.

Commands & Configuration

HorizontalPodAutoscaler (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 20
  metrics:
  - type: Resource
    resource:
      name: cpu
      target:
        type: Utilization
        averageUtilization: 70

Apply HPA: kubectl apply -f hpa.yaml

Cluster Autoscaler

Enable on cloud provider, e.g., AWS: tag Auto Scaling Group with k8s.io/cluster-autoscaler/enabled. Install via Helm:

helm install cluster-autoscaler autoscaler/cluster-autoscaler \
  --set autoDiscovery.clusterName=<cluster-name> \
  --set awsRegion=<region>

Karpenter (optional)

apiVersion: karpenter.sh/v1beta1
kind: NodeClaim
spec:
  requirements:
    - key: kubernetes.io/arch
      operator: In
      values: ["amd64"]
    - key: karpenter.sh/capacity-type
      operator: In
      values: ["spot"]

Risk Controls

• Set reasonable min/max replicas to avoid extreme fluctuations.
• Use PodDisruptionBudget (PDB) to ensure service availability during rolling updates.
• Avoid overly frequent scaling; set cooldown delays.
• Use Reserved Instances or Savings Plans for baseline workloads.

Rollback Plan

• To disable autoscaling temporarily: kubectl delete hpa web-app-hpa.
• Remove Cluster Autoscaler: helm uninstall cluster-autoscaler.
• Revert to static node pools via cloud console.

Verification

• Check pod readiness: kubectl get pods -w.
• Monitor cost using cloud cost management tools.
• Run load tests to verify scaling behavior.

When to Submit an OpsGlobal Ticket

• Complex multi-region or multi-cluster configurations.
• Custom metrics (e.g., queue length) needed for scaling.
• Tight budget constraints requiring fine-grained optimization.
• Autoscaling failures beyond self-diagnosis.

OpsGlobal's SRE team provides expert configuration reviews to ensure your cloud infrastructure balances cost and performance effectively.