Hardening Kubernetes RBAC and Pod Security: A Practical Guide for SREs

Scenario

An SRE team discovers that a developer account has full cluster-admin privileges, allowing arbitrary pod exec and secret access, posing a major security risk.

Symptoms

• Audit logs show unexpected kubectl exec or kubectl get secrets calls from non-admin users.
• Developers report being able to access namespaces beyond their scope.
• Security scans flag excessive ClusterRoleBindings.

Diagnosis

1. Run kubectl auth can-i --list --as=developer to list capabilities for a specific user.
2. Execute kubectl get clusterrolebindings -o wide to identify subjects bound to high-privilege roles like cluster-admin.
3. Use kubectl describe clusterrolebinding <name> to inspect bindings.
4. Check Pod Security Policies: kubectl get psp — no output means PSPs are not enforced.

Example Commands

# Check user permissions
kubectl auth can-i --list --as=developer

# Apply a restricted PodSecurityPolicy
kubectl apply -f restricted-psp.yaml

# Create a read-only ClusterRole
kubectl create clusterrole readonly --verb=get,list,watch --resource=pods,services,endpoints

# Bind to developers group
kubectl create clusterrolebinding dev-readonly --clusterrole=readonly --group=developers

# Rollback: delete the policy
kubectl delete -f restricted-psp.yaml

Risk Controls

• Implement least privilege: create dedicated ServiceAccounts and RoleBindings per team or application.
• Enable Pod Security Admission (PSA) or OPA Gatekeeper for policy enforcement.
• Enable cluster audit logs and centralize monitoring (e.g., ELK or OpsGlobal SIEM integration).
• Use NetworkPolicies to restrict pod-to-pod communication.

Rollback Plan

1. Save existing ClusterRoleBindings: kubectl get clusterrolebinding dev-admin -o yaml > backup.yaml
2. Delete restrictive policy: kubectl delete -f restricted-psp.yaml
3. Restore the binding: kubectl apply -f backup.yaml

Verification

• As developer, run kubectl auth can-i delete pods — should return 'no' if properly restricted.
• Check audit logs for any denied operations.
• Attempt to create a privileged pod — should be rejected.

When to Submit an OpsGlobal Ticket

• Your internal team lacks Kubernetes security expertise.
• You need 24/7 monitoring and rapid incident response.
• An active breach or misconfiguration requires immediate expert intervention.
• Regular security audits and compliance (e.g., SOC2, PCI-DSS) need external validation.