/debug-rbac Skill

[Identify Deployment] → [Check Pod Status + Logs] → [Identify RBAC Errors] → [Analyze ServiceAccount] → [Check RoleBindings] → [Summary + Fix]

## RBAC Debugging

**Current OpenShift Context:**
- Cluster: [cluster]
- Namespace: [namespace]

Which deployment would you like me to debug for RBAC issues?

1. **Specify deployment name** — Enter the deployment name directly
2. **List deployments with issues** — Show deployments with unavailable or not-ready pods
3. **Search recent events** — Find pods with RBAC-related warning events

Select an option or enter a deployment name:

## Deployments with Issues in [namespace]

| Deployment | Available | Desired | Condition |
|------------|-----------|---------|-----------|
| [deploy-name] | 0 | 1 | MinimumReplicasUnavailable |

Which deployment would you like me to debug?

## Pod Analysis: [pod-name]

**Pod Status:**
| Field | Value |
|-------|-------|
| Phase | Running |
| Ready | false |
| Conditions | ContainersNotReady |
| Restart Count | [count] |

**Readiness Probe:**
| Field | Value |
|-------|-------|
| Type | [exec/httpGet/tcpSocket] |
| Command | [e.g., kubectl auth can-i list pods -n namespace] |
| Failure Count | [count] |
| Last Probe | [timestamp] |
| Message | [e.g., "probe returned: no"] |

**Container Logs (last 50 lines):**

[Highlight FORBIDDEN / 403 errors:]

| Timestamp | Error |
|-----------|-------|
| [time] | FORBIDDEN: pods is forbidden: User "system:serviceaccount:[ns]:[sa]" cannot list resource "pods" in API group "" in namespace "[ns]" |
| [time] | FORBIDDEN: pods is forbidden... (repeated) |

**Quick Assessment:**
[e.g., "Pod is running but readiness probe fails because the ServiceAccount cannot list pods. Logs confirm FORBIDDEN errors since [timestamp]."]

Continue with ServiceAccount analysis? (yes/no)

## Required Permissions Analysis

**FORBIDDEN Errors Found:**
| Resource | Verb | API Group | Namespace |
|----------|------|-----------|-----------|
| pods | list | "" (core) | [namespace] |
| pods | get | "" (core) | [namespace] |
| [other resources from logs] | [verb] | [group] | [namespace] |

**Readiness Probe Requires:**
| Permission | Currently Granted? |
|------------|-------------------|
| list pods in [namespace] | NO — probe returns "no" |

**Application Function Requires:**
| Permission | Evidence |
|------------|----------|
| get pods in [namespace] | Container main loop calls `kubectl get pods` |
| [other] | [from log analysis] |

**Minimum Role Needed:**
```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: <sa-name>-role
  namespace: <namespace>
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]

**WAIT for user confirmation before proceeding.**

### Step 4: Analyze ServiceAccount and RoleBindings

**MCP Tool**: `resources_get` (from openshift)

**Parameters**:
- `kind`: "ServiceAccount" (resource type)
- `name`: "<sa-name>" (from Deployment `.spec.template.spec.serviceAccountName`, default: `default`)
- `namespace`: "<namespace>"

**MCP Tool**: `resources_list` (from openshift)

**Parameters**:
- `kind`: "RoleBinding" (resource type)
- `namespace`: "<namespace>"

Optionally, if permissions allow:

**MCP Tool**: `resources_list` (from openshift)

**Parameters**:
- `kind`: "ClusterRoleBinding" (cluster-scoped)

**Expected Output**: ServiceAccount details and all RoleBindings/ClusterRoleBindings, checked for whether any grant the required permissions to the target ServiceAccount.

**Error Handling**:
- If listing RoleBindings is forbidden: note the limitation, infer from FORBIDDEN errors
- If ServiceAccount not found: report as a finding — SA may need to be created
- If multiple bindings exist: check each for matching subjects and sufficient verbs

Present to user:

```markdown
## ServiceAccount & RoleBinding Analysis

**ServiceAccount:** [sa-name] (namespace: [namespace])
| Field | Value |
|-------|-------|
| Exists | Yes |
| Created | [timestamp] |
| Secrets | [count] |
| Image Pull Secrets | [count] |

**RoleBindings in [namespace]:**
| RoleBinding | Role | Subjects | Grants Access? |
|-------------|------|----------|----------------|
| [binding-1] | [role-name] | [sa-1, sa-2] | [Yes/No — wrong SA] |
| [binding-2] | [role-name] | [sa-name] | [Missing — binding not found] |

**ClusterRoleBindings (if accessible):**
| ClusterRoleBinding | ClusterRole | Subjects | Grants Access? |
|--------------------|-------------|----------|----------------|
| [binding] | [role] | [subjects] | [Yes/No] |

[If listing RoleBindings is forbidden:]
**Note:** Agent lacks permission to list RoleBindings directly. Absence of the required binding is inferred from the FORBIDDEN errors in pod logs.

**Assessment:**
[e.g., "No RoleBinding grants the metrics-collector ServiceAccount 'list pods' in demo-rbac. The binding was either never created, or was deleted."]

Continue to diagnosis summary? (yes/no)

## RBAC Diagnosis Summary: [deployment-name]

### Root Cause

**Primary Issue:** [e.g., "Missing RoleBinding for ServiceAccount 'metrics-collector' — cannot list pods in namespace 'demo-rbac'"]

| Category | Status | Details |
|----------|--------|---------|
| Pod Running | OK | Pod is scheduled and container is running |
| Pod Ready | FAIL | Readiness probe fails — API access denied |
| ServiceAccount | EXISTS | [sa-name] in [namespace] |
| RoleBinding | MISSING | No binding grants required permissions |
| API Access | DENIED | 403 FORBIDDEN on [verbs] [resources] |

### Causal Chain (Five Whys)

1. **Signal**: Deployment [name] has 0 available replicas (MinimumReplicasUnavailable)
2. **Why?** Pod readiness probe (`kubectl auth can-i list pods`) returns "no"
3. **Why?** ServiceAccount [sa-name] lacks a RoleBinding granting `list` on `pods`
4. **Why?** The required Role/RoleBinding is absent or was deleted
5. **Root Cause**: [e.g., "Missing RBAC resources for this ServiceAccount — the binding was never created or was removed by a cleanup process/GitOps drift"]

### Recommended Actions

**Option A: Create the missing Role and RoleBinding (recommended)**

```bash
# Create the Role
oc create role <sa-name>-pod-reader \
  --verb=get,list,watch \
  --resource=pods \
  -n <namespace>

# Create the RoleBinding
oc create rolebinding <sa-name>-pod-reader-binding \
  --role=<sa-name>-pod-reader \
  --serviceaccount=<namespace>:<sa-name> \
  -n <namespace>

oc create rolebinding <sa-name>-view \
  --clusterrole=view \
  --serviceaccount=<namespace>:<sa-name> \
  -n <namespace>

# Check if the SA now has permission
oc auth can-i list pods -n <namespace> --as=system:serviceaccount:<namespace>:<sa-name>

# Check pod readiness
oc get pods -n <namespace> -l app=<app-label> -o wide

**WAIT for user confirmation before proceeding.**

## Dependencies

### Required MCP Servers
- `openshift` — Kubernetes/OpenShift resource access for Deployments, Pods, ServiceAccounts, Roles, RoleBindings, and Events ([setup](references/prerequisites.md))

### Required MCP Tools
- `resources_get` (from openshift) — Retrieve individual resource details (Deployment, Pod, ServiceAccount, Role, RoleBinding)
- `resources_list` (from openshift) — List resources by kind in a namespace (Deployments, RoleBindings, ClusterRoleBindings)
- `pods_list` (from openshift) — List pods matching label selectors
- `pods_log` (from openshift) — Retrieve container logs for FORBIDDEN error analysis
- `events_list` (from openshift) — Fetch events filtered by involved object

### Related Skills
- `/debug-scc` — If pods are blocked from creation by SCC admission (different from RBAC)
- `/debug-pod` — If pods are crashing due to application issues, not RBAC
- `/debug-network` — If pods can't reach services (network, not API access)

### Reference Documentation
- **Internal:** [references/debugging-patterns.md](references/debugging-patterns.md) — Common error patterns and troubleshooting trees
- **Official:** [Using RBAC - OpenShift](https://docs.openshift.com/container-platform/latest/authentication/using-rbac.html)

## Example Usage

**User**: My deployment `metrics-collector` in namespace `demo-rbac` shows 0/1 available. The pod is running but not ready. Logs show "FORBIDDEN: pods is forbidden". What's wrong?

**Skill response**: The skill checks the pod status (Running but not Ready), examines logs to find FORBIDDEN errors on `list pods`, identifies the ServiceAccount (`metrics-collector`), lists RoleBindings in the namespace and finds none granting the required permissions. It presents a diagnosis showing the missing RoleBinding as root cause, detects a regression pattern if prior remediation attempts were undone, and offers two fix options: create a minimal custom Role+RoleBinding, or bind the broader `view` ClusterRole.