Dev.to Security 🔐 Cybersecurity 👁 0 📖 9 min read

OpenShift SecurityContextConstraints: why your upstream Helm chart refuses to start

You helm install a popular chart that works perfectly on EKS, GKE or a kind cluster. On OpenShift, the Deployment is created, the ReplicaSet is created... and no Pod ever appears. Or the Pod appears and crashloops with P

You helm install a popular chart that works perfectly on EKS, GKE or a kind cluster. On OpenShift, the Deployment is created, the ReplicaSet is created... and no Pod ever appears. Or the Pod appears and crashloops with Permission denied on a directory it just tried to write to.

Welcome to SecurityContextConstraints (SCC). This article walks through the admission mechanics, how to diagnose exactly which SCC was applied (or refused), how to adapt an upstream chart without forking it, and — when there is no other option — how to create a narrowly scoped custom SCC instead of handing out anyuid to everyone.

Context: why public charts break on OpenShift

Most community charts are written against a permissive baseline. They assume:

  • the container can run as whatever UID the image declares (often 0, sometimes a hardcoded 1001),
  • fsGroup can be any value,
  • writing to /var/lib/<app> or /data inside the image layer is fine,
  • binding to port 80 or 443 inside the container is fine.

OpenShift denies all of that by default. Since OpenShift 4.11, every authenticated user and ServiceAccount is bound to the restricted-v2 SCC through the system:openshift:scc:restricted-v2 ClusterRole. restricted-v2 enforces:

  • runAsUser: MustRunAsRange — the UID must come from the namespace's allocated range, not from the image or the manifest,
  • fsGroup: MustRunAs — same, from the namespace range,
  • requiredDropCapabilities: ALL, with only NET_BIND_SERVICE optionally addable,
  • allowPrivilegeEscalation: false (no no_new_privs bypass, no setuid),
  • seccompProfiles: runtime/default,
  • seLinuxContext: MustRunAs with a per-namespace MCS label,
  • a tight volumes allowlist (no hostPath).

This is not Pod Security Admission (PSA). PSA also runs on OpenShift 4.11+ and will emit warnings, but SCC is the thing that actually mutates and rejects your Pod. Understanding which of the two is complaining saves a lot of time.

How SCC admission actually works

SCC is a mutating and validating admission plugin. For every Pod creation it:

  1. Collects the SCCs the requesting identity (the ServiceAccount of the Pod, plus the user who created the controller) may use.
  2. Sorts them: highest priority first, then "most restrictive" first, then alphabetically.
  3. For each SCC in order, it tries to mutate the Pod to satisfy the constraints (fill in runAsUser, fsGroup, SELinux labels, drop capabilities) and then validate it.
  4. The first SCC that produces a valid Pod wins, and its name is stamped into the openshift.io/scc annotation.
  5. If none matches, the Pod is rejected.

Two consequences that trip people up:

  • The ServiceAccount matters, not your user. When a Deployment creates Pods, the ReplicaSet controller acts on behalf of the Pod's ServiceAccount. If you gave yourself cluster-admin, your oc run test will pass while the Deployment still fails.
  • Explicit beats implicit. If the chart hardcodes runAsUser: 1001, SCC can no longer mutate it to a value in the namespace range, so restricted-v2 validation fails hard.

Where the UID range comes from

Each namespace gets an allocated block at creation time:

oc get ns myapp -o jsonpath='{.metadata.annotations}' | jq
{
  "openshift.io/sa.scc.mcs": "s0:c26,c10",
  "openshift.io/sa.scc.supplemental-groups": "1000680000/10000",
  "openshift.io/sa.scc.uid-range": "1000680000/10000"
}

So in this namespace, valid UIDs are 1000680000–1000689999. A chart asking for 1001 is out of range, full stop.

Diagnosing: which SCC was applied, and why it was refused

Step 1 — read the real error

When no Pod exists, the error is on the ReplicaSet, not the Deployment:

oc get events -n myapp --sort-by=.lastTimestamp | tail -20
oc describe rs -n myapp -l app.kubernetes.io/name=myapp

A typical message:

Error creating: pods "myapp-7d9f8c5b4-" is forbidden: unable to validate
against any security context constraint:
[provider "restricted-v2": .spec.securityContext.fsGroup: Invalid value:
[]int64{1001}: 1001 is not an allowed group,
provider "restricted-v2": .containers[0].runAsUser: Invalid value: 1001:
must be in the ranges: [1000680000, 1000689999]]

That message lists every SCC that was tried and why each one refused. Read it in full; it is the most useful output OpenShift gives you here.

Step 2 — check what was granted when a Pod does start

oc get pod -n myapp -o custom-columns=\
'NAME:.metadata.name,SCC:.metadata.annotations.openshift\.io/scc,UID:.spec.securityContext.runAsUser'

If you see anyuid or privileged here, someone already widened the cluster — that is a finding, not a fix.

Step 3 — simulate before deploying

scc-subject-review answers "which SCC would this Pod spec get, for this ServiceAccount?":

helm template myapp ./chart --namespace myapp > /tmp/rendered.yaml

oc adm policy scc-subject-review \
  -n myapp \
  -z system:serviceaccount:myapp:myapp \
  -f /tmp/rendered.yaml
RESOURCE                 ALLOWED BY
deployment.apps/myapp    <none>

<none> means rejection. After a fix, you want to see restricted-v2.

To list what a ServiceAccount is allowed to use:

oc adm policy who-can use securitycontextconstraints.security.openshift.io/anyuid -n myapp
oc get scc
oc describe scc restricted-v2

Adapting the chart (the right first move)

Roughly 80% of failures are fixed with values, no fork required.

Remove hardcoded UIDs

Most well-maintained charts expose podSecurityContext and securityContext. The goal is to delete the UID keys so SCC can inject namespace-valid values. In Helm, null removes a key during values merge:

# values-openshift.yaml
podSecurityContext:
  runAsUser: null
  runAsGroup: null
  fsGroup: null
  seccompProfile:
    type: RuntimeDefault

securityContext:
  runAsUser: null
  runAsNonRoot: true
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  capabilities:
    drop:
      - ALL
helm upgrade --install myapp ./chart -n myapp -f values-openshift.yaml

Note runAsNonRoot: true is kept: it is compatible with restricted-v2 and documents intent. Keep seccompProfile: RuntimeDefault too — it satisfies both SCC and PSA restricted.

If the chart hardcodes the UID in templates/ instead of values.yaml, use a Kustomize post-renderer or a strategic patch rather than forking:

helm upgrade --install myapp ./chart -n myapp \
  --post-renderer ./kustomize-wrapper.sh

Fix writable paths

readOnlyRootFilesystem: true plus a random UID means the app cannot write anywhere it expects. Mount emptyDir volumes on the paths it needs:

extraVolumes:
  - name: tmp
    emptyDir: {}
  - name: cache
    emptyDir: {}
extraVolumeMounts:
  - name: tmp
    mountPath: /tmp
  - name: cache
    mountPath: /var/cache/myapp

For PersistentVolumeClaims, ownership comes from fsGroup, which SCC sets for you — as long as you did not hardcode it. If the image stores state in a directory baked into the image layer with root:root 0755, no fsGroup will help. Either rebuild the image with group-0 ownership (chgrp -R 0 /data && chmod -R g=u /data, the OpenShift image convention) or move the state to a mounted volume.

Fix privileged ports

Change the container port to something >= 1024 and let the Service do the mapping:

service:
  port: 443
containerPort: 8443

restricted-v2 does allow requesting NET_BIND_SERVICE, but with allowPrivilegeEscalation: false file capabilities are ignored, so relying on it is fragile. Changing the port is cheaper and more portable.

Check the image

skopeo inspect docker://docker.io/library/nginx:1.27 | jq '.Config.User, .Config.ExposedPorts'

An empty User means the image runs as root. Many projects now publish a -nonroot, -unprivileged or UBI-based variant (nginxinc/nginx-unprivileged, bitnami/* images are UID-agnostic). Switching the image tag is often the entire fix.

When you genuinely need a custom SCC

Some workloads cannot be made UID-agnostic: proprietary binaries that getpwuid() on startup, legacy apps with baked-in /etc/passwd entries, CNI or storage agents that need hostPath.

The wrong answer is oc adm policy add-scc-to-user anyuid -z myapp. anyuid allows any UID including 0, cluster-wide semantics, and is an audit red flag. Build a minimal SCC that grants exactly the one deviation you need.

Example: allow one fixed non-root UID

apiVersion: security.openshift.io/v1
kind: SecurityContextConstraints
metadata:
  name: legacy-fixed-uid-1001
  annotations:
    kubernetes.io/description: >-
      Like restricted-v2 but pins runAsUser to 1001 for images that cannot
      run with a random UID. Root is still forbidden.
allowHostDirVolumePlugin: false
allowHostIPC: false
allowHostNetwork: false
allowHostPID: false
allowHostPorts: false
allowPrivilegeEscalation: false
allowPrivilegedContainer: false
allowedCapabilities: []
defaultAddCapabilities: []
requiredDropCapabilities:
  - ALL
runAsUser:
  type: MustRunAs
  uid: 1001
fsGroup:
  type: MustRunAs
  ranges:
    - min: 1001
      max: 1001
supplementalGroups:
  type: RunAsAny
seLinuxContext:
  type: MustRunAs
seccompProfiles:
  - runtime/default
readOnlyRootFilesystem: false
volumes:
  - configMap
  - csi
  - downwardAPI
  - emptyDir
  - ephemeral
  - persistentVolumeClaim
  - projected
  - secret
users: []
groups: []
priority: null

Two important details: users: [] and groups: [] — never put subjects inline. Grant access through RBAC so it is reviewable, namespaced and GitOps-friendly:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: scc-use-legacy-fixed-uid-1001
rules:
  - apiGroups: ["security.openshift.io"]
    resources: ["securitycontextconstraints"]
    resourceNames: ["legacy-fixed-uid-1001"]
    verbs: ["use"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: scc-use-legacy-fixed-uid-1001
  namespace: myapp
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: scc-use-legacy-fixed-uid-1001
subjects:
  - kind: ServiceAccount
    name: myapp
    namespace: myapp

A RoleBinding (not ClusterRoleBinding) scopes the grant to a single ServiceAccount in a single namespace. That is the whole point.

Make sure the chart actually uses a dedicated ServiceAccount:

serviceAccount:
  create: true
  name: myapp

Apply and verify:

oc apply -f scc-legacy-fixed-uid.yaml
oc apply -f rbac-scc.yaml
oc rollout restart deploy/myapp -n myapp

oc get pod -n myapp -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.metadata.annotations.openshift\.io/scc}{"\n"}{end}'
oc rsh deploy/myapp id

If the Pod still picks restricted-v2, that is good news: the SCC selection found a more restrictive match and your custom SCC was not needed.

Security best practices

  • Never grant privileged or anyuid cluster-wide. oc adm policy add-scc-to-group anyuid system:authenticated is a cluster compromise waiting to happen. Audit with oc get clusterrolebinding -o json | jq '.items[] | select(.roleRef.name | startswith("system:openshift:scc:"))'.
  • One ServiceAccount per workload. Never attach an SCC to default; every Pod in the namespace that omits serviceAccountName would inherit it.
  • Grant SCCs with RBAC, not with the users/groups fields. RBAC bindings are namespaced, visible in oc describe rolebinding, and reversible in Git.
  • Derive custom SCCs from restricted-v2, not from privileged. Start from the strictest and relax one field. oc get scc restricted-v2 -o yaml > base.yaml is a good starting point.
  • Keep allowPrivilegeEscalation: false, requiredDropCapabilities: [ALL], allowHostNetwork/PID/IPC: false in every custom SCC unless you can name the exact syscall you need.
  • Align Pod Security Admission labels on the namespace so SCC and PSA agree, otherwise you get confusing warnings:
  oc label ns myapp \
    pod-security.kubernetes.io/enforce=restricted \
    pod-security.kubernetes.io/audit=restricted \
    pod-security.kubernetes.io/warn=restricted --overwrite
  • Treat a custom SCC as technical debt. Record why it exists in the kubernetes.io/description annotation and open an upstream issue or PR on the chart — "support arbitrary UIDs" is a fix that benefits everyone.
  • Gate it in CI. Run oc adm policy scc-subject-review against rendered manifests in your pipeline so a chart bump never silently requires more privileges.

Conclusion

The SCC rejection is not OpenShift being awkward; it is OpenShift telling you the workload assumes privileges it does not need. The diagnostic path is short and repeatable:

  1. Read the full unable to validate against any security context constraint message on the ReplicaSet — it names every SCC and the exact field that failed.
  2. Check the namespace's openshift.io/sa.scc.uid-range annotation to know what UIDs are legal.
  3. Use oc adm policy scc-subject-review -z <sa> -f rendered.yaml to simulate before deploying.
  4. Fix the chart first: null out runAsUser/runAsGroup/fsGroup, add emptyDir for writable paths, move off privileged ports, prefer -unprivileged image variants.
  5. Only if the image is genuinely UID-bound, write a custom SCC derived from restricted-v2, keep users/groups empty, and bind it with a namespaced RoleBinding to a dedicated ServiceAccount.

The failure mode to avoid is the one-line oc adm policy add-scc-to-user anyuid that unblocks the demo and stays in the cluster for three years. Ten minutes of values tuning usually gets the same chart running under restricted-v2 — and then it runs the same way on every other Kubernetes distribution with Pod Security Admission turned on.

📰 Read the original article on Dev.to Security

Originally published by Dev.to Security. Aggregated on AIWithGhost for educational purposes — full credit and traffic to the original publisher.