guardian · lat. custos

The missing terraform plan
for HashiCorp Vault policies.

Write test specifications for your Vault ACL policies, run them offline or against a live Vault instance, and catch misconfigurations, overprivileged access, and policy conflicts — all before they reach production.

Star on GitHub

$ brew install timkrebs/tap/custos copy

~/infra/vault-policies · zsh

$ custos test -f tests/read-secret.yaml Loaded 4 specs from tests/read-secret.yaml Mode: offline · 0 live calls · seed 0x8f2a ✓ developer can read staging/* 3 capabilities · 2ms ✓ developer cannot write staging/* 1 capability · 1ms ✗ developer cannot read prod/* expected: deny · got: read policy.hcl:14 path "secret/data/prod/*" capabilities = ["read"] // widened in PR #412 ✗ ci-bot has bounded token ttl expected ≤ 1h · got 768h FAIL 2 specs / 4 · offline / 0 drift analyzer: 3 warnings (broad_wildcard, missing_sudo_gate) report: .custos/runs/2026-04-23T14-32.json

test specs / project

~120

avg across design partners

time to first fail

< 800ms

offline, single policy

static checks

24

wildcard, ttl, capability, sudo

dependencies

0

single static Go binary

how it works

Write specs. Run them. Ship policies you trust.

custos runs offline against your policy HCL by default — no Vault instance required. Opt into live mode when you want to verify behavior against a real cluster.

1. Specify

Describe who should be able to do what, in YAML. One file per policy or logical unit. Refactor-safe — specs reference policies by path.

2. Run — offline

custos parses your HCL and simulates vault token capabilities. No cluster, no secrets, no network. Runs in CI with a 200ms cold start.

3. Verify — live

Point custos at a live Vault instance to confirm the rendered HCL matches what's actually enforced. Detects drift, ghost policies, and role bindings.

why custos

A widened wildcard is a one-line diff. custos makes it loud.

Vault policy changes look innocent in review. "*" vs "staging/*" is a six-character diff with production-sized consequences. custos turns every policy change into a testable artifact and every review into a verdict.

policies/developer.hcl — diff

capability widened

  path "secret/data/staging/*" {
-   capabilities = ["read", "list"]+   capabilities = ["create", "read", "update", "delete", "list"]  }

  path "secret/data/prod/*" {
-   capabilities = ["deny"]+   capabilities = ["read"]  }

2 spec failures · 3 analyzer warnings · blocks merge

positioning

How custos fits with what you already run.

capability	custos	vault policy fmt	sentinel	opa / conftest
tests capability logic	✓ specs	—	—	generic
runs offline	✓	✓	—	✓
live drift detection	✓	—	—	—
static wildcard / ttl analyzer	✓ 24 rules	—	paid	custom
designed for Vault HCL	✓	✓	✓	—
requires Vault Enterprise	✗ never	✗	yes	✗

HARBOR LABS

westwind.ai

PIER/SEVEN

kinetic

outpost

meridian

Docs CLI reference GitHub Issues Security

02 · Spec format

The .custos.yaml test spec

Describes the authorization outcomes you expect from a Vault ACL policy. One spec file per policy (or per logical role). custos parses the HCL referenced in policy, walks every case, and reports pass / fail.

version: 1 YAML 1.2 offline by default

Extension*.custos.yaml · *.custos.yml

Default locationtests/ or policies/*/tests/

Validated bycustos validate

Schemacustos.dev/spec.v1.json

Annotated example

A real spec for a developer policy that can read staging, list nothing in prod, and never use sudo.

tests/developer.custos.yaml

58 lines

version: 1
policy: policies/developer.hcl
identity:
  display: developer
  entity_aliases:
    - oidc/google/dev@harbor.dev
  group: engineers

assert:
  - name: can read staging secrets
    path: secret/data/staging/api
    capabilities: [read, list]
    expect: allow

  - name: cannot write staging secrets
    path: secret/data/staging/api
    capabilities: [create, update, delete]
    expect: deny

  - name: cannot touch prod
    paths:
      - secret/data/prod/*
      - secret/metadata/prod/*
    capabilities: [read, list, create, update, delete]
    expect: deny

  - name: no sudo on auth backend
    path: auth/token/create-orphan
    capabilities: [sudo]
    expect: deny

  - name: token ttl is bounded
    kind: token
    constraint:
      ttl_max: "1h"
      renewable: true
    expect: allow

live:
  enabled: false        # set true in CI against staging
  cluster: staging
  login:
    method: oidc
    role: developer

analyzer:
  severity_floor: warn
  ignore: [glob_permissive_root]

policy

Relative path to the .hcl file under test. custos reads this file, not a Vault API call — so specs survive cluster outages and run in any clone.

identity

What token the spec simulates. In offline mode this is metadata for the report; in live mode, custos logs in with these credentials to evaluate token capabilities.

assert[]

Each case has path(s), capabilities, and expect: allow | deny. That's the whole assertion grammar — intentionally tiny so specs read like a truth table.

kind: token switches to token-shape assertions (ttl, orphan, policies attached). Use for role-config tests.

live

Off by default. Flipping it on makes custos run the same specs against a real cluster to detect drift between the rendered HCL and what Vault actually enforces.

analyzer

Per-spec tuning for the static analyzer. Rules can be silenced with justification — the comment becomes part of the audit trail.

HCL, for reference

The policy the spec above tests against. Nothing custos-specific in here — standard Vault ACL.

policies/developer.hcl

vault ACL

# Read + list staging secrets
path "secret/data/staging/*" {
  capabilities = ["read", "list"]
}

# Everything under prod is off-limits
path "secret/data/prod/*" {
  capabilities = ["deny"]
}

path "secret/metadata/prod/*" {
  capabilities = ["deny"]
}

# Let devs self-lookup their own token
path "auth/token/lookup-self" {
  capabilities = ["read"]
}

Design choices

why yaml, not hcl

Tests should read like English. HCL is great for policies but noisy for assertion tables. YAML lets the spec scan as a contract — one case per line, expect at the end of each.

why offline first

Most policy bugs are logic bugs, not cluster bugs. Offline mode means every engineer can run the full test suite with zero infra — and CI gets to fail a PR in under a second.

deny beats absence

custos evaluates like Vault does: most-specific path wins, explicit deny overrides anything else. Unwritten paths default to deny. Specs don't need to enumerate the universe.

what specs can't do

They don't test templated policies that interpolate identity metadata at request time, and they can't prove emergent behavior across policy stacks. For that, run custos scan --policies across the whole bundle.

03 · CLI reference

custos — command-line

A single static Go binary. No runtime, no daemon, no sidecar. Outputs pretty-printed text by default, JSON for CI, JUnit XML for the test-reporter your org already uses.

v0.3.0 darwin / linux / windows arm64 / amd64 go 1.22+

Binary size~12 MB

Cold start~180 ms

Exit codes0 pass · 1 fail · 2 error · 3 config

Shell completionzsh · bash · fish

custos test

$ custos test [flags] [spec-files...]

Runs the assertion grammar of one or more *.custos.yaml files against the referenced HCL. Recursively discovers specs under the working directory if none are passed.

-f, --file	path…	Spec file(s). Glob supported. Defaults to */.custos.yaml.
--policies	dir	Root directory to resolve policy: paths against. Defaults to repo root.
--live	bool	Flip specs marked live.enabled on. Needs VAULT_ADDR, VAULT_TOKEN (or OIDC).
--format	enum	text (default) · json · junit · sarif
--filter	regex	Run only specs whose name matches. Like go test -run.
--fail-on	enum	any (default) · error · warn — raise exit threshold.
-v, --verbose	bool	Print every matched path + the decision trace.
--seed	hex	Deterministic ordering for parallel runners. Defaults to commit sha.

custos scan

$ custos scan [flags] [policy-dir]

Runs the static analyzer over every HCL file in the directory. No specs required — works the moment you drop the binary in a policy repo.

--rules	list	Comma-separated rule ids. Defaults to the built-in 24. See the full catalog →
--severity	enum	Floor to print. info · warn · error.
--ignore-file	path	Load .custosignore of "rule_id: reason" pairs.
--format	enum	text · json · sarif (uploadable to GitHub code scanning)

custos validate

$ custos validate [flags]

Type-checks specs against the v1 schema. Useful as a pre-commit hook — catches misspellings (capabilites) and invalid path shapes before test runs.

custos drift

$ custos drift --cluster <name>

Diffs the HCL checked into your repo against what's currently written on a live cluster. Outputs a unified diff per policy. Designed to run nightly, alert on discrepancy.

--cluster	string	Named context from ~/.custos/clusters.yaml.
--only	glob	Only diff policies matching the pattern.
--ignore-absent	bool	Don't flag policies that exist on cluster but not in repo.

custos init

$ custos init

Writes a .custos/ skeleton — config, sample spec, GitHub Actions workflow, pre-commit hook. Detects existing policies and stubs one spec per file.

custos completion

$ custos completion {bash|zsh|fish}

Prints shell completion to stdout. Installation one-liner for zsh: custos completion zsh > ~/.zfunc/_custos.

Output formats

--format json · single failed spec (elided)

machine-readable

{
  "$schema": "custos.dev/run.v1.json",
  "runId": "2026-04-23T14-32",
  "mode": "offline",
  "summary": { "total": 4, "passed": 2, "failed": 2, "warnings": 3 },
  "specs": [
    {
      "name": "developer cannot read prod/*",
      "file": "tests/developer.custos.yaml:26",
      "status": "fail",
      "path": "secret/data/prod/api",
      "expected": "deny",
      "actual":   "read",
      "ruleOrigin": { "file": "policies/developer.hcl", "line": 14 }
    }
  ]
}

Exit codes

0	pass	All specs pass, no analyzer errors (warnings allowed unless --fail-on warn).
1	fail	One or more spec failures or analyzer issues at the configured floor.
2	error	Runtime problem — can't read file, HCL parse error, Vault unreachable.
3	config	Spec schema invalid or unknown flag. Doesn't consume a CI minute.

04 · Security scan

Built-in analyzer rules

Even with zero specs, custos scan flags the most common classes of Vault-policy bug: too-broad wildcards, unbounded token TTLs, missing sudo gates, orphaned deny clauses. Curated by the maintainer, open to PRs.

24 rules · v0.3 SARIF output GitHub Code Scanning ready

Familiesglob · capability · ttl · identity · mount

Severityinfo · warn · error

Runtime~400ms for 200 policies

Ignore file.custosignore

Catalog

error

C001 · glob_permissive_root

Wildcard at root path

Any policy that matches "*" or "secret/*" with more than read, list. Almost always a mistake in copy-pasted starter policies.

✗ path "*" { capabilities = ["create", "update"] }

error

C002 · capability_write_on_metadata

Write on KV metadata

Metadata writes allow version tombstoning and permanent secret destruction. Should be reserved for a narrow kv-admin policy.

✗ path "secret/metadata/*" { capabilities = ["update", "delete"] }

warn

C003 · broad_wildcard

Glob broader than needed

Heuristic: if a policy covers x/y/* but the spec only asserts x/y/api and x/y/web, propose narrowing.

! path "secret/data/staging/*" → try "secret/data/staging/{api,web}"

warn

C004 · missing_sudo_gate

Sudo-required path without sudo

Certain paths (e.g. sys/auth, sys/policies) need the sudo capability. Policy grants create but omits sudo → request will 403 at runtime.

! path "sys/auth/oidc" { capabilities = ["update"] } # missing "sudo"

error

C005 · ttl_unbounded

Role TTL missing or >24h

Token roles with explicit_max_ttl = 0 or above the org policy ceiling. Flagged as error by default; tighten with ttl.max_hours.

✗ role "ci-bot" { explicit_max_ttl = "768h" }

warn

C006 · orphan_token_allowed

Can create orphan tokens

Policy grants update on auth/token/create-orphan. Orphans bypass lease revocation — a compromised caller can persist.

! path "auth/token/create-orphan" { capabilities = ["update"] }

error

C007 · deny_overridden

Deny shadowed by broader allow

A later, broader allow path makes an earlier deny unreachable. Runtime behavior: Vault picks most specific, but this is usually a refactor bug.

✗ deny on "secret/data/prod" shadowed by allow on "secret/data/*"

warn

C008 · mount_mismatch

Path doesn't exist on any mount

Run with --mounts mounts.json to cross-check paths in policies exist on the cluster. Dead paths are usually refactors that forgot to update the policy.

! path "kv-v1/old-team/*" — mount "kv-v1/" disabled 2024-08

how rules evolve

Every rule starts life as a warn. After two releases and real-world feedback, the maintainer promotes to error if the false-positive rate is low enough. Reverse the move is possible — no rule is eternal.

suppression, with receipts

.custosignore takes rule_id: "because ...". The reason is rendered in every report and audit, so silencing a rule is a tracked decision, not a private one.

05 · CI integration

From local edit → merge-blocking check

Three ways to run custos on every PR: GitHub Actions, GitLab CI, and pre-commit. Pick one — they produce the same run record, the same exit code, the same SARIF output.

Actions · GitLab · Jenkins · Circle SARIF · JUnit · JSON

Typical run time20–45s, offline

Cachecustos binary cached by version

Artifactsrun.json, report.sarif, trace.log

GitHub Actions

Drop in .github/workflows/custos.yml. Surfaces failures as annotations + the run summary in the check tab.

.github/workflows/custos.yml

GitHub Actions

name: custos
on:
  pull_request:
    paths: ["policies/**", "tests/**/*.custos.yaml"]

jobs:
  test:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
      security-events: write  # for SARIF upload
    steps:
      - uses: actions/checkout@v4
      - uses: timkrebs/custos-action@v1
        with:
          version: "0.3.0"
      - name: custos scan
        run: custos scan policies/ --format sarif > custos.sarif
      - uses: github/codeql-action/upload-sarif@v3
        with: { sarif_file: custos.sarif }
      - name: custos test
        run: custos test --format junit --fail-on warn > report.xml

pre-commit hook

Run offline specs before the commit even lands. Sub-second for a typical repo.

.pre-commit-config.yaml

pre-commit.com

repos:
  - repo: https://github.com/timkrebs/custos
    rev: v0.3.0
    hooks:
      - id: custos-validate   # schema check
      - id: custos-test       # offline specs
        files: "^(policies|tests)/"

PR check — what reviewers see

custos · failed in 38s

merge blocked

2 passed 2 failed 3 warnings commit 8f2a19c · run 2026-04-23T14-32

developer cannot read prod/*

expected deny, got read · policies/developer.hcl:14

spec

ci-bot has bounded token ttl

ttl_max ≤ 1h · got 768h · policies/ci-bot.hcl:4

spec

C001 wildcard at root path

policies/developer.hcl:2 · path "*" with write capabilities

warn

what we don't do

custos never commits back to your branch, never auto-fixes HCL, never runs in privileged mode. It's a read-only checker.

live-mode tokens

For drift and live specs in CI: use short-lived OIDC tokens against Vault (federated workload identity). No long-lived PATs in secrets. Example workflow is in the docs.

06 · Install

Up and running in 3 minutes

Single binary. Write one spec, run it. No account, no cluster, no telemetry. Walk the stepper below or skip straight to the docs.

1

Install the binary

brew / go / curl / scoop

2

Initialize your repo

custos init

3

Write your first spec

one case at a time

4

Run it locally

custos test

5

Wire up CI

GitHub Actions · GitLab · pre-commit

6

Go live against Vault

optional · OIDC login

The missing terraform planfor HashiCorp Vault policies.

Write specs. Run them. Ship policies you trust.

1. Specify

2. Run — offline

3. Verify — live

A widened wildcard is a one-line diff. custos makes it loud.

How custos fits with what you already run.

The .custos.yaml test spec

Annotated example

policy

identity

assert[]

live

analyzer

HCL, for reference

Design choices

custos — command-line

custos test

custos scan

custos validate

custos drift

custos init

custos completion

Output formats

Exit codes

Built-in analyzer rules

Catalog

Wildcard at root path

Write on KV metadata

Glob broader than needed

Sudo-required path without sudo

Role TTL missing or >24h

Can create orphan tokens

Deny shadowed by broader allow

Path doesn't exist on any mount

From local edit → merge-blocking check

GitHub Actions

pre-commit hook

PR check — what reviewers see

Up and running in 3 minutes

The missing terraform plan
for HashiCorp Vault policies.