Skip to content

Keeper security review (v0.1.0)

This is a review of the controls the design requires (DESIGN.md "Security and encryption"), how they are implemented, and what remains open. Report vulnerabilities privately to the repository owners.

Data at rest and in transit

Control Implementation Verified by
Client-side encryption of every data object zstd → age (X25519) streaming encoder. A backup is encrypted to every store recipient; the private keys are not in Keeper's config. internal/pipeline tests; e2e restores decrypt with the fixture identity only
Integrity SHA-256 of every stored object in the manifest, checked at the end of every read. The manifest is written last, with a conditional PUT. pipeline and store tests; chaos checkInvariants (every manifest's objects verify)
Catalog separation Inventories and masked previews are also encrypted to optional catalog recipients. keeper-api reads them with a catalog identity that cannot decrypt dumps, base backups, WAL or binlogs (ADR 0009). e2e TestCLIAndAPI (inventory and diff through the catalog key)
TLS to stores HTTPS endpoints; http:// only for in-cluster test stores config validation
Downloads Presigned GET URLs expire after 1 h. Only restorers can create them, and every request is audited. API role tests

Secrets

  • Credentials are never in argv or the environment of child tools.
  • MySQL tools read a client options file on /dev/fd/3.
  • Postgres tools read a pgpass file on /dev/fd/3.
  • Both are anonymous, sealed, owner-read-only memory files (memfd), so nothing touches disk.
  • The Go code (pgx, go-sql-driver, pglogrepl) builds connection configs from structs, never from DSN strings.
  • Logs never contain secrets.
  • The slog handler redacts any attribute whose key looks secret.
  • Error messages carry tool stderr tails, never configs.
  • Audit records hold the query text, never results or credentials.
  • Keeper reads only secrets named with the configured prefix (default keeper-) in target namespaces (ADR 0008).
  • Sandbox credentials are random per sandbox. They are kept in the sandbox namespace, with a copy in Keeper's namespace for the query proxy. Revealing them through the API is limited to the owner or a restorer-admin, and is audited.

Access to the console and API

  • Authentication. With auth.provider=cloudflare-access, every request needs a valid Access JWT, checked for RS256, issuer = team domain, audience tag and expiry, against the team's JWKS (refreshed on key rotation). Requests without one get 401. Bypassing the tunnel does not help.
  • Authorization.
  • Roles come from the roles map: viewer < operator < restorer < restorer-admin.
  • Each write checks the role it needs. Restores check it per mode: sandbox needs operator; new database and download need restorer; in place needs restorer-admin plus a typed confirmation.
  • Org/project scopes filter every listing and lookup. Out-of-scope objects return 404.
  • CSRF. Form posts must be same-origin (Sec-Fetch-Site or Origin). The JSON API requires application/json.
  • Browser hardening. CSP default-src 'self' with no inline scripts, nosniff, same-origin referrer. Assets are embedded, with no third-party origins.
  • Query console.
  • It reaches sandboxes only, never live targets.
  • Transactions are read-only by default, with a 30 s server-side timeout, 1,000 rows and 4 KiB per cell.
  • Identifiers in the table browser are allow-listed, and mask patterns apply to results.
  • Every query is audited.

Kubernetes footprint

  • Pods.
  • Keeper pods run as non-root (65532) with a read-only root filesystem, all capabilities dropped and the RuntimeDefault seccomp profile.
  • The controller never moves data; Jobs and the streamer do.
  • Data-path pods mount only emptyDir: no hostPath, PVC by default, FUSE or block devices.
  • RBAC is split per component:
  • controller: Keeper resources, Jobs and Deployments in its namespace, sandbox namespaces.
  • data path: read Keeper resources, update status.
  • API: read everything Keeper, write requests, secrets only in Keeper's namespace.
  • Target namespaces: only get secrets, granted per listed namespace.
  • Sandboxes. Each gets its own namespace, and its NetworkPolicy allows only keeper-api, the tunnel connector, the restorer and pods labelled as sandbox clients. TTLs delete sandboxes, and finalizers never block (removed after a timeout).
  • Database users are least-privilege (docs/sql): read-only plus replication. Postgres needs pg_read_all_data, pg_monitor and REPLICATION. MySQL needs SELECT, SHOW VIEW, TRIGGER, EVENT, LOCK TABLES, PROCESS, RELOAD and REPLICATION CLIENT/SLAVE.

Exposure

Sandbox routes use Cloudflare Tunnel with an Access application per hostname. The app is created before the DNS record, so a route is never reachable unprotected, and it is deleted after the route. The provider refuses to create a route without an Access policy. Live tests can only touch keeper-test-* names and never the production zone.

Supply chain

  • One image, built from a pinned base. Upstream apt repositories are verified with vendored keys.
  • Release images are pushed with SBOM and provenance, and signed keyless with cosign.
  • CLI binaries are published with SHA-256 sums. CI needs one secret (REGISTRY_PASSWORD).

Findings and open items

# Finding Status
1 Postgres tools received the password via PGPASSWORD (visible in /proc/<pid>/environ to the same user) Fixed: pgpass on a memfd
2 MySQL option file passed as a pipe was read once; some tools read option files twice Fixed: memfd (re-readable, sealed, 0400)
3 A PITR target's replication slot was left behind when PITR was turned off or the target deleted (database disk risk) Fixed: cleanup Job and finalizer (ADR 0011)
4 Audit records are kept per API replica until flushed (every minute) to the store; a crash loses at most the last minute of records Accepted. Stdout JSON lines are also collected by the cluster's log pipeline
5 The query console allows writes with an explicit flag (sandboxes are disposable) Accepted, audited. Operators can be restricted by role scopes
6 Without a catalog key, keeper-api falls back to the store identity (can decrypt data) Documented (ADR 0009). Production stores should set catalogRecipients
7 The Cloudflare token for the live test has no permissions yet; the exposure provider is verified against the mock only Open (manual step)