quay.ceph.io

quay.ceph.io migrated from AWS to the lab on 2026-09-09. It's Quay (operator-managed) in the quay namespace in OpenShift. Blobs are in an ODF RGW bucket, postgres on a pre-created RBD PVC. See https://github.com/ceph/sepia-openshift/tree/main/quay - MIGRATION.md there is the full history.

Public path: NS1 A record → 192.86.31.16 → edge proxy → the OpenShift route. There is deliberately no Anubis in front of it - registry clients can't answer challenges.

quay-int is a separate namespace and must never be public. It also hosts the proxy-cache orgs (mirror → quay.io, mirror-docker → docker.io) that CI pulls through.

quayadmin. Creds are in 1Password.

The in-pod cert is manual acme.sh - see MIGRATION.md. It does not auto-renew.

cephadm's stored registry login must use the bare hostname. docker normalizes any URL to the hostname; podman matches path components verbatim, so a login stored as quay.ceph.io/ceph/prerelease doesn't cover ceph/prerelease-amd64 and the pull goes out anonymous.

# diagnose without touching secrets
sudo podman login --get-login --authfile /etc/ceph/podman-auth.json quay.ceph.io
# fix: re-run registry-login with url: quay.ceph.io (hostname only)
ceph cephadm registry-login -i login.json

On the Quay side, app pod nginx logs show GET /v2/auth?account=<user> - no account= means the client sent no credentials at all.

Quay presign-redirects blob GETs to RGW. Two requirements, both already set but worth knowing:

• FEATURE_PROXY_STORAGE: true - otherwise clients get redirected to the cluster-internal RGW service, dead from outside
• The DISTRIBUTED_STORAGE_CONFIG hostname must be the full .svc.cluster.local FQDN - the storage-proxy nginx resolver ignores search domains and 404s on short .svc names

Proxy-cache repos are born private, and only authenticated pulls create cache entries. The daily reconcile CronJob (quay-int/proxy-cache.yaml) flips them public - check it ran, or flip the repo public by hand.

The operator force-overrides some keys when the component is managed (e.g. REPO_MIRROR_TLS_VERIFY: true). If a bundle edit doesn't stick, that's why.

Console Logs view, audit tenant:

{log_type="audit"} |= "quay" | json | objectRef_namespace="quay"

See loki.