In Quay's web UI, in the “Pull this container with the following Podman command:” fields, you'll see commands like this:
podman pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph
These will work just fine when connected to the Sepia VPN. A public endpoint is available at quay.ceph.io, so you may pull images without the VPN like so:
podman pull quay.ceph.io/ceph-ci/ceph
This is a VM in RHEV that was created because quay.io was having lots of issues the week of 25MAY2020 which broke the lab (https://tracker.ceph.com/issues/45343).
SSH only listens on the front interface at quay.front.sepia.ceph.com.
Due to the potentially large number of container images we'd be storing, I opted to use the longrunningcluster which is mounted at /lrc.
I just chose quay because it was the same software/tool that was already in use. Figured it'd be easy to just s/quay.io/quay.ceph.io wherever needed.
Passwords are in magna001.ceph.redhat.com:/root/secrets. I reused the same username (Dan's), password, and repo name so using the new registry would be plug-n-play with our CI.
I had some trouble getting the containers to communicate with one another. The Quay docs don't cover setting up the br_netfilter kernel module or firewall rules so I wrote https://access.redhat.com/solutions/5254621.
From dmick, 3Nov22: apparently this has changed a bit; it looks like the client.container auth doesn't exist anymore; rather, client.admin is used. Also, the cluster path is /containers/quay
## On reesi001 ceph auth add client.containers mds 'allow rw path=/containers' mon 'allow r' osd 'allow rw pool=data' ceph auth get client.containers # Copy the key output ## On quay.front.sepia.ceph.com # run the ansible_managed and common roles yum localinstall http://download.ceph.com/rpm-octopus/el7/noarch/ceph-release-1-1.el7.noarch.rpm yum install ceph-common mkdir /lrc echo "172.21.2.201,172.21.2.202,172.21.2.203:/containers/mirror /lrc/ ceph name=containers,secretfile=/etc/ceph/secret,_netdev 0 2" >> /etc/fstab echo "KEY_FROM_REESI001" > /etc/ceph/secret mount -a # Then I just followed https://access.redhat.com/documentation/en-us/red_hat_quay/3.3/html/deploy_red_hat_quay_-_basic/preparing_for_red_hat_quay_basic
Since the quay container listens on port 80 and 443, we have to temporarily stop it to renew the cert. To avoid doing this too frequently, I have it done on the first Saturday of even-numbered months early in the morning when traffic should be minimal.
[root@quay ~]# crontab -l
# On the first Saturday of Feb,Apr,Jun,Aug,Oct,Dec, renew quay cert
0 4 * 2,4,6,8,10,12 6 [ $(date +\%d) -le 06 ] && /root/bin/quay-cert-renew.sh
[root@quay ~]# cat /root/bin/quay-cert-renew.sh
#!/bin/bash
for container in $(docker ps | grep "quay\.io" | awk '{ print $1 }'); do docker stop $container; done
certbot renew
docker run --restart=always -p 443:8443 -p 80:8080 --sysctl net.core.somaxconn=4096 --privileged=true -v /etc/quay:/conf/stack:Z -v /lrc:/datastorage/registry:Z -d quay.io/redhat/quay:v3.3.0