====== docker-mirror.front.sepia.ceph.com ======
===== Summary =====
This is a VM in [[services:RHEV]] that is a caching mirror for docker.io. We were being rate-limited by docker.io (https://tracker.ceph.com/issues/45343).
I was originally going to create an Ansible playbook to set this up but it was easy enough it wasn't worth the time.
https://docs.docker.com/registry/recipes/mirror/
===== Setup Commands =====
## On reesi001
ceph auth add client.containers mds 'allow rw path=/containers' mon 'allow r' osd 'allow rw pool=data'
ceph auth get client.containers
# Copy the key output
## On docker-mirror.front.sepia.ceph.com
wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -
apt update
apt install ceph-common
mkdir /lrc
echo "172.21.2.201,172.21.2.202,172.21.2.203:/containers/mirror /lrc/ ceph name=containers,secretfile=/etc/ceph/secret,_netdev 0 2" >> /etc/fstab
echo "KEY_FROM_REESI001" > /etc/ceph/secret
mount -a
apt install docker.io
docker run -it --rm --entrypoint cat registry:2 /etc/docker/registry/config.yml > /lrc/config.yml
# Used example from https://www.cloudkb.net/configure-docker-local-registry-proxy-cache/
# Then used parts of https://medium.com/@ifeanyiigili/how-to-setup-a-private-docker-registry-with-a-self-sign-certificate-43a7407a1613
mkdir /lrc/certs
openssl req -newkey rsa:4096 -nodes -sha256 -keyout /lrc/certs/domain.key -x509 -days 3650 -addext "subjectAltName = DNS:docker-mirror.front.sepia.ceph.com" -out /lrc/certs/domain.crt
docker run -d --restart=always -p 5000:5000 --name registry-mirror -e REGISTRY_HTTP_TLS_CERTIFICATE=/var/lib/registry/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/var/lib/registry/certs/domain.key -v /lrc:/var/lib/registry registry:2 /var/lib/registry/config.yml
Super simple.
===== Using the mirror =====
# Example using grafana
podman pull --tls-verify=false docker-mirror.front.sepia.ceph.com:5000/grafana/grafana
Or you can also now just run the [[https://github.com/ceph/ceph-cm-ansible/tree/master/roles/container-host|container-host role]] which will configure ''podman'' and/or ''docker'' to use our mirror for docker.io.
===== Admin Tasks =====
==== Renewing the SSL certiciate ====
The first time I created the cert, I accidentally left the date out so the cert was only good for a month. The second cert is good for 10 years.
- ''%%root@docker-mirror:~# openssl req -newkey rsa:4096 -nodes -sha256 -keyout /lrc/certs/domain.key -x509 -days 3650 -addext "subjectAltName = DNS:docker-mirror.front.sepia.ceph.com" -out /lrc/certs/domain.crt%%''
- Copy the contents of ''/lrc/certs/domain.crt'' and update ''container_mirror_cert'' in https://github.com/ceph/ceph-sepia-secrets/blob/master/ansible/inventory/group_vars/all.yml
- Run ''ansible-playbook container-host.yml'' against the appropriate hosts.
- ''%%root@docker-mirror:~# docker stop registry-mirror; docker rm registry-mirror; docker run -d --restart=always -p 5000:5000 --name registry-mirror -e REGISTRY_HTTP_TLS_CERTIFICATE=/var/lib/registry/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/var/lib/registry/certs/domain.key -v /lrc:/var/lib/registry registry:2 /var/lib/registry/config.yml; docker logs -f registry-mirror%%''
===== Other Notes =====
The mirror logs in to dockerhub using my personal API key. I just have a personal account but docker-mirror was getting rate-limited when anonymous.
root@docker-mirror:~# cat /lrc/config.yml
version: 0.1
log:
fields:
service: registry
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/registry
http:
addr: :5000
headers:
X-Content-Type-Options: [nosniff]
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
proxy:
remoteurl: https://registry-1.docker.io
username: XXXXX
password: XXXXX