User Tools

Site Tools


Sidebar

General Lab Info (Mainly for Devs)

Hardware

Lab Infrastructure Services

Misc Admin Tasks
These are infrequently completed tasks that don't fit under any specific service

Production Services

OVH = OVH
RHEV = Sepia RHE instance
Baremetal = Host in Sepia lab

The Attic/Legacy Info

services:docker-mirror

docker-mirror.front.sepia.ceph.com

Summary

This is a VM in RHEV that is a caching mirror for docker.io. We were being rate-limited by docker.io (https://tracker.ceph.com/issues/45343).

I was originally going to create an Ansible playbook to set this up but it was easy enough it wasn't worth the time.

https://docs.docker.com/registry/recipes/mirror/

Setup Commands

## On reesi001
ceph auth add client.containers mds 'allow rw path=/containers' mon 'allow r' osd 'allow rw pool=data'
ceph auth get client.containers
# Copy the key output

## On docker-mirror.front.sepia.ceph.com
wget -q -O- 'https://download.ceph.com/keys/release.asc' | sudo apt-key add -
apt update
apt install ceph-common
mkdir /lrc
echo "172.21.2.201,172.21.2.202,172.21.2.203:/containers/mirror    /lrc/           ceph    name=containers,secretfile=/etc/ceph/secret,_netdev 0 2" >> /etc/fstab
echo "KEY_FROM_REESI001" > /etc/ceph/secret 
mount -a
apt install docker.io
docker run -it --rm --entrypoint cat registry:2 /etc/docker/registry/config.yml > /lrc/config.yml
# Used example from https://www.cloudkb.net/configure-docker-local-registry-proxy-cache/

# Then used parts of https://medium.com/@ifeanyiigili/how-to-setup-a-private-docker-registry-with-a-self-sign-certificate-43a7407a1613
mkdir /lrc/certs
openssl req  -newkey rsa:4096 -nodes -sha256 -keyout /lrc/certs/domain.key -x509 -days 3650 -addext "subjectAltName = DNS:docker-mirror.front.sepia.ceph.com" -out /lrc/certs/domain.crt

docker run -d --restart=always -p 5000:5000 --name registry-mirror -e REGISTRY_HTTP_TLS_CERTIFICATE=/var/lib/registry/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/var/lib/registry/certs/domain.key -v /lrc:/var/lib/registry registry:2 /var/lib/registry/config.yml

Super simple.

Using the mirror

# Example using grafana
podman pull --tls-verify=false docker-mirror.front.sepia.ceph.com:5000/grafana/grafana

Or you can also now just run the container-host role which will configure podman and/or docker to use our mirror for docker.io.

Admin Tasks

Renewing the SSL certiciate

The first time I created the cert, I accidentally left the date out so the cert was only good for a month. The second cert is good for 10 years.

  1. root@docker-mirror:~# openssl req -newkey rsa:4096 -nodes -sha256 -keyout /lrc/certs/domain.key -x509 -days 3650 -addext "subjectAltName = DNS:docker-mirror.front.sepia.ceph.com" -out /lrc/certs/domain.crt
  2. Copy the contents of /lrc/certs/domain.crt and update container_mirror_cert in https://github.com/ceph/ceph-sepia-secrets/blob/master/ansible/inventory/group_vars/all.yml
  3. Run ansible-playbook container-host.yml against the appropriate hosts.
  4. root@docker-mirror:~# docker stop registry-mirror; docker rm registry-mirror; docker run -d --restart=always -p 5000:5000 --name registry-mirror -e REGISTRY_HTTP_TLS_CERTIFICATE=/var/lib/registry/certs/domain.crt -e REGISTRY_HTTP_TLS_KEY=/var/lib/registry/certs/domain.key -v /lrc:/var/lib/registry registry:2 /var/lib/registry/config.yml; docker logs -f registry-mirror

Other Notes

The mirror logs in to dockerhub using my personal API key. I just have a personal account but docker-mirror was getting rate-limited when anonymous.

root@docker-mirror:~# cat /lrc/config.yml 
version: 0.1
log:
  fields:
    service: registry
storage:
  cache:
    blobdescriptor: inmemory
  filesystem:
    rootdirectory: /var/lib/registry
http:
  addr: :5000
  headers:
    X-Content-Type-Options: [nosniff]
health:
   storagedriver:
    enabled: true
    interval: 10s
    threshold: 3
proxy:
  remoteurl: https://registry-1.docker.io
  username: XXXXX
  password: XXXXX
services/docker-mirror.txt · Last modified: 2021/01/20 15:50 by djgalloway