paddles is a very simple JSON-based API used to report teuthology test results. See https://github.com/ceph/paddles
The service runs on a baremetal host, pulpito.front.sepia.ceph.com, deployed via the ceph-cm-ansible role: https://github.com/ceph/ceph-cm-ansible/tree/main/roles/paddles
It is operated by docker service: https://docs.docker.com/engine/reference/commandline/service/, utilizing replication, health checks, and automatic rollbacks.
The 'paddles' db is backed up daily by the backup.sh script on gitbuilder-archive.
Backups are located in gitbuilder-archive:/home/backup/paddles.front.sepia.ceph.com-psql/paddles
For jobs that indicate they're running but aren't, expire_jobs can be used.
The following example would expire any queued jobs 14 days old or older and any running jobs that haven't been updated in 30 minutes.
oc exec -n paddles $(oc get pods -n paddles -o name | grep -v build | head -1) -- pecan expire_jobs config.py -q 14 -r 30
From your workstation or anywhere with a valid teuthology.yaml,
cd ~/src/teuthology source ./virtualenv/bin/activate # Edit docs/_static/create_nodes.py # (paddles_url, machine_type, lab_domain, and machine_index_range) # These can all be found in teuthology.yaml on a teuthology host python docs/_static/create_nodes.py
The paddles API / Web UI services live in Openshift in the paddles namespace. There is a cronjob that checks quay.io/ceph/paddles for a new image every 5 minutes. The ImageStream will automatically deploy any new images. See https://github.com/ceph/sepia-openshift/tree/main/paddles