Table of Contents

braggi{01..22}

https://opendcim.engineering.redhat.com/cabnavigator.php?cabinetid=501

Summary

We have 22 braggi systems. The Ceph Foundation graciously purchased these late 2019/early 2020 to help reduce our reliance on ephemeral Jenkins slaves in OVH's Openstack public cloud.

Hardware Specs

Count Manufacturer Model Capacity Notes
Chassis N/A Supermicro SYS-1029P-MTR N/A
Mainboard N/A Supermicro X11DPL-i N/A
CPU 2 Intel Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz 48 cores total ARK
RAM 8 DIMMs Micron Technology 36ASF4G72PZ-2G6E1 32GB 256GB total
SSD 1 Intel MZ7KH480HAHQ-00005 480GB
SSD 1 Intel MZ7LH960HAJR-00005 960GB Mounted at /home/jenkins-build
NIC 2 ports Intel ? 1Gb Unused
NIC 2 ports Intel 82599ES 10Gb Port 1 cabled as uplink

Setting up as Jenkins slaves

sda race condition

There is an incredibly annoying race condition when installing CentOS 9 Stream on these nodes where sda and sdb get flip flopped. dgalloway tried fixing this with https://github.com/ceph/ceph-cm-ansible/pull/703/commits/1058a3d0ca85f00a0f7a2df416c4a22a2e08883c which did help but not entirely.

You can check if anaconda picked the right disk by pulling up the Java KVM, switching to another terminal (Ctrl+Alt+F2) and checking if /mnt/sysroot is on the 800GB or 400GB drive. You want it to be on the 400GB drive. If it's not, just reboot and try again.

How To

  1. Reimage using Cobbler
  2. Once the OS install is done,
    1. Run the ceph-cm-ansible users role
    2. Run ansible-playbook common.yml --tags vars,nagios,monitoring-scripts --limit braggi##
    3. Run https://github.com/djgalloway/sepia/blob/master/ansible-playbooks/jenkins-build-home.yml
    4. Run https://github.com/ceph/ceph-build/blob/master/ansible/examples/slave.yml
      1. ansible-playbook -M ./library/ -e permanent=True slave.yml --extra-vars '{"labels": "amd64 XXXXX huge sepia x86_64 braggi gigantic"}' --extra-vars '{"token": "XXXXX", "jenkins_credentials_uuid": "jenkins-build", "api_uri": "https://jenkins.ceph.com"}' --limit braggiXX*