Sepia Lab Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== braggi{01..22} ======
https://opendcim.engineering.redhat.com/cabnavigator.php?cabinetid=501

===== Summary =====
We have 22 braggi systems.  The Ceph Foundation graciously purchased these late 2019/early 2020 to help reduce our reliance on ephemeral Jenkins slaves in OVH's Openstack public cloud.

===== Hardware Specs =====
|            ^ Count    ^ Manufacturer       ^ Model                                       ^ Capacity        ^ Notes                                                                                                                           ^
^ Chassis    | N/A      | Supermicro         | SYS-1029P-MTR                               | N/A             |                                                                                                                                 |
^ Mainboard  | N/A      | Supermicro         | X11DPL-i                                    | N/A             |                                                                                                                                 |
^ CPU        | 2        | Intel              | Intel(R) Xeon(R) Silver 4214 CPU @ 2.20GHz  | 48 cores total  | [[https://ark.intel.com/content/www/us/en/ark/products/193385/intel-xeon-silver-4214-processor-16-5m-cache-2-20-ghz.html|ARK]]  |
^ RAM        | 8 DIMMs  | Micron Technology  | 36ASF4G72PZ-2G6E1                           | 32GB            | 256GB total                                                                                                                     |
^ SSD        | 1        | Intel              | MZ7KH480HAHQ-00005                          | 480GB           |                                                                                                                                 |
^ SSD        | 1        | Intel              | MZ7LH960HAJR-00005                          | 960GB           | Mounted at /home/jenkins-build                                                                                                  |
^ NIC        | 2 ports  | Intel              | ?                                           | 1Gb             | Unused                                                                                                                          |
^ NIC        | 2 ports  | Intel              | 82599ES                                     | 10Gb            | Port 1 cabled as uplink                                                                                                         |

===== Setting up as Jenkins slaves =====
==== sda race condition ===
There is an incredibly annoying race condition when installing CentOS 9 Stream on these nodes where sda and sdb get flip flopped.  dgalloway tried fixing this with https://github.com/ceph/ceph-cm-ansible/pull/703/commits/1058a3d0ca85f00a0f7a2df416c4a22a2e08883c which did help but not entirely.

You can check if anaconda picked the right disk by pulling up the Java KVM, switching to another terminal (Ctrl+Alt+F2) and checking if ''/mnt/sysroot'' is on the 800GB or 400GB drive.  You want it to be on the 400GB drive.  If it's not, just reboot and try again.

==== How To ====
  - Reimage using Cobbler
  - Once the OS install is done,
    - Run the ceph-cm-ansible ''users'' role
    - Run ''%%ansible-playbook common.yml --tags vars,nagios,monitoring-scripts --limit braggi##%%''
    - Run https://github.com/djgalloway/sepia/blob/master/ansible-playbooks/jenkins-build-home.yml
    - Run https://github.com/ceph/ceph-build/blob/master/ansible/examples/slave.yml
      - ''%%ansible-playbook -M ./library/ -e permanent=True slave.yml --extra-vars '{"labels": "amd64 XXXXX huge sepia x86_64 braggi gigantic"}' --extra-vars '{"token": "XXXXX", "jenkins_credentials_uuid": "jenkins-build", "api_uri": "https://jenkins.ceph.com"}' --limit braggiXX*%%''
Sepia Lab Wiki

User Tools

Site Tools

Page Tools
