User Tools

Site Tools


hardware:reesi

This is an old revision of the document!


reesi{001..006}

Summary

The Sepia lab has 6 storage hosts purchased January 2018 to replace the miras used in the LONGRUNNINGCLUSTER.

Purchasing details

Hardware Specs

Count Manufacturer Model Capacity Notes
Chassis 2U Supermicro SSG-6028R-E1CR12H N/A
Mainboard N/A Supermicro X10DRH-iT N/A
CPU 1 Intel Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz N/A ARK
RAM 4 DIMMs Samsung M393A2G40EB1-CRC 16GB 64GB total
SSD 2 Intel SSDSC2BB150G7 (S3520) 150GB Software RAID1 for OS
HDD 12 Seagate ST4000NM0025 4TB SAS 7200RPM for OSDs
NVMe 1 Micron MTFDHBG800MCG-1AN1ZABYY 800GB Carved up as logical volumes on two partitions. 400GB as an OSD and the other 400GB divided by 12 for HDD OSD journals
NIC 2 ports Intel X540-AT2 10Gb RJ45 (not used)
NIC 2 ports Intel 82599ES 10Gb 1 port cabled per system on front VLAN
BMC 1 Supermicro N/A N/A Reachable at $host.ipmi.sepia.ceph.com

NVMe Partitions/Provisioning

Here's my bash history that can be used to set up a reesi machine's NVMe card.

ansible -a "sudo parted -s /dev/nvme0n1 mktable gpt" reesi001*
ansible -a "sudo parted /dev/nvme0n1 unit '%' mkpart foo 0 50" reesi001*
ansible -a "sudo parted /dev/nvme0n1 unit '%' mkpart foo 51 100" reesi001*
ansible -a "sudo pvcreate /dev/nvme0n1p1" reesi001*
ansible -a "sudo pvcreate /dev/nvme0n1p2" reesi001*
ansible -a "sudo vgcreate journals /dev/nvme0n1p1" reesi001*
ansible -a "sudo vgcreate osd /dev/nvme0n1p2" reesi001*
for disk in sd{a..l}; do ansible -a "sudo lvcreate -L 31G -n $disk journals" reesi001*; done

Set up a new OSD with journal on NVMe logical volume

ceph-volume lvm create --bluestore --data /dev/sda --block.db journals/sda

Example of successfully deployed OSD

root@reesi001:~# ceph-volume lvm list


====== osd.94 ======

  [block]    /dev/ceph-28f7427e-5558-4ffd-ae1a-51ec3042759a/osd-block-63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9

      type                      block
      osd id                    94
      cluster fsid              28f7427e-5558-4ffd-ae1a-51ec3042759a
      cluster name              ceph
      osd fsid                  63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
      db device                 /dev/journals/sda
      encrypted                 0
      db uuid                   X2SlQ5-0zx2-CuHJ-GEbJ-5JS8-ly5O-emmFI9
      cephx lockbox secret      
      block uuid                Xvjsm3-95vU-KNmw-5DuK-i3cx-fmic-xfSK2w
      block device              /dev/ceph-28f7427e-5558-4ffd-ae1a-51ec3042759a/osd-block-63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
      crush device class        None

  [  db]    /dev/journals/sda

      type                      db
      osd id                    94
      cluster fsid              28f7427e-5558-4ffd-ae1a-51ec3042759a
      cluster name              ceph
      osd fsid                  63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
      db device                 /dev/journals/sda
      encrypted                 0
      db uuid                   X2SlQ5-0zx2-CuHJ-GEbJ-5JS8-ly5O-emmFI9
      cephx lockbox secret      
      block uuid                Xvjsm3-95vU-KNmw-5DuK-i3cx-fmic-xfSK2w
      block device              /dev/ceph-28f7427e-5558-4ffd-ae1a-51ec3042759a/osd-block-63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
      crush device class        None

Checking NVMe Card SMART Data

nvme smart-log /dev/nvme0n1

Updating BIOS

TBD

hardware/reesi.1518818572.txt.gz · Last modified: by djgalloway