hardware:reesi
This is an old revision of the document!
Table of Contents
reesi{001..006}
Summary
The Sepia lab has 6 storage hosts purchased December 2017 to replace the miras used in the LONGRUNNINGCLUSTER.
Purchasing details
https://docs.google.com/spreadsheets/d/1-CqEo-927duC0sClr78WcZgVIFdY55iE_K8oU2G7aSw
Racking ticket: https://redhat.service-now.com/surl.do?n=PNT0117459
Hardware Specs
| Count | Manufacturer | Model | Capacity | Notes | |
|---|---|---|---|---|---|
| Chassis | 2U | Supermicro | SSG-6028R-E1CR12H | N/A | |
| Mainboard | N/A | Supermicro | X10DRH-iT | N/A | |
| CPU | 1 | Intel | Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz | N/A | ARK |
| RAM | 4 DIMMs | Samsung | M393A2G40EB1-CRC | 16GB | 64GB total |
| SSD | 2 | Intel | SSDSC2BB150G7 (S3520) | 150GB | Software RAID1 for OS |
| HDD | 12 | Seagate | ST4000NM0025 | 4TB | SAS 7200RPM for OSDs |
| NVMe | 1 | Micron | MTFDHBG800MCG-1AN1ZABYY | 800GB | Carved up as logical volumes on two partitions. 400GB as an OSD and the other 400GB divided by 12 for HDD OSD journals |
| NIC | 2 ports | Intel | X540-AT2 | 10Gb | RJ45 (not used) |
| NIC | 2 ports | Intel | 82599ES | 10Gb | 1 port cabled per system on front VLAN |
| BMC | 1 | Supermicro | N/A | N/A | Reachable at $host.ipmi.sepia.ceph.com |
NVMe Partitions/Provisioning
Here's my bash history that can be used to set up a reesi machine's NVMe card.
ansible -a "sudo parted -s /dev/nvme0n1 mktable gpt" reesi*
ansible -a "sudo parted /dev/nvme0n1 unit '%' mkpart foo 0 50" reesi*
ansible -a "sudo parted /dev/nvme0n1 unit '%' mkpart foo 51 100" reesi*
ansible -a "sudo pvcreate /dev/nvme0n1p1" reesi*
ansible -a "sudo vgcreate journals /dev/nvme0n1p1" reesi*
for disk in sd{a..l}; do ansible -a "sudo lvcreate -L 31G -n $disk journals" reesi*; done
Set up a new OSD with journal on NVMe logical volume
ceph-volume lvm create --bluestore --data /dev/sda --block.db journals/sda
Example of successfully deployed OSD
root@reesi001:~# ceph-volume lvm list
====== osd.94 ======
[block] /dev/ceph-28f7427e-5558-4ffd-ae1a-51ec3042759a/osd-block-63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
type block
osd id 94
cluster fsid 28f7427e-5558-4ffd-ae1a-51ec3042759a
cluster name ceph
osd fsid 63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
db device /dev/journals/sda
encrypted 0
db uuid X2SlQ5-0zx2-CuHJ-GEbJ-5JS8-ly5O-emmFI9
cephx lockbox secret
block uuid Xvjsm3-95vU-KNmw-5DuK-i3cx-fmic-xfSK2w
block device /dev/ceph-28f7427e-5558-4ffd-ae1a-51ec3042759a/osd-block-63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
crush device class None
[ db] /dev/journals/sda
type db
osd id 94
cluster fsid 28f7427e-5558-4ffd-ae1a-51ec3042759a
cluster name ceph
osd fsid 63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
db device /dev/journals/sda
encrypted 0
db uuid X2SlQ5-0zx2-CuHJ-GEbJ-5JS8-ly5O-emmFI9
cephx lockbox secret
block uuid Xvjsm3-95vU-KNmw-5DuK-i3cx-fmic-xfSK2w
block device /dev/ceph-28f7427e-5558-4ffd-ae1a-51ec3042759a/osd-block-63c0a64e-a0d2-4daf-87ec-c4b00b9f3ab9
crush device class None
Checking NVMe Card SMART Data
nvme smart-log /dev/nvme0n1
Updating BIOS
TBD
hardware/reesi.1519321919.txt.gz · Last modified: by djgalloway
