Differences

This shows you the differences between two versions of the page.

--- devplayground [2023/10/17 19:35] – pdonnell
+++ devplayground [2026/09/14 14:23] (current) – djgalloway
@@ Line 8: / Line 8: @@
 ^ Name                                             ^ Notes            ^
-| [[hardware:senta|senta01.front.sepia.ceph.com]]  | General purpose  |
-| [[hardware:senta|senta02.front.sepia.ceph.com]]  | General purpose  |
-| [[hardware:senta|senta03.front.sepia.ceph.com]]  | General purpose  |
 | [[hardware:vossi|vossi01.front.sepia.ceph.com]]  | General purpose  |
 | [[hardware:vossi|vossi04.front.sepia.ceph.com]]  | CephFS Team      |
@@ Line 19: / Line 16: @@
 ==== Playing Nice ====
-Developer playgrounds should be able to build the ''main'' branch. It is okay to use ''./install-deps.sh'' top-level scrip from the ceph source tree to update dependencies. Do not run that script from an older release of Ceph as it may break other developer's work. If you need to build an older release, lock a throwaway node like [[hardware:smithi]] and build there. Or, use a container to do the build/testing!
+Developer playgrounds should be able to build the ''main'' branch. It is okay to use ''./install-deps.sh'' top-level script from the ceph source tree to update dependencies. Do not run that script from an older release of Ceph as it may break other developer's work. If you need to build an older release, lock a throwaway node like [[hardware:smithi]] and build there. Or, use a container to do the build/testing!
 Using the developer machines to look at teuthology QA artifacts is encouraged. Try to avoid using a text editor to look at large (1GB+) debug logs as this can be RAM intensive/disruptive. Instead, prefer ''less'' or use ''tail -c xM | $EDITOR -'' to look at portions of the log in a text editor.
@@ Line 48: / Line 45: @@
 *******************************************************************************
 </code>
 ==== Configuring CephFS Mounts ====
-The following script can be run to setup CephFS mounts on a new developer playground machines:
+The following ansible playbook can be run to setup CephFS mounts on a new developer playground machines: https://wiki.sepia.ceph.com/doku.php?id=services:cephfs
+==== Teuthology scheduling ====
+Configure the dev playground node to schedule jobs:
 <code>
-#!/bin/bash
+sudo tee /etc/teuthology.yaml <<EOF
+default_machine_type: smithi
+queue_host: teuthology.front.sepia.ceph.com
+queue_port: 11300
+active_machine_types:
+  - smithi
+  - trial
+  - gibba
+EOF
+</code>
-HOST="$1"
+Note: killing a run is (generally) still necessary on [[services:teuthology|teuthology host]]. This is because teuthology-kill requires killing the test processes running there.
-function run {
+==== Configuring LVM volumes using spare disks ====
-  printf '%s\n' "$*" >&2
-  "$@"
-}
-function mssh {
+When setting up a fresh Developer Playground machine, configure an LVM VolumeGroup for use by users. Volumes can be provisioned for a build directory, OSD block device, or anything else needed.
-  run ssh "$HOST" -- "$*"
-}
-mssh sudo mkdir -p -m 755 /etc/ceph
+Note: no redundancy is configured below (i.e. RAID). If a disk is lost, all volumes will be affected.
-ssh reesi001.front.sepia.ceph.com 'env CEPH_KEYRING=/etc/ceph/client.sepian.keyring ceph --id sepian config generate-minimal-conf' | mssh sudo tee /etc/ceph/ceph.conf
-mssh sudo chmod 644 /etc/ceph/ceph.conf
-ssh reesi001.front.sepia.ceph.com 'cat /etc/ceph/client.sepian.keyring' | mssh sudo tee /etc/ceph/client.sepian.keyring
-mssh sudo chmod 644 /etc/ceph/client.sepian.keyring
-ssh reesi001.front.sepia.ceph.com 'ceph-authtool /etc/ceph/client.sepian.keyring -n client.sepian -p' | mssh sudo tee /etc/ceph/client.sepian.secret
-mssh sudo chmod 600 /etc/ceph/client.sepian.secret
+<code>
+sudo pvcreate /dev/$DISK
+</code>
-function genmount {
+Do this for every disk. This is an ad-hoc process because all nodes are different. Also, some disks may have been used in the past so they will need wiped first:
-  local secret=$(mssh sudo cat /etc/ceph/client.sepian.secret)
-  # create mountpoint
-  mssh sudo mkdir -p -- "$2"
-  # make the mountpoint directory (shadowed) unwriteable to prevent accidental modification
-  mssh sudo chmod 000 -- "$2"
-  # set it immutable to enforce that even for root
-  mssh sudo chattr +i -- "$2"
-  printf '172.21.2.201,172.21.2.202,172.21.2.203:%s\t%s\tceph\tname=sepian,secret=%s,mds_namespace=%s,_netdev\t0\t2\n' "$1" "$2" "$secret" "$3" | mssh sudo tee -a /etc/fstab
-}
-genmount /teuthology-archive /teuthology teuthology
+<code>
-genmount / /scratch scratch
+sudo wipefs -a /dev/$DISK
-genmount / /postfile postfile
+</code>
-mssh sudo systemctl daemon-reload
+Once all disks are added as physical volumes, it's then possible to add to a VolumeGroup:
-mssh sudo mount /teuthology
-mssh sudo mount /scratch
+<code>
-mssh sudo mount /postfile
+sudo vgcreate DevPlayground $DISKS
-mssh sudo ln -s /teuthology /a
 </code>
+Finally make a volume for yourself:
+<code>
+sudo lvcreate -L 256G DevPlayground -n $(whoami)-build
+sudo mkfs.xfs /dev/DevPlayground/$(whoami)-build
+mkdir $HOME/build
+chmod 000 $HOME/build
+sudo chattr +i $HOME/build
+echo "/dev/DevPlayground/$(whoami)-build $HOME/build xfs defaults 1 1" | sudo tee -a /etc/fstab
+sudo systemctl daemon-reload
+sudo mount $HOME/build
+</code>
+and some OSD block devices:
+<code>
+for i in `seq 0 8`; do sudo lvcreate -L 16G DevPlayground -n $(whoami)-osd.$i ; done
+</code>
+Make those OSDs owned by you:
+<code>
+printf 'ENV{DM_VG_NAME}=="DevPlayground" ENV{DM_LV_NAME}=="%s-*" OWNER="%s" GROUP="users"\n' $(whoami) $(whoami) | sudo tee -a /etc/udev/rules.d/99-lvmowner.rules
+sudo udevadm control --reload-rules
+sudo udevadm trigger
+</code>
+Then you can use those devices with vstart.sh:
+<code>
+wipefs -a /dev/DevPlayground/$(whoami)-osd.*
+shred -v -n 0 -z -s 16M /dev/DevPlayground/$(whoami)-osd.*
+env OSD=8 ~/ceph/src/vstart.sh \
+    --bluestore-devs $(echo /dev/DevPlayground/$(whoami)-osd.* | tr ' ' ',')
+</code>
+Feel free to make any other volumes that you require.