This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
services:cephfs [2023/06/12 19:18] pdonnell |
services:cephfs [2024/01/05 20:05] (current) ljflores Need sudo unless you are running as root. |
||
---|---|---|---|
Line 1: | Line 1: | ||
====== CephFS ====== | ====== CephFS ====== | ||
- | There are a few CephFS file systems available in the sepia lab. These reside on the [[services:longrunningcluster]]. Access to this Ceph cluster is available for any machine on the VPN. | + | There are a few CephFS file systems available in the sepia lab. These reside on the [[services:longrunningcluster|Long Running Cluster]]. |
+ | |||
+ | Access to this Ceph cluster is available for any machine on the [[:vpnaccess|sepia VPN]]. You can and **should** mount these file systems on your laptop or development machine. This reduces load on some shared machines, like [[services:teuthology|teuthology]], and usually provides faster access depending on your client or network. However, for everyday use, you may prefer to access CephFS from a machine co-located with the other lab infrastructure, such as a [[:devplayground|Developer Playground]] machine, where OSD latency/bandwidth will be optimal. | ||
+ | |||
+ | In the scripts/commands below, we will ssh to the ''reesi001.front.sepia.ceph.com'' machine. When [[:vpnaccess#requesting_access|given access to the sepia VPN]], the ssh key you shared should allow accessing this machine. | ||
==== Authorization ==== | ==== Authorization ==== | ||
- | Get access to CephFS with the LRC ''ceph.conf'' and ''client.sepian'' credential: | + | On your development machine or laptop, get access to CephFS with the LRC ''ceph.conf'' and ''client.sepian'' credential: |
<code bash> | <code bash> | ||
Line 18: | Line 22: | ||
</code> | </code> | ||
+ | The ''client.sepian'' credential is suitable for all Ceph developers to access appropriate LRC resources. In particular, it gives access to the ''teuthology'', ''scratch'', and ''postfile'' CephFS file systems. | ||
- | ==== teuthology ==== | + | ==== Mounting all Sepia CephFS file systems ==== |
- | The majority of CephFS use is directed at the "teuthology" file system which hosts QA artifacts for analysis. | + | Generate mounts for your ''/etc/fstab'' using the script below. Copy it locally, mark it executable, and run: |
- | <code> | + | <code bash> |
+ | #!/bin/bash | ||
+ | |||
+ | function genmount { | ||
+ | local secret=$(sudo cat /etc/ceph/client.sepian.secret) | ||
+ | # create mountpoint | ||
+ | sudo mkdir -p -- "$2" | ||
+ | # make the mountpoint directory (shadowed) unwriteable to prevent accidental modification | ||
+ | sudo chmod 000 -- "$2" | ||
+ | # set it immutable to enforce that even for root | ||
+ | sudo chattr +i -- "$2" | ||
+ | printf '172.21.2.201,172.21.2.202,172.21.2.203:%s\t%s\tceph\tname=sepian,secret=%s,mds_namespace=%s,_netdev\t0\t2\n' "$1" "$2" "$secret" "$3" | ||
+ | } | ||
+ | |||
+ | genmount /teuthology-archive /teuthology teuthology | sudo tee -a /etc/fstab | ||
+ | genmount / /scratch scratch | sudo tee -a /etc/fstab | ||
+ | genmount / /postfile postfile | sudo tee -a /etc/fstab | ||
</code> | </code> | ||
+ | |||
+ | The fstab changes will cause these file systems to mount on boot. After adding these entries for the first time, you need to manually mount them: | ||
+ | |||
+ | <code bash> | ||
+ | sudo mount /teuthology | ||
+ | sudo mount /scratch | ||
+ | sudo mount /postfile | ||
+ | </code> | ||
+ | |||
+ | ==== The teuthology FS ==== | ||
+ | |||
+ | The majority of CephFS use is directed at the "teuthology" file system which hosts QA artifacts for analysis. Each test run has a directory in ''/teuthology-archive''. The ''/etc/fstab'' file (generated above) has this directory mounted locally at ''/teuthology'': | ||
+ | |||
+ | <code bash> | ||
+ | ls /teuthology/ | head -n 2 | ||
+ | abhi-2019-12-04_08:55:20-rgw-wip-abhi-testing-2019-12-03-1859-distro-basic-smithi | ||
+ | abhi-2019-12-04_17:41:25-rgw-wip-abhi-testing-2019-12-04-1433-distro-basic-smithi | ||
+ | </code> | ||
+ | |||
+ | It's also common for test artifact paths shared among developers to include a ''/a/'' prefix, such as: | ||
+ | |||
+ | ''/a/teuthology-2023-06-10_14\:23\:08-upgrade\:pacific-x-reef-distro-default-smithi/7301152/teuthology.log'' | ||
+ | |||
+ | You can generate this helper link using: | ||
+ | |||
+ | <code bash> | ||
+ | sudo ln -s /teuthology /a | ||
+ | </code> | ||
+ | |||
+ | ==== The scratch FS ==== | ||
+ | |||
+ | This is a general purpose file system for "scratch" space. Do what you want with it but consider all data in it as eligible for deletion at any time. You're encouraged to create a personal top-level directory. | ||
+ | |||
+ | |||
+ | ==== The postfile FS ==== | ||
+ | |||
+ | The [[https://docs.ceph.com/en/latest/man/8/ceph-post-file/|ceph-post-file]] utility dumps results in this file system. Users are encouraged to use this utility to share artifacts with Ceph developers. | ||
+ | |||
+ | ==== The home FS ==== | ||
+ | |||
+ | There is a "home" file system which hosts the home directories of users of teuthology and potentially other development nodes. Its access is restricted to administrators. |