testnodeaccess
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
testnodeaccess [2017/11/30 15:40] djgalloway |
testnodeaccess [2025/01/16 20:31] (current) djgalloway |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== Testnode Access ====== | ====== Testnode Access ====== | ||
| ===== Summary ===== | ===== Summary ===== | ||
| - | Generally, baremetal testnodes should already be installed with an OS and your ssh public key should be in your user's and the ''/home/ubuntu/.ssh/authorized_keys'' file. You should ssh as your username unless the testnode is a VPS. | + | We have about 250 baremetal testnodes that get automatically reserved (locked) and unlocked by teuthology "workers." Workers are daemons on the ''teuthology.front.sepia.ceph.com'' VM that are fed jobs via a beanstalk queue. This page will cover setting up your workstation to lock and unlock testnodes as well as schedule teuthology suites. |
| + | |||
| + | Baremetal testnodes get OSes automatically "installed" using [[services:FOG]] when you lock them either via ''teuthology-lock'' or ''teuthology-suite''. | ||
| + | |||
| + | ===== Teuthology Config ===== | ||
| + | Most developers schedule suites from the ''teuthology.front.sepia.ceph.com'' VM which automatically locks/unlocks machines. | ||
| + | |||
| + | However, if you wish to run ''teuthology'' commands from your workstation, see https://docs.ceph.com/projects/teuthology/en/latest/INSTALL.html#installation-and-setup. | ||
| + | |||
| + | Once you've got ''teuthology'' added to your workstation path, make sure you copy the current ''/etc/teuthology.yaml'' from ''teuthology.front.sepia.ceph.com'' to your **local** workstation's ''~/.teuthology.yaml''. | ||
| + | |||
| + | ===== SSH Config ===== | ||
| + | Baremetal testnodes get reprovisioned with an already-configured OS image including a home dir for your user account. Your ssh public key should be in your user's **and** the ''/home/ubuntu/.ssh/authorized_keys'' file on each testnode. You should always SSH as your username. | ||
| **Example** | **Example** | ||
| Line 7: | Line 19: | ||
| //You should replace// ''~/.ssh/id_rsa'' //with whatever private key corresponds with your private key listed [[https://github.com/ceph/keys/tree/master/ssh|here]]// | //You should replace// ''~/.ssh/id_rsa'' //with whatever private key corresponds with your private key listed [[https://github.com/ceph/keys/tree/master/ssh|here]]// | ||
| - | To avoid interfering with other contributors' tests, you should refrain from logging into hosts that aren't locked by you via teuthology-lock. | + | To avoid interfering with other contributors' tests, you should refrain from logging into hosts that aren't locked by you via ''teuthology-lock''. |
| If you run into any issues with a testnode that appear to be OS, network, or environment relatated (in other words: not ceph/test related), please [[http://tracker.ceph.com/projects/teuthology/issues/new|file a ticket]]. | If you run into any issues with a testnode that appear to be OS, network, or environment relatated (in other words: not ceph/test related), please [[http://tracker.ceph.com/projects/teuthology/issues/new|file a ticket]]. | ||
| - | ===== SSH Config ===== | ||
| Unless you have a separate public/private key pair on the teuthology machine, you'll want to use SSH agent forwarding when SSH'ing to teuthology.front.sepia.ceph.com. You can do this either by using ''ssh -A USER@teuthology.front.sepia.ceph.com'' or adding the following to your workstation's ''~/.ssh/config'': | Unless you have a separate public/private key pair on the teuthology machine, you'll want to use SSH agent forwarding when SSH'ing to teuthology.front.sepia.ceph.com. You can do this either by using ''ssh -A USER@teuthology.front.sepia.ceph.com'' or adding the following to your workstation's ''~/.ssh/config'': | ||
| Line 19: | Line 30: | ||
| IdentityFile ~/.ssh/id_rsa # (This should be the private key matching the public key you provided in your user access ticket) | IdentityFile ~/.ssh/id_rsa # (This should be the private key matching the public key you provided in your user access ticket) | ||
| ForwardAgent yes # <- This is the important part | ForwardAgent yes # <- This is the important part | ||
| + | |||
| + | host smithi* mira* gibba* | ||
| + | StrictHostKeyChecking no | ||
| + | UserKnownHostsFile=/dev/null | ||
| + | </code> | ||
| + | |||
| + | This will allow you to SSH from your workstation -> teuthology machine -> all testnodes | ||
| + | |||
| + | Your SSH config on ''teuthology.front.sepia.ceph.com'' should have this: | ||
| + | |||
| + | <code> | ||
| + | Host * | ||
| + | StrictHostKeyChecking no | ||
| + | UserKnownHostsFile=/dev/null | ||
| </code> | </code> | ||
| Line 40: | Line 65: | ||
| Host * | Host * | ||
| StrictHostKeyChecking no | StrictHostKeyChecking no | ||
| + | UserKnownHostsFile /dev/null | ||
| Host vpm* | Host vpm* | ||
| User ubuntu | User ubuntu | ||
| </code> | </code> | ||
| + | |||
| + | ===== Troubleshooting ===== | ||
| + | ==== SSHException: Error reading SSH protocol banner ==== | ||
| + | <code> | ||
| + | teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds | ||
| + | </code> | ||
| + | |||
| + | - If you're using a static key file (as in you have a ''~/.ssh/id_rsa'' file) on ''teuthology.front.sepia.ceph.com'', make sure its permissions are ''0600'' | ||
| + | - The SSH key can NOT have a passphrase (unless you're doing SSH Agent Forwarding?) | ||
| + | - The SSH key can NOT have been generated using OpenSSH version >= 7.8p1-1 (''[dpkg -l|rpm -qa] | grep openssh'' to find out) | ||
| + | - Either generate your SSH key from ''teuthology.front.sepia.ceph.com'' or try ''ssh-keygen -t rsa -m PEM'' | ||
| + | - ''rm -f ~/.ssh/known_hosts'' and add ''UserKnownHostsFile /dev/null'' to your SSH config. | ||
| + | - Ask Adam Kraitman or Dan Mick to capture new FOG images that include your public SSH key. | ||
| + | - You **must** have ''ForwardAgent yes'' set for ''teuthology.front.sepia.ceph.com'' in your workstation's ''~/.ssh/config'' file. | ||
| + | |||
| + | The newest version of paramiko doesn't support SSH keys that have ''BEGIN OPENSSH PRIVATE KEY'' in them. See https://github.com/paramiko/paramiko/issues/1015. | ||
| ----- | ----- | ||
| Line 55: | Line 97: | ||
| Baremetal testnodes are accessible via out-of-band (OOB) management controllers, or BMCs. If you're unable to reach a host via ssh on its front.sepia.ceph.com address, you can try accessing it using [[services:conserver]] and power cycle via ''ipmitool''. | Baremetal testnodes are accessible via out-of-band (OOB) management controllers, or BMCs. If you're unable to reach a host via ssh on its front.sepia.ceph.com address, you can try accessing it using [[services:conserver]] and power cycle via ''ipmitool''. | ||
| - | **Power Cycle Example** | + | ==== Power Cycle Example ==== |
| <code> | <code> | ||
| ipmitool -I lanplus -U inktank -P XXXXX -H testnode123.ipmi.sepia.ceph.com chassis power cycle | ipmitool -I lanplus -U inktank -P XXXXX -H testnode123.ipmi.sepia.ceph.com chassis power cycle | ||
testnodeaccess.1512056424.txt.gz · Last modified: 2017/11/30 15:40 by djgalloway
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
