hardware:robsoni
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
hardware:robsoni [2021/06/03 00:22] djgalloway [Table] |
hardware:robsoni [2021/06/03 00:30] (current) djgalloway [Table] |
||
|---|---|---|---|
| Line 23: | Line 23: | ||
| These PXE using Legacy/BIOS mode and can be provisioned via Cobbler normally. | These PXE using Legacy/BIOS mode and can be provisioned via Cobbler normally. | ||
| - | ==== Network Config ==== | + | ===== Network Config ===== |
| - | These nodes have a regular 1Gb uplink but then 2x 25Gb connections per server to a QFX5120 for backend storage traffic. | + | ==== qfx-rgw ==== |
| + | These nodes have a regular 1Gb uplink but then 2x 25Gb connections per server to a QFX5120-32C for backend storage traffic. This switch is managed by Ceph Infra and not uplinked. | ||
| + | |||
| + | I purchased qty 5 100Gb -> 4x 25Gb breakout cables with this. See https://docs.google.com/spreadsheets/d/1_ETaz5ollV_66ODFGXiVWEibecdBZIvHg9X1zN1pF88/edit#gid=0 | ||
| + | |||
| + | I just created a ''/24'' vlan on the switch giving each server it's own Aggregated Ethernet interface. | ||
| + | |||
| + | ^ Server ^ Private IP ^ | ||
| + | | robsoni01 | 172.20.23.1 | | ||
| + | | robsoni02 | 172.20.23.2 | | ||
| + | | robsoni03 | 172.20.23.3 | | ||
| + | | robsoni04 | 172.20.23.4 | | ||
| + | | robsoni05 | 172.20.23.5 | | ||
| The switch is named ''qfx-rgw.ipmi.sepia.ceph.com'' and can be accessed via SSH. Here's the current switch config that seems to be working. | The switch is named ''qfx-rgw.ipmi.sepia.ceph.com'' and can be accessed via SSH. Here's the current switch config that seems to be working. | ||
| Line 361: | Line 373: | ||
| } | } | ||
| } | } | ||
| + | </code> | ||
| + | |||
| + | ==== Server bond config ==== | ||
| + | Here's what I ran to set up the bonds initially. The bonding type may change. | ||
| + | <code> | ||
| + | nmcli connection add type bond con-name bond0 ifname bond0 bond.options "mode=802.3ad" | ||
| + | nmcli connection add type ethernet slave-type bond con-name bond0-port1 ifname eno1np0 master bond0 | ||
| + | nmcli connection add type ethernet slave-type bond con-name bond0-port2 ifname eno2np1 master bond0 | ||
| + | |||
| + | ### Change the IP in this line! | ||
| + | nmcli connection modify bond0 ipv4.addresses '172.20.23.X/24' | ||
| + | |||
| + | nmcli connection modify bond0 ipv4.gateway '172.20.23.254' | ||
| + | nmcli connection modify bond0 ipv4.method manual | ||
| + | nmcli connection modify bond0 connection.autoconnect-slaves 1 | ||
| + | nmcli connection up bond0 | ||
| </code> | </code> | ||
hardware/robsoni.1622679758.txt.gz · Last modified: 2021/06/03 00:22 by djgalloway
Page Tools
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
