User Tools

Site Tools


services:networking

Networking

Networks

The Sepia internal LANs are divided up into four /20 networks described below.

front.sepia.ceph.com back.sepia.ceph.com ipmi.sepia.ceph.com Other
Available IP Range 172.21.0.1 - 172.21.15.250 172.21.16.1 - 172.21.31.250 172.21.32.1 - 172.21.47.250 172.21.48.1 - 172.21.63.250
Gateway 172.21.15.254 172.21.31.254 172.21.47.254 172.21.63.254
VLAN ID 100 101 102 103
Use Main Network interface Not used at the moment Used to access OOB management 172.21.48.0/20 is currently used for VPN client IPs.

Hardware

All the switches in the Sepia lab are either Juniper EX4300 1Gb or QFX5100 10Gb (SFP+) with the exception of a Mellanox SX1012 private switch connected to the rhoda and incerta nodes.

Typical Switch config

In racks that only have 1Gb networking (e.g., Mira), the top-of-rack switch is most likely split

  • Ports 0-23 assigned to VLAN 100 (front)
  • Ports 24-47 assigned to VLAN 102 (ipmi)

The racks full of Smithi have a 10Gb and 1Gb switch. The 10Gb switch is assigned to vlan100 (front) and the 1Gb switch to vlan102 (ipmi).

The infra rack (houses RHEV, teuthology, gitbuilder.ceph.com), AA-06, has a 10Gb switch with no standard port assignments. I believe any port that's not already in use isn't configured.

Switch Port Config Changes

Red Hat IT manages all switches in the Sepia lab and switch port configuration changes should be submitted to servicedesk@redhat.com.

Providing the serial number of the switch helps IT determine which switch needs to be configured. Switch serial numbers can be found in PnT DevOps inventory tool, OpenDCIM. (Read the pop-up message for login credentials)

Example

Subject: Community Cage switch port config
Body:
Hi,

Please configure the top-of-rack switch (Serial number XXXXXX) in rack AA-02 as follows:
Ports 0 - 23 assigned to vlan100 / 172.21.0.0/20
Ports 24 - 47 assigned to vlan102 / 172.21.31.254/20

Thanks

Public IPs

The Community Cage has a block of public IPs dedicated to it. Ceph's usable IPs are 8.43.84.129 - 8.43.84.186.

IP Use
8.43.84.129 OpenVPN gateway and reverse proxies
8.43.84.130 gitbuilder.ceph.com
8.43.84.131 Jump host for Chinese devs
8.43.84.132 git.ceph.com
8.43.84.133 Old OpenVPN server (store01)
8.43.84.134 drop.ceph.com
8.43.84.135 RDU download.ceph.com mirror (rgw s3 only)
8.43.84.136 lists.ceph.io
services/networking.txt · Last modified: 2018/06/18 14:59 by djgalloway