User Tools

Site Tools


Sidebar

General Lab Info (Mainly for Devs)

Hardware

Lab Infrastructure Services

Misc Admin Tasks
These are infrequently completed tasks that don't fit under any specific service

Production Services

OVH = OVH
RHEV = Sepia RHE instance
Baremetal = Host in Sepia lab

The Attic/Legacy Info

services:dccmirrors

WIP - download.ceph.com mirrors

Summary

We have two RHCS clusters running on mira that serve as mirrors for download.ceph.com.

Purpose/Background

The goal of this project is to use/dogfood downstream RHCS software, docs, and ceph-ansible to uncover difficulties in installing and maintaining the product.

Setup notes: https://docs.google.com/document/d/197cRnSA8QF1vYcAejvIT6tCG9Vi1nhUWrX987HTg7P4/edit

Architecture

Systems

There are two clusters. One uses RGW and one uses RBD as the client. Both have 3 MONs running on VMs in RHEV. Their public gateways are also VMs in RHEV.

There are affinity rules in place to ensure the RGW and RBD public gateways all run on separate baremetal hyperisors.

There are also affinity rules in place in RHEV to ensure the MONs all run on separate baremetal hyperisors.

RBD Cluster

The RBD cluster's OSDs run on mira046, mira052, and mira084.

The MONs run on dl-rbd-mon{01..03}.front.sepia.ceph.com.

Since the RBD cluster is just serving a single block device, only one public VM (dl-rbd-pub) is required.

RGW Cluster

The RGW cluster's OSDs run on mira102, mira103, and mira106.

According to Red Hat downstream docs, at least two gateway hosts (dl-rgw-pub{01,02}) and two HAProxy servers (dl-rgw-ha{01,02}) are required to serve a static website.

Networking

The clusters were set up to have all Ceph traffic go over the front VLAN.

The RBD cluster's public VM has a NIC on front and a NIC on the WAN VLAN.

The RGW cluster's HAProxy VMs have a NIC on front and a NIC on the WAN VLAN. The RGW clients themselves only have a NIC on the front VLAN.

services/dccmirrors.txt · Last modified: 2017/06/13 15:58 by dgalloway