====== Wireguard Access ======
===== Summary =====
We use Wireguard for VPN access to the Sepia lab.  Lab access requests are now automated.

===== How To =====

==== Summary ====

  * Install Wireguard
  * Generate a public/private keypair
  * Provide the public key in the [[https://onboarding.sepia.ceph.com/wireguard|Onboarding Portal]]
  * Once your access is approved, you'll be e-mailed your Wireguard IP
  * Connect

==== Mac/Linux ==

1. For Mac, Install Wireguard from the App Store\\
{{:screenshot_2025-12-03_at_8.36.27_am.png?400|}}

For Linux,\\
<code>
sudo apt install wireguard-tools resolvconf
# or
sudo dnf install wireguard-tools
</code>

For Mac, install ''wireguard-tools'' using [[https://brew.sh/|Homebrew]]\\
<code>
brew install wireguard-tools
</code>

3. Create a directory for a Wireguard keypair.  E.g.,\\
<code>
mkdir ~/.wireguard
</code>

4. Generate a keypair

<code>
wg genkey | tee ~/.wireguard/private.key | wg pubkey > ~/.wireguard/public.key
</code>

**Keep the Private Key secret!!**

5. Provide the Wireguard public key (''~/.wireguard/public.key'' contents) in the [[https://onboarding.sepia.ceph.com/wireguard|Onboarding Portal]].

=== DO NOT ping any Lab Admins directly ===

6. Create ''~/.wireguard/client.conf''  (note that the $PRIVATE_KEY must expand to
the actual text of your private key)\\
<code>
PRIVATE_KEY=$(cat ~/.wireguard/private.key)

cat <<EOF > ~/.wireguard/client.conf
[Interface]
PrivateKey = $PRIVATE_KEY
Address = X.X.X.X/32
DNS = 10.20.192.13, front.sepia.ceph.com, ipmi.sepia.ceph.com
MTU = 1200

[Peer]
PublicKey = kyEHy3ZBewI5RiK4/a0/UQn6O1kMt3h8V3u0OwsfUXc=
AllowedIPs = 172.21.0.0/20, 172.21.32.0/20, 172.16.48.0/24, 10.20.192.0/20, 10.20.208.0/20, 172.16.50.0/23, 172.16.53.0/25, 172.16.55.0/26, 172.16.56.0/23, 172.16.59.0/25, 172.16.60.0/25, 172.21.64.0/24, 172.21.65.0/24, 172.21.66.0/24, 172.21.67.0/24
Endpoint = 192.86.31.5:1194
PersistentKeepalive = 25
EOF
</code>

6. Once you receive your private IP, replace ''X.X.X.X'' in ''client.conf'' with it.

7. Bring up the interface

**On Mac OS**, open the Wireguard GUI.  Press Command+O and open ''~/.wireguard/client.conf''.  Click **Activate**

Success looks like\\
{{::screenshot_2025-12-03_at_9.08.01_am.png?400|}}


**On Ubuntu**,\\
<code>
sudo mkdir -p /etc/wireguard
sudo mv ~/.wireguard/client.conf /etc/wireguard/wg0.conf
sudo chmod 600 /etc/wireguard/wg0.conf
sudo wg-quick up wg0
</code>

Use ''wg show'' to show status.

==== Windows ====
1. Install Windows client from https://www.wireguard.com/install/

2. In the Wireguard app, select "Add tunnel/Add empty tunnel".  A public and private key will be automatically generated for you.  Note them.  KEEP THE PRIVATE KEY SECRET.

3. Submit your public key in the [[https://onboarding.sepia.ceph.com/wireguard|Onboarding Portal]]

4. Wait to receive your Wireguard IP

5. Add the configuration to the wireguard app.  The first two lines
([Interface] and PrivateKey = <your private key>) will already be present.
Be very careful to not change anything besides your Address.  In 
particular, do not edit [Peer] PublicKey; that is the server's public key
and does not change.

<code>
[Interface]
PrivateKey = <your private key>
Address = <address from communication with David/Dan>/32
DNS = 10.20.192.13, front.sepia.ceph.com, ipmi.sepia.ceph.com
MTU = 1200

[Peer]
PublicKey = kyEHy3ZBewI5RiK4/a0/UQn6O1kMt3h8V3u0OwsfUXc=
AllowedIPs = 172.21.0.0/20, 172.21.32.0/20, 172.16.48.0/24, 10.20.192.0/20, 10.20.208.0/20, 172.16.50.0/23, 172.16.53.0/25, 172.16.55.0/26, 172.16.56.0/23, 172.16.59.0/25, 172.16.60.0/25, 172.21.64.0/24, 172.21.65.0/24, 172.21.66.0/24, 172.21.67.0/24
Endpoint = 192.86.31.5:1194
PersistentKeepalive = 25
</code>

6. Save the configuration, which should activate the wg0 tunnel.  If all goes well, you will see "Latest Handshake" in the Peer section be a few seconds out of date

7. despite DNS = , the current wireguard client does not properly handle split-horizon DNS.  You must add Powershell scripts to the configuration, and enable their execution with a Windows Registry setting.

Add the following after MTU = in the [Interface] section:

<code>
PostUp = powershell -ExecutionPolicy Bypass -File "C:\Wireguard\wg-up.ps1" -InterfaceAlias wg0
PostDown = powershell -ExecutionPolicy Bypass -File "C:\Wireguard\wg-down.ps1" -InterfaceAlias wg0
</code>

and add the scripts to C:\Wireguard:
<code>
# wg-up.ps1
param(
    [string]$InterfaceAlias = "wg0",
    [string[]]$Domains = @("front.sepia.ceph.com"),
    [string]$DNSServer = "10.20.192.11"
)

# Ensure admin
if (-not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")) {
    Write-Error "This script must be run as Administrator."
    exit 1
}

# Verify the requested interface exists and is up
$Interface = Get-NetAdapter -Name $InterfaceAlias -ErrorAction Stop

if ($Interface.Status -ne "Up") {
    Write-Error "WireGuard interface /$InterfaceAlias' is not up."
    exit 1
}

Write-Output "Using WireGuard interface: $InterfaceAlias"

# Add NRPT rules (idempotent)
foreach ($domain in $Domains) {
    if (-not (Get-DnsClientNrptRule | Where-Object { $_.Namespace -eq $domain })) {
        Write-Output "Adding NRPT rule for $domain -> $DNSServer"
        Add-DnsClientNrptRule -Namespace $domain -NameServers $DNSServer
    }
}

# Set connection-specific DNS suffix (short names)
Set-DnsClient -InterfaceAlias $InterfaceAlias -ConnectionSpecificSuffix $Domains[0]
Write-Output "Set connection-specific suffix: $($Domains[0])"
</code>

<code>
# wg-down.ps1
param(
    [string]$InterfaceAlias = "wg0",
    [string[]]$Domains = @("sepia.ceph.com")
)

# Ensure admin
if (-not ([Security.Principal.WindowsPrincipal] [Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] "Administrator")) {
    Write-Error "This script must be run as Administrator."
    exit 1
}

Write-Output "Using WireGuard interface: $InterfaceAlias"

# Remove NRPT rules safely
foreach ($domain in $Domains) {
    Get-DnsClientNrptRule |
        Where-Object { $_.Namespace -eq $domain } |
        Remove-DnsClientNrptRule -Force -ErrorAction SilentlyContinue
    Write-Output "Removed NRPT rule for $domain"
}

# Clear connection-specific suffix
Set-DnsClient -InterfaceAlias $InterfaceAlias -ConnectionSpecificSuffix ''
Write-Output "Cleared connection-specific suffix"

</code>

Add to the registry:

<code>
HKEY_LOCAL_MACHINE\Software\WireGuard\DangerousScriptExecution
</code>

set it to 1.  You can do this with a Command prompt running as administrator:

<code>
reg add HKLM\Software\WireGuard /v DangerousScriptExecution /t REG_DWORD /d 1 /f
</code>


===== DNS Configuration using dnsmasq on Linux =====

If you're using systemd-resolved, stop.  It is incapable of handling split DNS.  I would not use resolvconf.

The way I (dmick) chose on Fedora 41 was to disable systemd-resolved and set up a dnsmasq instance behind /etc/resolv.conf.  Here's what I did:

1) sudo systemctl stop systemd-resolved; sudo systemctl disable systemd-resolved
2) add this to /etc/systemd/networkd.conf and restart networkd (because I'm running networkd to configure networks):

<code>
[Network]
DynamicUser=no
ManageResolver=false
</code>
   
3) install dnsmasq if necessary, and configure it.  I chose to put my configuration in /etc/dnsmasq.d/dnsmasq.conf (because my system runs dnsmasq with a conf search path of /etc/dnsmasq.d):

<code>
# add domain to shortnames in /etc/hosts; may be helpful
expand-hosts
# don't use /etc/resolv.conf for servers configured here
no-resolv

# for debugging, enable log-queries
# log-queries

# set the server that should handle these three domains
server=/sepia.ceph.com/10.20.192.11
server=/front.sepia.ceph.com/10.20.192.11
server=/ipmi.sepia.ceph.com/10.20.192.11

# set the upstream servers for anything else
server=1.1.1.1
server=9.9.9.9
</code>
  
4) remove the existing /etc/resolv.conf and replace with this:

<code> 
nameserver 127.0.0.1
search front.sepia.ceph.com ipmi.sepia.ceph.com sepia.ceph.com
options ndots:2
</code>

5) systemctl restart dnsmasq

Now lookups of shortname or shortname.front or shortname.ipmi should work.  Note that dig does not respect the search domains in /etc/resolv.conf by default; you must use dig +search <domain>

===== Split DNS on systemd-resolved distros (Fedora, recent Ubuntu, etc.) =====

If all web browsing / external DNS stops working while connected to the Sepia VPN, but works again after ''sudo resolvectl revert sepia'', your distro is using systemd-resolved and wg-quick has set the lab DNS server as the **default** resolver for all queries — not just Sepia ones.

Do **not** use ''resolvectl revert'' as a workaround; it also breaks resolution of lab hostnames. Configure split DNS instead so only Sepia domains are sent to the lab resolver:

  - Edit ''/etc/wireguard/sepia.conf''
  - **Remove** (or comment out) the ''DNS ='' line
  - Add the following lines under ''[Interface]'':

<code>
PostUp = resolvectl dns %i 10.20.192.13
PostUp = resolvectl domain %i ~sepia.ceph.com ~front.sepia.ceph.com ~ipmi.sepia.ceph.com
PostUp = resolvectl default-route %i false
</code>

Then restart the tunnel:

<code>
sudo wg-quick down sepia && sudo wg-quick up sepia
</code>

Verify:

<code>
resolvectl status sepia
</code>

You should see ''Default Route: no'' and the three sepia domains listed. Lab hostnames (e.g. ''smithi001.front.sepia.ceph.com'') will resolve via the VPN; everything else uses your normal DNS.

==== Alternative: NetworkManager ====

On Fedora you can instead import the tunnel into NetworkManager, which handles split DNS natively:

<code>
sudo nmcli connection import type wireguard file /etc/wireguard/sepia.conf
sudo nmcli connection modify sepia ipv4.dns-search "~sepia.ceph.com;front.sepia.ceph.com;ipmi.sepia.ceph.com" ipv4.dns-priority 50
sudo nmcli connection up sepia
</code>

Only use one method — don't run wg-quick and the NetworkManager connection at the same time.