===== Managing packages in lab-extras =====
lab-extras is a repo of miscellaneous packages we want testnodes to have access to.

http://apt-mirror.front.sepia.ceph.com/lab-extras/

An example of a time we'd want to use this is when a package from CentOS 7.3 has a bugfix we want to test on CentOS 7.2.  The RPM can be downloaded from the CentOS 7.2 CM (testing) repo, placed in the CentOS 7 lab-extras repo and installed.  Example: http://tracker.ceph.com/issues/18094

===== How-To =====
  - Obtain the RPM from a trusted source
    - e.g., ''yum install $package %%--%%downloadonly %%--%%downloaddir=/root''
    - serve it from http: accessible from the POK lab (say, on soko04, with "python3 -m http.server")
    - read https://github.com/ceph/sepia-openshift/blob/main/apt-mirror/README.rst and follow the procedure there to update the repo in question

===== Installing the package =====
Since ceph-cm-ansible makes sure the lab-extras repo is present on all testnodes, a quick way to update a single package on all testnodes would be to run an ansible playbook.  Here's an example from when nfs-utils was done.

<code>
## ansible-playbook thisplaybook.yml --limit="smithi*,mira*"

- hosts:
    - testnodes
  become: true
  tasks:

  - name: Upgrade nfs-utils to lab-extras version
    yum:
      name: nfs-utils
      state: latest
    when: ansible_distribution == 'CentOS'
</code>

This updated the version to nfs-utils-1.3.0-0.**35**.el7.x86_64.  You can make sure no hosts got left behind by running ''ansible -a "rpm -qa nfs-utils" mira | grep -B 1 nfs-utils-1.3.0-0.33.el7.x86_64''.




===== Micron NVMe Firmware Management =====

This page describes how the Micron NVMe firmware used by trial nodes is managed.

===== Supported firmware =====

| Drive family  | Target firmware  |
| Micron 7450   | E2MU300          |
| Micron 7500   | E3MQ005          |

The testnode Ansible role automatically detects supported Micron drives and upgrades them when the installed firmware is older than the target version.

===== Firmware directory layout =====

Firmware images are stored in the `lab-extras` repository using the following directory structure:

<code>
micron-nvme-firmware/
├── 7450/
│   └── E2MU300/
│       └── Micron_7450_E2MU300_release.ubi
└── 7500/
    └── E3MQ005/
        └── Micron_7500_E3MQ005_release.ubi
</code>

When adding support for a new firmware release, create a new version directory instead of replacing the existing files. For example:

<code>
7450/
└── E2MU400/
    └── Micron_7450_E2MU400_release.ubi
</code>

This page intentionally does not duplicate those instructions.

===== Updating the Ansible role =====

When a new firmware version is released:

1. Upload the firmware image to the appropriate directory in lab-extras.

2. Update the target firmware version in roles/testnode/tasks/micron_nvme_firmware.yml.

3. Update the firmware filename if necessary.

4. Verify on a trial node that the firmware is detected and upgraded successfully.

===== Verifying firmware =====

Check the currently installed firmware:

<code>
nvme list
</code>

or

<code>
nvme fw-log /dev/nvmeXn1
</code>

Example:

<code>
Firmware Log for device:nvme0n1
frs1 : E2MU300
</code>

or

<code>
Firmware Log for device:nvme0n1
frs1 : E3MQ005
</code>

===== Notes =====

* Slot 1 is read-only on supported Micron drives.

* Firmware is committed to slot 2 using activation action 3.

* The Ansible role skips non-Micron NVMe devices automatically.