SUSE Manager/PowerVM virtualization info gathering

From MicroFocusInternationalWiki
Jump to: navigation, search

PPC virtualization gathering

There are multiple virtualization options for the Power architecture:

1. PowerKVM

Single KVM hypervisor system installed on a PPC machine. In this case, the poller approach could be used to gather the virtualization data.

Seems that PowerKVM is not used that often as PowerVM. Moreover, for PowerKVM, the same tooling as for KVM on x86 can be used (libvirt, virsh,....). Solving the virtualization gathering should be same/similar as with our established "poller" solution.

2. PowerVM

  • 1 PPC machine is able to host logical partitions - "LPARs" (basically virtual machines)
  • 1 special LPAR - VIO Server:
    • responsible for "mapping" the virtual resources of other LPARs to physical resources
    • runs some specialized OS (provided by IBM)
    • user doesn't interact with it directly (e.g. by "logging via SSH"), but using other ways:
      • HMC (hardware management console) - a dedicated machine runs some software (webpage, REST API) that manages hypervisor and resources on it (LPARs, virtualized resources). Seems this is a most common way to manage PowerVM machines.
      • NovaLink (only Power8 and higher!) - a separate LPAR on hypervisor that manager hypervisor and resources on it (LPARs, virtualized resources). NovaLink LPAR usually runs some specialized Ubuntu Linux made by IBM. It's also possible to run SUSE Linux with the NovaLink tooling, but this is more or less experimental and we don't support this. NovaLink CLI uses a python client library pypowervm that talks to REST API of NovaLink.

This page focuses on PowerVM only!!!

The problem

We need to collect these pieces of information:

  • Unique identifiers from the guests (see the "Guest part" below)
  • Unique identifiers of the guests running on hypervisor (see the "Hypervisor part" below)
  • Number of CPU/Sockets on hypervisor - essential for subscription counting

With these we can construct a mapping of which guests run on which hypervisors.

We'll need to use gatherer for this (as running poller on non SLE system (HMC, NovaLink machines)) is not desired.


Guest part

LPAR UUID is in /sys/firmware/devicetree/base/ibm,partition-uuid file. The biggest problem with this is that the file is not always present, see below. I asked our (2) PPC experts about this (what are the conditions for the presence of this file). Answers pending.

SLE12-SP2

  • Installed on our PPC hardware: NovaLink machine: root@power.mgr.suse.de
  • LPAR name: franky-abid-awesome-lpar , IP 10.161.26.5, root/linux
  • Installed from an ISO:
    • Create a virtual optical media repository [2] (via the viosvrcmd command)
    • Upload the ISO: pvmctl vom upload
  • See instructions about installation here [1]

/sys/firmware/devicetree/base/ibm,partition-uuid contains the UUID that matches with pvmctl lpar list --display-fields LogicalPartition.uuid LogicalPartition.name

SLE12-SP3

  • shiraz-3.arch machine
  • No ibm,partition-uuid under device tree!!!
  • Only system-id - is it usable?

SLE15

  • Running on our PPC hardware: NovaLink machine: root@power.mgr.suse.de
  • Not installed ("installation error" with no other hints), but linuxrc "FS" has the same file as SLE12-SP2, the ibm,partition-uuid is present.

Another SLE15 guest - mania-1

  • In QA lab [4]
  • IP 10.162.6.211, root/susetesting
  • SLE for SAP 15 RC1
  • LPAR UUID under the device tree (/sys)


Hypervisor part

This section contains ways to retrieve the virtualization information from the NovaLink and HMC machines, both via CLI and API.

NovaLink

CLI

Run these commands on the NovaLink machine.

LPARs and their uuids
pvmctl lpar list --display-fields LogicalPartition.uuid LogicalPartition.name
CPU information of the hypervisor
pvmctl sys list \
   --display-fields ManagedSystem.system_name ManagedSystem.uuid \
   ManagedSystem.proc_units ManagedSystem.proc_units_avail

API

Successful with local access only on power.mgr.suse.de. The password authentication didn't work for me (it seems it's disabled).

Logon

Create a file logon_request.xml with this content:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LogonRequest xmlns="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" schemaVersion="V1_0">
   <UserID>padmin</UserID>
   <GenerateX-API-SessionFile>true</GenerateX-API-SessionFile>
</LogonRequest>


  • Auth using curl:

Port 12443 (HTTPS) or 12080 (HTTP).

curl -k -X PUT \
   -H "Content-Type: application/vnd.ibm.powervm.web+xml; type=LogonRequest" \
   -H "Accept: application/vnd.ibm.powervm.web+xml; type=LogonResponse" \
   -d @logon_request.xml \
   https://hmc_or_novalink_url:12443/rest/api/web/Logon
Get info about LPARs
  • The previous logon via curl gives you a path with the token
  • Cat the file - this is the token
  • Use the token in request for LPARs
curl -X GET \
   -H "X-API-Session: <THE TOKEN>" \
   http://localhost:12080/rest/api/uom/LogicalPartition


You should get info about all logical partitions including their MACs and UUIDs.

Get info about the hypervisors (CPU Sockets)
  • Get the login token
curl -X GET \
   -H "X-API-Session: $TOKEN" \
   https://url/rest/api/uom/ManagedSystem


Logoff

You also need the token from the logon curl


curl -k -X DELETE \
   -H "Content-Type: application/vnd.ibm.powervm.web+xml; type=LogonRequest" \
   -H "X-API-Session: $TOKEN" \
   https://localhost:12080/rest/api/web/Logon

HMC

Interacting with the HMC API is the same as with the NovaLink API. There should be just subtle differences [3].

Successfully tested on powerhmc1.arch.suse.de:12080 including the remote authentication.

For authentication, use this form of login_request.xml and continue as in "Logon" section for NovaLink.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<LogonRequest xmlns="http://www.ibm.com/xmlns/systems/power/firmware/web/mc/2012_10/" schemaVersion="V1_0">
    <UserID>hscroot</UserID>
    <Password>our_super_secret_password</Password>
</LogonRequest>

Alternative approach

Do not use UUID, but a MAC address as UUID seems to be unreliable.

Drawbacks

  • multiple MAC addresses per client
  • duplicate MAC addresses (different LANs?)

Advantages

  • no change on the client tools - we already gather information about NICs!!

We'd need to adjust the gathering and UUID matching algorithm:

  • either we will store a MAC address of a single NIC as a UUID in the rhnVirtualInstance table and the matching algorithm will check if this MAC address is present among the addresses of the guest
  • or we store MAC addresses of ALL interfaces as the UUID (currently it is a varchar(128))
  • or we store a hash of sorted MAC addresses as a UUID? This would be hard to debug.

Guest part

Done - we store MAC addresses in the db already.

Hypervisor part

NovaLink

  • CLI
pvmctl eth list
  • API - Same snippets as for listing LPARs.

HMC API

Same as NovaLink, the output differs.

Further reading

Links

[1]: https://gitlab.suse.de/jloeser/documentation/wikis/ibm/novalink#create-lpar

[2]: https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/Power%20Systems/page/Using%20PowerVM%20NovaLink%20to%20create%20LPARs%20on%20a%20Power%20Server%20without%20an%20HMC/comment/18399406-52ba-4737-bf6d-7fc457ff4b80

[3]: https://www.ibm.com/support/knowledgecenter/8247-21L/p8eig/p8eig_api.htm

[4]: https://gitlab.suse.de/hsehic/qa-css-docs/blob/master/infrastructure/ppc64le-mania-usage.md