PowerShell module for IBM/LENOVO servers management

Cover

Foreword

  • As a rule of thumb, most diligent server manufacturers equip their servers with hardware and software independent control board, which has a separate network interface. For example, Dell has the integrated Dell Remote Access Controller (iDRAC), Hewlett-Packard has the integrated Lights-Out (iLO) and IBM has the Integrated Management Module (IMM).

  • Currently there are two versions for IBM/LENOVO servers: IMM and IMM2 – their differences will be discussed at a later point in this article. Access and control of both the board and the server are carried out either through a Web interface, the SSH client (e.g. putty) or with special command-line utilities.

  • In addition to these methods, there are other expensive complex control systems such as the IBM Systems Director, but the article will not address them.

  • To manage their servers IBM offers a tool, or rather a set of utilities called the Advanced Settings Utility (ASU). This tool is free and requires no installation, a user must simply download the exe-file and extract its contents to a disk.

The purpose of the article

  • The main objective of this article is to create and share the PowerShell module with a set of easy to use functions for big farm automation, which was built on the basis of IBM servers.

  • Functionality of this module will be based on the ASU package’s utilities (specifically asu.exe or ​asu64.exe depending on the type of your operating system and two intermediate utilities rdmount.exe and rdumount.exe to work with a virtual media drive of the server). To connect via SSH, plink.exe will be used.

  • A secondary objective is to write a PowerShell script using the module for a specific task. The task at hand is quite challenging: To fully automate the process of server’s Firmware upgrade. In order to do so, we need to have the Firmware itself. The instructions on how to create an ISO disk containing the new Firmware versions will be discussed in later paragraphs.

IMM-Module

  • For those who are not familiar with the use/import of PowerShell modules and want to explore more, there is an excellent article written by Jonathan Medd or you can turn to Microsoft Help for instructions.

  • Briefly, the process consists of two stages. First, copy the folder IMM-Module with the module file (file psm1 must be inside any folder) to any of the directories contained in the variable $env:PSModulePath and second, import module by command Import-Module IMM-Module. Make sure that the module is successfully imported by command Get-Module -ListAvailable.

Import-Module IMM-Module -Force
gmo -l |? {$_.Name -eq 'imm-module'}
  • In this article I will not go into detail of each module function (Get-Command -Module IMM-Module), especially since each function has content based help and a minimum of 3-5 examples of utilization:
Get-Help Get-IMMServerBootOrder –Full
Get-Help Unmount-IMMISO –Examples
Get-Help Shutdown-IMMServerOS -Online
  • Instead I will discuss the general principles of the approach.

  • I will assume that you have already downloaded ASU package and extracted its content.

  • The first step I recommend is to edit the value of the variable $ASUDIR at the beginning of the module. It is located in the region Global variables used by all functions in the module.

  • In general it is not necessary to do this—it is a matter of convenience – as all the functions support the option -ASUExec, but then you have to specify it each time you recall each function.

  • The same applies to the variable $Plink, used in the function Connect-IMMSSH.

  • All functions take (based on your selection) one of the two parameter sets for authentication: 1) A pair of User/Password (-IMMLogin/-IMMPwd respectively) or 2) –IMMCred parameter, which is the PowerShell object of type PSCredential. You may choose to pre-initialize it, e.g. in PowerShell profile, and then to use in all subsequent calls of all functions:

$immCred = Get-Credential -UserName loginid -Message "Supervisor LoginID"
Get-IMMInfo "10.99.1.150" -IMMCred $immCred
  • Or utilize a small helper Get-IMMSupervisorCred function:
Get-IMMInfo "10.99.1.150" -IMMCred (Get-IMMSupervisorCred -PSCredential)
Get-IMMInfo "10.99.1.150" -IMMLogin (Get-IMMSupervisorCred -ClearText Username) -IMMPwd (Get-IMMSupervisorCred)
  • To learn how to create secure file for the Get-IMMSupervisorCred, please read this article.

  • If you omit authentication parameters, then the default values will be used (new IBM server has built-in user USERID with password PASSW0RD, after the «W» is a zero).

  • All functions take on a parameter of one or more IMM both as IP address and as a name (if you are using the names, they must be resolved in DNS).

  • All functions return PowerShell objects, which are easy and convenient to work with using the standard PowerShell cmdlets: filter (Where-Object), sort (Sort-Object), format (Select-Object, Format-Table, Format-List, Format-Wide), export (Export-Csv) and save to variable.

  • So-called paired functions, e.g. Get-IMMParam/Set-IMMParam support pipeline input:

Get-IMMParam -IMM "10.99.1.120" -Param DNS_IP_Address1 | Set-IMMParam -Value "10.1.35.50"
  • A special mention goes to a helper function Get-IMMSubnet. Since all functions support pipeline input and are capable of taking not one, but several IMM in parameter -IMM, this function creates a list of IP addresses belonging to the same class C network within the specified range, for example:
Get-IMMSubnet -Subnet "10.99.1.0" -StartIP 50 -EndIP 100 | Get-IMMParam -Param TimeZone
  • This code will give the time zone (parameter TimeZone) for all IMM, found in a range of IP addresses between 10.99.1.50 and 10.99.1.100.

  • By combining the two previous examples, we can easily perform the following task: to scan the whole network (e.g. 10.99.1.1-254), check any parameter for all servers (for example, take the same time zone), check for compliance with the required value (not contains +2:) and in the case of non-compliance assign the desired value GMT + 2 Jerusalem. Therefore:

Get-IMMSubnet -Subnet "10.99.1.0" | Get-IMMParam -Param TimeZone |? {$_.Value –notmatch "\+2:"} | Set-IMMParam –Value "GMT+2:00" –Confirm:$false
  • Note the parameter -Confirm:$false. All the functions that make any changes (starting with verbs Set- or Add-) or functions that perform some action (Shutdown-, Reboot-, Restart-, Clear-) support parameter -Confirm, which by default is $true, i.e. it requires confirmation of your intentions before the execution of its direct action.

1.1-module_confirm

  • While using these functions to make mass changes in scripts, you should cancel the confirmation, which is exactly what we did in the previous example. However, I advise you to use this option very carefully or risk shutting down several hundred servers with just one following line of code:
Get-IMMSubnet "10.99.1.0" | Shutdown-IMMServerOS -Confirm:$false
  • There are some restrictions placed on the functions that deal with a virtual media drive (Get-IMMISO/Mount-IMMISO/Unmount-IMMISO).

  • This is not a limitation of the functions. It is as a result of the functions being based on IMM functionality, which is optional and paid (Feature on Demand, or FoD).

  • In this instance IBM is no different from its competitors – in order to mount the ISO images to the virtual drive of the server or remotely connect to the server console, you need to purchase and activate «Advanced Upgrade» license key.

1.2-module_imm_lic_web

  • To check and activate purchased license keys there are two functions in the module (Get-IMM2FoDKeys/Add-IMM2FoDKey).

1.3-module_fodkeys

  • Unfortunately it is impossible to check the current Firmware versions using the ASU .

  • As a result the module function Connect-IMMSSH was added, which is PowerShell skin to plink.exe utility. This function uncovers limitless possibilities. Once connected to the IMM through this function, you can run any commands via SSH, but only interactively.

  • What does this mean? The IMM’s SSH-server implementation does not allow to simultaneously (within one command) create a connection via SSH and run a command in the session. I presume that this was purposefully done by the IBM engineers to improve the security level. It is important to note that a license is required to connect to IMM2, but at the moment the less expensive Standard version will suffice.

Connect-IMMSSH "10.99.1.0"
  • After a successful connection PowerShell prompt changes to system> and you will be able to run any SSH commands directly in the PowerShell window.

  • For example the command vpd fw (Vital Product Data Firmware) will give you the required information about the versions of both the Server Firmware (UEFI), and the IMM Firmware.

1.4-module_sshvpd

  • For a complete list of supported commands, enter the help.

1.5-module_sshhelp

  • To close the SSH session and return to PowerShell run exit.
  • Server management capabilities by SSH are truly unlimited. You can even control the status of the Locator LED (blue light bulbs) to identify the server in the rack.

  • To do so, run the command identify -s on, then identify -s blink and finally identify -s off and watch as the light first turns on, then blinks and turns off after the last command.

Firmware ISO image creation

  • For those who are familiar with the process, the only requirement to the disc is that it should be automatic – Unattended (see Image 2.3).

  • For beginners I will explain the process in more detail, focusing your attention only on the most important points for the upgrade process automation (full description of the BoMC is not the purpose of this article).

  • IBM, unlike other manufacturers, does not supply Firmware updates in the form of ready bootable ISO image. Instead, you can create it yourself using a special utility named Bootable Media Creator (BoMC).

  • On one hand this is an unnecessary labor cost, on the other hand it is a flexibility in any type of task. Once you downloaded exe-file (the one I use is called ibm_utl_bomc_9.63_windows_i386.exe). Run it on behalf of the Administrator account (Right click -> Run as administrator), otherwise it will not function.

  • BoMC has a fairly simple and intuitive GUI and going through the steps of the Wizard on the output you get ready to use a bootable ISO image containing the new Firmware versions for selected server models. In general you can select all utility supported server models, but then the image creation process may be delayed for hours and the resulting size of the image (in our case ISO file) will be around 7 GB.

  • The first important step in creating the image will be the «Media Purpose». Make sure that you mark only the checkbox «Updates».

2.1-bomc_updates

  • At the stage of «Media Format» select the Device Type «CD/DVD» and «Write to image file».

2.2-bomc_media

  • Last and most important, in the «Unattended Mode» step select the «Use unattended mode». If you do not select this configuration, then the server will remain powered even after the upgrade is complete. The selection of this configuration provides visual confirmation of the successful upgrade and allows to dump the logs. However, during automation we want to avoid it altogether. After the upgrade is complete the server will be powered off. This event we will be tracked in the script, and it will tell us whether the upgrade is complete.

2.3-bomc_unattend

  • At the end you will have a ready-to-use ISO file.

2.4-bomccomplete

Upgrade-IMMServerFirmware.ps1 script

  • Now that you have mastered the module, the rest is up to your imagination. In my day-to-day work, I use the functions of the module in the interactive mode more frequently, i.e. I write short one-liners directly in the PowerShell console, but I also have a few full-fledged scripts based on the module. One of these scripts will help us better understand the IMM-module.

  • Introducing the Upgrade-IMMServerFirmware.ps1.

  • The script’s only requirement is that PowerShell version must not lower than 3. You can check it with the following command ​$PSVersionTable.PSVersion.Major by running it in PowerShell console.

$PSVersionTable.PSVersion.Major -ge 3
True
  • All the script needs to «know» is the IP address or the name of the server’s IMM and where the ISO image that we prepared with the BoMC is located.

  • One and a half or two hours of waiting (depending on the server configuration), several restarts and you will have a powered-on and ready-to-work server with a fully upgraded Firmware.

  • How will the operating system “react” to the restart of the system, you ask? It should be noted that the existence of IBM supported installed operating system does not interfere with the upgrade process, since the power management functions (Shutdown-IMMServerOS and Reboot-IMMServerOS) are able to properly shutdown the operating system and turn off the server after that.

  • At the very least, Microsoft Windows, RHEL и VMware Hypervisor are supported OS. However if this is ESXi Host and it is an HA-cluster member, it is advisable to enter it into Maintenance mode before running the script.

  • For readability and consequent editing, the script is divided into named regions using tags #region/#endregion. The script has only one mandatory parameter –IMM, all other parameters are optional and have default values, which for convenience should be changed according to your infrastructure.

  • Parameter -IMMCred is initialized using the built-in PowerShell cmdlet Get-Credential during script runtime (if you have not specified it in the command line). It is very convenient because Get-Credential has a graphical interface and it looks like this:

3.1-script_getcred

  • Parameter -IMMLog points to a text file, which logs the upgrade process. The next parameter –IMMConfig is worth discussing in more details – it a common CSV file (which may be created and edited with Excel) that contains the settings you want to apply to all servers, for example, the time zone or NTP/DNS servers.

  • The first row of the file must contain the column headers IMMKey and IMMValue:

3.2-script_immconfig

  • In principle it is possible to have several configuration files, such as different geo-sites setting IMMInfo_Location, or who should be notified of the temporary unavailability of the server – IMMInfo_Contact. The IMMValue field’s values are spaces allowed.

  • Available values for the IMMKey are defined in the -Param parameter of Get-IMMParam. Use Intellisense (Ctrl+Space or TAB completion) to view all of them.

3.3-script_getparam

  • Did you happen to notice that I used two keys for NTP server setting: NTPHost and NTPHost1? This is done for compatibility with different IMM versions: IMM and IMM2 (more advanced version).

  • IMM2, unlike IMM, support four NTP servers, respectively NTPHost1, NTPHost2, NTPHost3, NTPHost4 for time synchronization; while IMM only supports one – NTPHost.

  • When you run the script, you will always get an error on one of them depending of IMM version, the following figure shows the server with IMM2 on the board.

3.4-script_ntp

  • The last parameter -BoMC indicates BoMC ISO image. Once you have edited the default settings according to your infrastructure, the number of required parameters is reduced to a minimum, i.e. to only one!

  • Navigate to the folder where you saved the script and run it as follows:

cd C:\scripts
.\Upgrade-IMMServerFirmware.ps1 "10.99.1.116"
  • You can track the script progress in the PowerShell window, and in the log file (parameter -IMMLog).

  • The font color of messages intuitively tell you about the category to which the information belongs (Error, Success, Info). White font messages or informative or can be ignored.

  • Finally, all of this can be downloaded here. The downloaded archive contains four files:

  1. PowerShell module files IMM-Module.psm1 and IMM-Module.psd1 inside the IMM-Module folder
  2. Batch file install_module.cmd for automatic module installation (for optional use)
  3. PowerShell v3 script Upgrade-IMMServerFirmware.ps1
  4. Configuration file example immSettings.csv

One thought on “PowerShell module for IBM/LENOVO servers management

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s