Geeking Out On IBM i - Part 1
I remember the first time I tried to work on Linux. Having spent most of my computer-time on DOS, Windows, and OS/2, many things on Linux were foreign to me. I had unix access in college and had to develop programs in C using VI and GCC, but it was often the little things like having to unmount a floppy disk or CD before being able to remove it that confused me.
IBM i has been an even greater leap for my brain still.
On Linux, you have a bootloader which loads a kernel, which runs /sbin/init, which starts up everything else. On Linux, you can substitute /bin/bash in place of /sbin/init to start an interactive shell as the main process of the operating system. On Linux, you have a filesystem which starts with '/' and descends like an upside-down tree with tree-branches to create complex filesystems. On Linux, configurations are split between files (textual and binary) in both the /etc directory and some files/directories in your home directory (likely in the '.config/' directory), like '/home/matt/.config'. With Linux, you download an DVD/ISO image from the Internet, burn the contents to DVD or copy them to a thumb-drive, boot from that ISO image and install the operating system with fairly good help and guidance. On Linux, the primary user interfaces (UI) are either a command-line console or a graphical UI which logs you into a graphical desktop environment like KDE or Unity or Gnome. That's if you don't use the industry standard Secure Shell (SSH), like OpenSSH or PuTTY to connect remotely, potentially even running GUI apps through SSH. Licensing? If you have a non-GPL license for some piece of software, you know it... and typically have paid for the software up front... and the cost and license agreement is likely plainly available on the vendor's website. These concepts are generally well understood in the IT and Hacking communities.
Not so with IBM i.
On The Topic of IBM i...
We wanted to get a better understanding of IBM i, so we bought one. $10k, and a great deal of working with a vendor later, and we had our 4-processor system. Many thanks to National Motor Freight Traffic Association (NMFTA) for sponsoring this research!
AS/400 and IBM i architecture:
The AS/400’s hardware has undergone many changes over it’s 33 year lifespan. In 1995, the original 48-bit architecture was updated to “PowerAS”, a 64-bit AS/400 processor. Later, in 2001, IBM refactored its AS/400 hardware again, updating it to leverage the 64-bit POWER4 architecture (IBM’s version of PowerPC), based on the RS/6000 and original AS/400.
Currently, the POWER architecture powers the IBM Power Systems Servers. Largely the same architecture, these servers now run IBM i and others (like AIX). The system we purchased has a POWER8 processor, which has 4 cores. IBM i enables or disables these cores based on paid licensing. The POWER8 processor core is a 64-bit implementation of the IBM Power Instruction Set Architecture (ISA) Version 2.07. More detailed information can be obtained here: https://www.redbooks.ibm.com/redpapers/pdfs/redp5097.pdf (architecture and technical overview starts on page 41)
TL;DR
Starting up the IBM i is called IPL, or Initial Program Load. On PC's there's something called the "Initial Program Loader" but they're not a 1:1 overlap. On IBM i, IPL is the boot process; whereas on a PC, IPL is a <1k snippet of code that helps find a "big boy bootloader" like GRUB or NTLDR.
IBM i user interface is typically Menus, but most systems takes typed-commands as well
Common Terminal Emulators in IBM i: QP2TERM and QSH
QSECOFR is the "root" user, or for Windows users, "Administrator"
Supervisor on the IBM i is similar to the Linux or Windows Kernel
Patches, or updates you might install using `apt update/apt upgrade` or through applying Service Packs are called Program Temporary Fixes (PTFs), and there are menus associated with applying them.
Programs, which might be found on Linux in /usr/bin/ or /opt, or on Windows in C:\Program Files, are located in QSYS.LIB on IBM i.
Shutting down the system, such as "sudo halt" is performed by IBM i command PWRDWNSYS
Windows Task Manager (or Linux "top" or others) are similar to IBM i's WRKSYSSTS and WRKACTJOB commands
*LIBL (Library List) is the IBM i equivalent the Linux or Windows PATH.
Buying an IBM i
We needed to have access to a machine which we control. This is the only way to get down to the bits and the bytes at the level we want. At the very least, we needed sysadmin access and the ability to royally dork up the installation. Ideally, we could tear the hardware apart and investigate individual components.
The main goal: Asserting our complete control over the system.
Like ya do.
We went through a local vendor who sells and supports IBM i systems as well as many other types of systems. After a few “sales foibles” which almost made us choose a different vendor, we ended up selecting a system which should allow us to exercise the whole system as time goes on. My apologies to NMFTA, we ended up having had to go back to “the well” and ask them for more funding than we originally agreed on.
Unboxing the Systems
Two large brown boxes were delivered to us: The IBM i Server and the Hardware Management Console (HMC). The latter is a IBM x3550 M4 (Intel-processor-based) server, while the real meat of this deal was an IBM i Power S814. It came with 4 processors (only one of which we chose to pay to license), and 32GB of RAM. The system came installed, and with all memory and resources associated with one “partition,” which could be considered a Virtual Machine, or in XEN parlance, a Domain.
Wiring the machines up was pretty straightforward for the most part. Power cables to both (two power supplies each), network cables to each (picking one of 8 ports which came with the IBM i was interesting…. more on that in the Configuring the Network section), and the one unknown: the cable that connects the two systems together.
The Intel-based server, called the HMC, is used to provision the IBM i. We had to connect a CAT6 cable between them for low-level management, which seems to be running high-speed ethernet. We haven’t had a chance to sniff that cable, but that’s something of interest in the future. It could be nothing more than a specialty serial cable, we’ll have to find out!
The HMC - Intel IBM Server (with a special serial/ethernet connection to the 400)
Once the HMC booted up and I entered some administrative information as prompted, I was greeted by a GUI with a Web browser/UI. This web UI is accessible over the network as well. However, if you are SSH port-forwarding to gain access to the WebUI, you have to forward port 443 (https). The UI uses static absolute URLs, forcing your browser to use port 443. On *nix systems, that means you have to port-forward with root access. Sorry.
From the Hardware Management Console web page, you can manage All Systems/Resources managed by this HMC (the upper left icon), the HMC itself (next icon down), Users and Security of the HMC, and “Serviceability” logs and events.
In order to start the IBM i server, you must “Activate” a partition. For our test system, all resources are currently assigned to one single partition. If we wanted to run multiple partitions, we would have to reapportion hard drive, RAM, and CPU utilization, and likely setup virtual services to share the hardware appropriately between the partitions.
... which should provide results like this:
Once the partition was activated, I had to open a TN5250 session to boot the system.. Ahem… I mean “IPL” it. IPL stands for “Initial Program Load.” We Unix geeks like to think we invented Acronyms and shortened names… but IBM is no slouch in this regard.
Until we connected to the TN5250 session and told the system to IPL, the system was just sitting idle. This is similar to sitting at a GRUB Boot menu but not actually starting the OS.
Once I trigger the IPL, I get a rolling boot screen as follows:
As the boot continues, some of the screen changes to show where the process is at:
And finally we’re left at a standard login screen!
The 90 day Full Feature model… and licensing
A word of caution. IBM engages in an interesting (and vicious) marketing scheme called the “90-day full feature” license model. Basically, the first 90 calendar days come with all features enabled, including all CPU’s licensed and several base OS options enabled, even without paying for a license. Once the 90 days are over, you lose these features, and things may start to complain at you…. For instance, apparently the vendor forgot to actually enter our OS license, because we literally had none. I had to enter in the license key manually. It’s 18 characters and is broken into three 6-character chunks (see in the picture).
Once this license was entered, thankfully most of our experience returned to normal.
As an aside, this can become quite terrifying if, for example, you experience this immediately following, let’s say some hypothetical person accidentally killing power to the entire IBM i unit and having to power up from scratch. The first time for each of these things, let me tell you, can cause a minor panic attack.
System Console - Shared Version (pictured here)
Once the IBM i partition is active, the HMC provides a TN5250 session. This session only works locally, as it’s running the TN5250 software on the HMC. However, we were able to access the TN5250 session over the network by pointing TN5250 software from another machine at TCP port 2301 (it works over SSH-tunneled connection as well, assuming you have port forwarding setup correctly).
TN5250 seems a lot like Telnet and TN3270. In fact, it’s very similar. I’ve used 3270 software to access the IBM i, although moving around the menu system can be less than pleasant at times. Telnet is simply not pleased, as the control characters used for 5250 differ from those used for Telnet.
Speaking of which, I’ve tried getting free versions of TN5250 software. To date, I’ve not found one I like that still compiles. I’ve found open-source software that is very difficult to get running. Chances are, if your company uses an IBM i, they can purchase (or already have) commercial software such as MochaSoft’s TN5250 or InterConnect’s PowerTerm.
Provisioning a Partition (already done by vendor)
Our IBM i arrived having been provisioned already. The default provisioning assigned all four processors and all RAM and all hard drive resources to the default partition. If making use of multiple partitions, a Virtual IO Server will likely be necessary (we’re still learning about this), to share out virtual hardware to the multiple partitions. Somewhat like what XEN does on Intel architectures. You can assign hardware directly to a XEN Domain, or you can assign all hardware to a specific domain, which then virtualizes access to the hardware and allows other domains to access it in an “appropriate” fashion.
At one point I started messing around with partitioning, and long story short I dorked up my configuration. Thankfully, with some help from ServiceExpress, we were able to recover my system by Activating the partition and specifying the Default Profile (from the HMC web gui), not the “current” profile, which I had apparently mucked up.
Conclusion… part 1
This has been the first installment in this brief blog miniseries. It talks about our initial interactions with the system, and the startup process.
Watch for another post within the next few weeks for what followed!
Matt (with help from Roni)
