Install OpenSolaris/ OpenIndiana on M1015/ LSI9240 RAID controller

We’ve purchased a few IBM X3650-M3 servers, to be used as new storageservers with Solaris and ZFS. The X3650 comes with an IBM M1015 RAID-controller, which is a bit of a budget controller, since it lacks features as NVRAM and a BBU. However, it is one of the few controllers that supports JBOD, and that is ideal, because I would like to let ZFS handle the RAID part. Furthermore, it will only be used to control the OS-disks and some SSD cache-devices in the server itself, and all data-disks will be in separate enclosures.
While installing these machines we ran into some problems with a missing driver for the M1015-controller, with both OpenSolaris (snv_134) and OpenIndiana (oi_148), and an outdated firmware. I hope I can save someone else a few days of troubleshooting by describing the process of installing a driver during the installation of OpenSolaris, and upgrading the firmware.
Insert CD with text-install, reboot the server and let it boot from CD (if it doesn’t, reset the uEFI-BIOS to defaults, but enable ‘Legacy Only’ in the Boot Manager). Once textinstall is loaded, choose your keyboard, language. In the installation menu, choose ‘[2] Install Additional Drivers’. If you’re in the same situation as I, you’ll (eventually) see a notification that no driver can be found for an ‘LSI Logic MegaRAID SAS 9250’. The IBM M1015 is actually a rebranded LSI-controller.

no driver

Alright, so we need a Solaris-driver for the LSI 9240 controller. Luckily, LSI provides this :). Hit [F9] (Quit) and return to the installation menu. In the installation menu, choose ‘[3] Shell’.

I like to do the next steps not in the console, but via SSH from another computer, so first I set up the network and the SSH-server. Use ifconfig to configure a simple network as you like.

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

Now you’re ready to SSH to your server.
Verify that no disks are currently visible:

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

If you’re in the same situation as I, you should see no disks.
Download the appropriate driver from the LSI-site. Check the link under ‘Accept’ for use in the wget-command below, it might have changed.

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

If all went well, you should have a number of files, like imr_sas.img, imr_sas.iso and imr_sas.Z. We’ll be using the nice Solaris package-file, so an upgrade or removal should be easily possible. This package is hidden in the .Z-file:

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

Do NOT reboot, like the system suggests, because you’ll loose the changes :). The driver is already loaded by the installscript, but not correctly. We’ll load the driver ourselves, but first we have to remove it.

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

This will yield an error, which I don’t quite get, but it seems no big deal.

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

Now it’s time to add the driver correctly:

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

If all went well, the last command should not give any output, and definitely no error. format should show some disks now:

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

Hurray! We’ve got disks to install Solaris on! Return to the installation, choose ‘[1] Install’ and complete the installation normally.
After the installation completes and the system reboots, go into the IMM-setup and enable ‘Legacy Only’ in the Boot Manager. I had to do that to have the system boot the newly created install.

Firmware

Our M1015 was delivered with an older firmware (20.5.1-0014, while 20.10.1-0020 was available). For some reason the old firmware make the disks behind the controller unbootable. Solaris would install just fine (after above steps), but would not load after a reboot. Not even grub was loaded. I did two things to fix this: upgrade the firmware of the RAID-controller, and add ‘Legacy Only’ to the list of bootable devices in the IMM-setup (F1 during boot). I’m not sure what ‘Legacy Only’ does, but it needs to be enabled, together with (in my case) ‘Disk 0’ and probably ‘Disk 1’ if I want to boot from the ZFS-mirrored root-disks.
I’ll shortly list the steps I took for the firmware upgrade. Apply brain.
Download a recent version of the MegaCli tool:

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

Download the newest firmware:

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

Find out what the adapter-ID is of the M1015-controller:

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

In my case it’s 1, since there is another adapter present which controls the enclosures.
Flash the new firmware to the correct adapter. Tripplecheck that you’ve got the right one!

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

$ nano /etc/ssh/sshd_config
  # Change 'PermitRootLogin no' to 'PermitRootLogin yes' and save the file
$ svcadm enable ssh
$ passwd    # you won't be able to login without a password :)

You’ll need to reboot for the upgrade to finish. After that, the disks on the M1015/LSI 9240 should be bootable!
[widget id=”magereport_widget-4″ /]