Running ESXi on a Cisco SRE 910 K9

Date: 2018-09-22
Author: Simon Jackson

Backstory

I inherited a Cisco ISR Router and didn't really have any plans for it.

A friend of mine convinced me to make it my home router, and play about with the features on there.
I thought, why not?

I bought a couple of Cisco 3750x Switches to go with it, and planned on a future VMware LAB, of physical serves. Maybe even run a vCenter Server Embedded appliance on my router, and use DPM to power my lab down when it's not required... that'd be neat.

Cisco SRE BLADE: SM-SRE-910-K9

Spec: Intel Core2Duo (@1.86Ghz), 8GB DRAM, 2x 500GB SATA @7200RPM (RAID 1 config). 1x USB 1.0, 1x RJ-45 and 4x Gbe on the SRE Backplane.

If you are looking for part 1 of this article, please click here.

Looking at the Interfaces?

This took me a long time to interpret and get the config correct for. Once i worked out the following diagram, the config was easy.
Note my Service Module, operating on SM1/0 was attached to port-channel 0.1, which was natively a member of VLAN 1.

Therefore the "Management Network" and the "Virtual Machine Network" were one and the same thing. As these are both routed interfaces, we don't have to be concerned with spanning-tree. We should NOT use VLAN 1 on the multi-gigabit-fabric; this does cause complications.

Configuring interfaces on the router

Establishing a router IP-interface has already been performed on Po1.1 (192.168.1.1/24). And I must have spent about 45 minutes re-configuring these interfaces; to get a reliable configuration.
It's probably worth noting the SRE blade has 2x routed interfaces, and upto only 1x switched interface.
And for the router to route traffic into the device, add a static route.

Configure the Service Module uplink interfaces and the Gigabit 0/1 interface.

By routing traffic towards VMNIC0 (192.168.1.100), we FORCE traffic to that destination to reach the observed service-module mac-address. This is similar to creating a static ARP entry on a client on same network segment.

Setting up a console-connection to the SRE

We know the SRE is a micro server with a serial connection to handshake with, and a network interface.

I never captured the output of the `show lines` command; this informs you of which serial-line numbers are configured for each of the ESM/ISM/SM interfaces. I did capture my SM is on line 67.

Lets configure the Service Module vty interface -->

Installing SRE-V-k9 on the Service-Module

My laptop was bound to 192.168.1.50/24 on VLAN 1 (Po1.1) - so the installation command was pulling the file from my local web-service. TFTPD would have worked too.
I attempted to deploy the release v 2.0.2 (which is a packaged ESXi v5.1.0 installation).
I would recommend opening TWO SSH connections to your router, as during installation, the ssh console will freeze. The second SSH console can be used with these commands debug sre-install info and service-module sm1/0 status.

The above attempt was probably my 9th attempt. Yeah i know :(
I had plenty of config mistakes; but until i had a console session open, at the same time as starting the installation, i wasn't fully aware what was happening.

Also one of my HDDs had failed, so i had to buy new and replace copy the firmware off the other working one.. i might blog that process one day, but it's not really the point of this blog post).

Logging in to the service-module console session

A simple serial console to the micro-server (SRE) worked increadibly well. Maximise the SSH session to full-screen, or at least wider than the usual 800x600px!

I saw no reason to interrupt the bootloader; but I guess there is a CLI for amending boot-loader configuration, like TFTP boot paths etc. So i just waited.

Around 5 minutes after... the console cleared with new-line inserts... as the SRE booted into ESXi. None of it was really readable, until we get to this view.

Alternate keystroke mappings for DCUI mode:

Esc + 1 -> F1
Esc + 2 -> F2
Esc + 3 -> F3
Esc + 4 -> F4
Esc + 5 -> F5
Esc + 6 -> F6
Esc + 7 -> F7
Esc + 8 -> F8
Esc + 9 -> F9
Esc + 0 -> F10
Esc + ! -> F11
Esc + @ -> F12

WOAH!!! An ESXi DCUI console via Serial.. now thats cool. F2 Keyboard operation was a little confusing - my google-fu kicked in and I got a nice keystroke mappings list (to the right of the above console for your reference).

Upon the first F2 key-press; I was prompted to enter a default password.

Note the DHCP assigned address above.

Logging in to ESXi

The vSphere Flex Client is needed to access the VMware ESXi Hypervisor, and the specific build no 780721 (or higher) is required.

Launch the vSphere client, and target the DHCP assigned address above. Click Login, accept the SSL certificate thumbprint and wait for it to connect.

Configuring the ESXi Hypervisor, is really out of scope for this blog.

However important factors to consider.

I set a static IP on vmk0. added vmk1, bound to the "virtual machine network", and set the port-group to use the decicated "Intel" vmnic. So i had another way in, whilst i worked on console logging with SM1/0.
ESXi cannot install on a RAID-1 disk-set. The storage controller driver doesn't detect the disk upon boot. Therefore "disk-cfg-mode=nonraid" parameter was necessary. This leaves 2x 500GB SAS disks visible to ESXi, with `datastore1` pre-formatted with vmfs. I formatted the second as datastore2.
I managed to get 8GB of RAM operational; and following cisco community posts, it's easy to achieve 12GB (8+4 GB sticks), and quite possible tok reach 16GB, yet i've not seen any part/specs to confirm that.
I had a Synology NAS, with 8x spindles, 4x SSDs for cache and 4x 1GBE NIC. Logically the 14TB of space i had available was going to be my VM datastore. Registering the ISCSI service on there, and installing 2x ESXi vmKernels, bound to iSCSI, creating port-groups with dedicated vmnics, attaching port-bindings. And scanning the iscsi software adapter worked first time.

Next i'll document the ESXi Hypervisor software update, upgrading from v5.1, to v5.5 and then v6.0 finally reaching v6.5!
Click here to read more.

Page updated

Report abuse