Roy Tang

A client updated their EC2 CentOS 6.4 instance to 6.9 using yum update. After the update and a reboot, the instance wouldn’t start up anymore (became unreachable)

The system log (recovered from the AWS console) goes something like this:

Xen Minimal OS!

  start_info: 0x10d3000(VA)

    nr_pages: 0xe4f53

  shared_inf: 0xeea31000(MA)

     pt_base: 0x10d6000(VA)

nr_pt_frames: 0xd

    mfn_list: 0x9ab000(VA)

   mod_start: 0x0(VA)

     mod_len: 0

       flags: 0x300

    cmd_line: root=/dev/sda1 ro console=hvc0 4

  stack:      0x96a100-0x98a100

MM: Init

      _text: 0x0(VA)

     _etext: 0x7b824(VA)

   _erodata: 0x97000(VA)

     _edata: 0x9cce0(VA)

stack start: 0x96a100(VA)

       _end: 0x9aa700(VA)

  start_pfn: 10e6

    max_pfn: e4f53

Mapping memory range 0x1400000 - 0xe4f53000

setting 0x0-0x97000 readonly

skipped 0x1000

MM: Initialise page allocator for 1807000(1807000)-e4f53000(e4f53000)

MM: done

Demand map pfns at e4f54000-20e4f54000.

Heap resides at 20e4f55000-40e4f55000.

Initialising timer interface

Initialising console ... done.

gnttab_table mapped at 0xe4f54000.

Initialising scheduler

Thread "Idle": pointer: 0x20e4f55050, stack: 0xe4810000

Thread "xenstore": pointer: 0x20e4f55800, stack: 0xe4820000

xenbus initialised on irq 3 mfn 0xfeffc

Thread "shutdown": pointer: 0x20e4f55fb0, stack: 0xe4830000

Dummy main: start_info=0x98a200

Thread "main": pointer: 0x20e4f56760, stack: 0xe4840000

"main" "root=/dev/sda1" "ro" "console=hvc0" "4" 

vbd 2048 is hd0

******************* BLKFRONT for device/vbd/2048 **********





backend at /local/domain/0/backend/vbd/2334/2048

629145600 sectors of 512 bytes

**************************

vbd 2128 is hd1

******************* BLKFRONT for device/vbd/2128 **********





backend at /local/domain/0/backend/vbd/2334/2128

314572800 sectors of 512 bytes

**************************




    GNU GRUB  version 0.97  (3751244K lower / 0K upper memory)





+-------------------------------------------------------------------------+||||||||||||||||||||||||+-------------------------------------------------------------------------+


    Use the ^ and v keys to select which entry is highlighted.


    Press enter to boot the selected OS, 'e' to edit the


    commands before booting, or 'c' for a command-line.  CentOS (2.6.32-696.18.7.el6.x86_64)                                      CentOS (2.6.32-573.el6.x86_64)                                           CentOS (2.6.32-358.18.1.el6.x86_64)                                      CentOS-6.4-x86_64-GA-03 2.6.32-358.el6.x86_64                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     The highlighted entry will be booted automatically in 1 seconds.     Booting 'CentOS (2.6.32-696.18.7.el6.x86_64)'





root (hd0)


 Filesystem type is ext2fs, using whole disk


kernel /boot/vmlinuz-2.6.32-696.18.7.el6.x86_64 root=/dev/xvde ro crashkernel=a


uto LANG=en_US.UTF-8 KEYTABLE=us


initrd /boot/initramfs-2.6.32-696.18.7.el6.x86_64.img





============= Init TPM Front ================

Tpmfront:Error Unable to read device/vtpm/0/backend-id during tpmfront initialization! error = ENOENT

Tpmfront:Info Shutting down tpmfront

close blk: backend=/local/domain/0/backend/vbd/2334/2048 node=device/vbd/2048

close blk: backend=/local/domain/0/backend/vbd/2334/2128 node=device/vbd/2128

We’re all not too experienced with AWS so we don’t have much clue what to do. Any idea what we should be looking at or anything to point us in the right direction? The instance was actually a clone of production that we tested the upgrade on before doing it on actual production, so we have a chance to try it again and adjust any settings or whatever before or after the update.