Recovery from LED 551, 555, or 557 in AIX V4 and V5


Contents

About this document
Causes of an LED 551, 555, or 557
Recovery procedure
Recommended fixes

About this document

This document describes the typical causes for LED 551, 555, or 557 during an IPL. Also outlined is a recovery procedure. This document applies to AIX Version 4 and 5.


Causes of an LED 551, 555, or 557

Some typical causes of an LED 551, 555, or 557 during an IPL are:


Recovery procedure

To diagnose and fix these typical problems, you must boot from bootable media, run logform on /dev/hd8, and run fsck to fix any file systems that may be corrupted.

WARNING: Do not use this document if the system is a /usr client, diskless client, or dataless client.

  1. Power down your system. If it has a key mode select switch (that is, if it is a Microchannel-based system), turn the key to the Service position.

  2. With bootable media of the same version and level as the system, boot the system.

    The bootable media can be any one of the following:

    • Bootable CD-ROM
    • NON_AUTOINSTALL mksysb
    • Bootable Install Tape

    Follow the screen prompts to the following menu:

       Welcome to Base Operating System 
       Installation and Maintenance 
    
  3. Choose Start Maintenance Mode for System Recovery (Option 3). The next screen has the Maintenance Menu.

    • Choose Access a Root Volume Group (Option 1).

      The next screen displays a warning that indicates you will not be able to return to the Base OS menu without rebooting.

    • Choose 0 continue.

      The next screen displays information about all volume groups on the system.

    • Select the root volume group by number.

    • Choose Access this volume group and start a shell before mounting the filesystems (Option 2).

    If you get errors indicating that a physical volume is missing from the rootvg, run diagnostics on the physical volumes to find out if you have a bad disk. Do not continue with the rest of the steps in this document.

    If you get other errors from the preceding option, do not continue with the rest of the steps in this document. Correct the problem causing the error. If you need assistance correcting the problem, contact one of the following:

    • your local branch office
    • your point of sale
    • your AIX support center

  4. Format the default jfslog for the rootvg JFS file systems.
       /usr/sbin/logform /dev/hd8 
    

    Answer yes when asked if you want to destroy the log.

  5. Next, run the following commands to check and repair file systems. (The -y option gives fsck permission to repair file systems when necessary.)
       fsck /dev/hd1 
       fsck /dev/hd2 
       fsck /dev/hd3 
       fsck /dev/hd4 
       fsck /dev/hd9var 
    
  6. Type exit. The file systems will automatically mount after you type exit.

  7. If you are running the Andrew File System (AFS), use the following commands to find out whether you have more than one version of the v3fshelper file. Otherwise, skip to step 8.
       cd /sbin/helpers 
       ls -l v3fshelper* 
    

    If you have only one version of the v3fshelper file (for example, v3fshelper), proceed to step 8.

    If there is a version of v3fshelper marked as original (for example, v3fshelper.orig), run the following commands:

       copy v3fshelper v3fshelper.afs 
       copy v3fshelper.orig v3fshelper 
    
  8. If rootvg is mirrored, verify that all drives in the volume group are active:
         lsvg -p rootvg
    

    If either one of the drives is either in a "Missing" or "Removed" state, or if you see just the PVID along with the following error:

         0516-304 Unable to find device id in the device configuration database
    
    you may have encountered a known defect with AIX V4.3.3., which is corrected at 4.3.3.27 of bos.rte.lvm (APAR IY15442). You will need to perform the approriate procedures for reducing that drive out of rootvg. This procedure can be found in the document, "Removing and Replacing a Fixed Disk." Once that has been completed, you can proceed to the next step.

  9. Determine which disk is the boot disk with the lslv command. The boot disk will be shown in the PV1 column of the lslv output.
       lslv -m hd5 
    

    WARNING: Do not proceed further if the system is a /usr client, diskless client, or dataless client.

  10. Recreate the boot image and alter the boot list. (hdisk# is the boot disk determined in step 9.)
       bosboot -a -d /dev/<hdisk#>
       bootlist -m normal <hdisk#>
    

    NOTE: If the preceding bosboot command fails with an error message indicating the boot logical volume does not exist on the device specified, this generally indicates the hdisk numbering in Normal mode is different than the hdisk numbering in Service mode. You must specify the special device file representing the PV with the PVID which matches the PVID in the lslv -m hd5 output in step 9.

  11. If you copied the v3fshelper file in step 7, copy AFS file system helper back to v3fshelper:
       copy v3fshelper.afs v3fshelper 
    
  12. With the key in Normal position, run
       shutdown -Fr 
    

If you followed all of the preceding steps and the system still stops at an LED 551, 555, or 557 during a reboot in normal mode, you may want to pursue further system recovery assistance from one of the following:

For reasons of time and the integrity of your AIX operating system, the best alternative at this point may be to reinstall AIX.


Recommended fixes

NOTE: In AIX Versions 4 and 5, use the command instfix -ik to determine if a particular fix is installed.

     APAR          Description                        			AIX Level
   IY15442      ipl_varyon failure if any disk in rootvg goes bad	4.3.3.



[ Doc Ref: 90605189014742     Publish Date: Dec. 14, 2001]