Fixing pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2 umount error

Tags: , , , ,


So last night a RHEL3 machine at went down for patching and sadly it didn’t come back. Here’s what was sitting on the console when I got to the console. Nice… a kernel panic :/

pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2 umount
/initrd/proc failed: 2 Freeing ununsed kernel memory: 228k freed Kernel
panic: No init found. Try passing init= option to kernel.

So after spending a significant amount of time troubleshooting the file systems, rebuilding the initrd, trying different init=arguments and trying different kernels I took a good hard look at the error message. Its talking about /sysroot/initrd but I don’t see a directory named /initrd when I look at the root filesystem. So I tried this:

mkdir /initrd
reboot

And to my surprise the system starts happily booting along. All that over a little directory, sheesh. Turns out that this directory was removed by someone or something that thought it wasn’t needed, doh.

[ad]

6 Responses to “Fixing pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2 umount error”

  1. Kristian Reese Says:

    I came across this problem as well with iscsi LUNs using qlogic HBA’s. The solution in a nutshell for me was to rebuild initrd with the new SCSI ID’s of the LUNs and update the multipath configuration.

    boot from cdrom and linux rescue

    mkdir /target
    mount /dev/sda1 /target

    SDA_ID=”`/sbin/scsi_id -g -u -s /block/sda`”
    SDB_ID=”`/sbin/scsi_id -g -u -s /block/sdb`”
    SDC_ID=”`/sbin/scsi_id -g -u -s /block/sdc`”

    cp /target/root/multipath-files/multipath.conf /target/etc/multipath.conf
    sed -i -e “s:@SDAID@:${SDA_ID}:g” \
    -e “s:@SDBID@:${SDB_ID}:g” \
    -e “s:@SDCID@:${SDC_ID}:g” /taret/etc/multipath.conf

    losetup /dev/loop1 /target/root/multipath-files/initrd.img

    mkdir /target/root/multipath-files/tmp
    mount /dev/loop1 /target/root/multipath-files/tmp
    cp /target/etc/multipath.conf /target/root/multipath-files/tmp/etc/multipath.conf
    umount /target/root/multipath-files/tmp
    losetup -d /dev/loop1
    rmdir /target/root/multipath-files/tmp

    cp /target/root/multipath-files/initrd.img /target/root/multipath-files/initrd.img.tmp
    gzip /target/root/multipath-files/initrd.img.tmp
    mv /target/root/multipath-files/initrd.img.tmp.gz /target/boot/initrd-KERNEL_VERSION-mpath.img


    Kristian Reese

    [Reply]

    Naim Shaikh Reply:

    But i am not able to reach to command prompt. Where i can pass this mkdir command? Please suggest.Thanks.

    [Reply]

    Keith Reply:

    You’ll need to mount the filesystem from a rescue kerenel or disc. I typically use the knoppix livecd system.

    [Reply]

    Kristian Reese Reply:

    As Keith indicates, boot from a linux installation CD and at the boot: prompt, type: linux rescue

    To my original post, the contents of /target/root/multipath-files/mutipath.conf are:

    devnode_blacklist {
    devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
    devnode “^hd[a-z]”
    devnode “^cciss!c[0-9]d[0-9]*”
    }

    defaults {
    udev_dir /dev
    rr_min_io 128
    failback immediate
    no_path_retry queue
    default_path_grouping_policy failover
    polling_interval 5
    }

    devices {
    device {
    vendor “NETAPP”
    product “LUN”
    path_grouping_policy failover
    prio_callout none
    features “1 queue_if_no_path”
    path_checker directio
    failback immediate
    }
    }

    multipaths {
    multipath {
    wwid @SDAID@
    alias mpsda
    }

    multipath {
    wwid @SDBID@
    alias mpsdb
    }

    multipath {
    wwid @SDCID@
    alias mpsdc
    }
    }

    so you could simply take the contents of multipath.conf and input the results of each scsi_id command for each wwid instead of following my post command for command. The reason I had to update mine was because I migrated the luns to another netapp and therefore, the scsi_id changed for each lun.

    [Reply]

  2. virtualization of a live linux mandrake 10 server into vmware gives error Says:

    […] a /initrd directory and that is supposed to solve the problem but it didn't. here is the thread: http://backdrift.org/fixing-pivotroo…2-umount-error does anyone know why it might be not loading fully and how to solve it? thanks for any […]

  3. Kristian Reese Says:

    nearly 3 years later, I’ve encountered this again. This time, the solution was from the original post… mkdir /initrd

    [Reply]

Join the Conversation