Fixing pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2 umount error
Tags: fix, initrd, kernel, redhat, rhel3
So last night a RHEL3 machine at went down for patching and sadly it didn’t come back. Here’s what was sitting on the console when I got to the console. Nice… a kernel panic :/
pivotroot: pivot_root(/sysroot,/sysroot/initrd) failed: 2 umount /initrd/proc failed: 2 Freeing ununsed kernel memory: 228k freed Kernel panic: No init found. Try passing init= option to kernel. |
So after spending a significant amount of time troubleshooting the file systems, rebuilding the initrd, trying different init=arguments and trying different kernels I took a good hard look at the error message. Its talking about /sysroot/initrd but I don’t see a directory named /initrd when I look at the root filesystem. So I tried this:
mkdir /initrd reboot |
And to my surprise the system starts happily booting along. All that over a little directory, sheesh. Turns out that this directory was removed by someone or something that thought it wasn’t needed, doh.
[ad]
May 20th, 2010 at 12:04 am
I came across this problem as well with iscsi LUNs using qlogic HBA’s. The solution in a nutshell for me was to rebuild initrd with the new SCSI ID’s of the LUNs and update the multipath configuration.
boot from cdrom and linux rescue
mkdir /target
mount /dev/sda1 /target
SDA_ID=”`/sbin/scsi_id -g -u -s /block/sda`”
SDB_ID=”`/sbin/scsi_id -g -u -s /block/sdb`”
SDC_ID=”`/sbin/scsi_id -g -u -s /block/sdc`”
cp /target/root/multipath-files/multipath.conf /target/etc/multipath.conf
sed -i -e “s:@SDAID@:${SDA_ID}:g” \
-e “s:@SDBID@:${SDB_ID}:g” \
-e “s:@SDCID@:${SDC_ID}:g” /taret/etc/multipath.conf
losetup /dev/loop1 /target/root/multipath-files/initrd.img
mkdir /target/root/multipath-files/tmp
mount /dev/loop1 /target/root/multipath-files/tmp
cp /target/etc/multipath.conf /target/root/multipath-files/tmp/etc/multipath.conf
umount /target/root/multipath-files/tmp
losetup -d /dev/loop1
rmdir /target/root/multipath-files/tmp
cp /target/root/multipath-files/initrd.img /target/root/multipath-files/initrd.img.tmp
gzip /target/root/multipath-files/initrd.img.tmp
mv /target/root/multipath-files/initrd.img.tmp.gz /target/boot/initrd-KERNEL_VERSION-mpath.img
—
Kristian Reese
[Reply]
Naim Shaikh Reply:
February 13th, 2012 at 2:07 am
But i am not able to reach to command prompt. Where i can pass this mkdir command? Please suggest.Thanks.
[Reply]
Keith Reply:
February 13th, 2012 at 11:50 am
You’ll need to mount the filesystem from a rescue kerenel or disc. I typically use the knoppix livecd system.
[Reply]
Kristian Reese Reply:
February 19th, 2012 at 9:36 am
As Keith indicates, boot from a linux installation CD and at the boot: prompt, type: linux rescue
To my original post, the contents of /target/root/multipath-files/mutipath.conf are:
devnode_blacklist {
devnode “^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*”
devnode “^hd[a-z]”
devnode “^cciss!c[0-9]d[0-9]*”
}
defaults {
udev_dir /dev
rr_min_io 128
failback immediate
no_path_retry queue
default_path_grouping_policy failover
polling_interval 5
}
devices {
device {
vendor “NETAPP”
product “LUN”
path_grouping_policy failover
prio_callout none
features “1 queue_if_no_path”
path_checker directio
failback immediate
}
}
multipaths {
multipath {
wwid @SDAID@
alias mpsda
}
multipath {
wwid @SDBID@
alias mpsdb
}
multipath {
wwid @SDCID@
alias mpsdc
}
}
so you could simply take the contents of multipath.conf and input the results of each scsi_id command for each wwid instead of following my post command for command. The reason I had to update mine was because I migrated the luns to another netapp and therefore, the scsi_id changed for each lun.
[Reply]
August 1st, 2012 at 6:47 am
[…] a /initrd directory and that is supposed to solve the problem but it didn't. here is the thread: http://backdrift.org/fixing-pivotroo…2-umount-error does anyone know why it might be not loading fully and how to solve it? thanks for any […]
August 13th, 2013 at 2:35 pm
nearly 3 years later, I’ve encountered this again. This time, the solution was from the original post… mkdir /initrd
[Reply]