A VM with SLES 11.4 (64bit) was installed a while ago, and is used as application server.
Everything has been working properly for a while (~1 month), but as of a few days back, some filesystems (/ and /opt) have turned to read-only (!?).
Pretty much every task fails, be it starting vi or even man pages (can’t create files on disk).
Background and solutions
- try to backup the affected FS to another server
- try a remount to rw of the affected FS (be careful with rootFS) :
mount -o remount,rw <FS>
- reboot and either go to
single user mode and
fsck the disk or sometimes the automatic
fsck on reboot would catch it
- automatic fsck on reboot : touch /forcefsck and then reboot, with SLES the checkfs.sh script does not exist, here it is : /etc/init.d/boot.rootfsck
When on VMware ESXi systems with back-ended to SAN storage
modify the disk timeouts for the block devices in
# cat /sys/block/sd*/device/timeout
Increased the value of this number from
180, this modifies the time before a “timeout” occurs, essentially allowing you to suffer more latency.