SLES File System(s) turns Read-Only

A VM with SLES 11.4 (64bit) was installed a while ago,  and is used as application server.

Everything has been working properly for a while (~1 month), but as of a few days back, some filesystems (/  and /opt)  have turned to read-only (!?).

Pretty much every task fails, be it starting vi or even man pages (can’t create files on disk).

Background and solutions

  • try to backup the affected FS to another server
  • try a remount to rw of the affected FS (be careful with rootFS) :   mount -o remount,rw <FS>
  • reboot and either go to single user mode and fsck the disk or sometimes the automatic fsck on reboot would catch it
  • automatic fsck on reboot : touch /forcefsck and then reboot,  with SLES the checkfs.sh script does not exist,  here it is : /etc/init.d/boot.rootfsck

When on VMware ESXi systems with back-ended to SAN storage

modify the disk timeouts for the block devices in /sys

# cat /sys/block/sd*/device/timeout

60

Increased the value of this number from 60 to 180, this modifies the time before a “timeout” occurs, essentially allowing you to suffer more latency.


Source:  http://unix.stackexchange.com/questions/16156/suse-filesystem-turns-read-only

http://unix.stackexchange.com/questions/145882/read-only-root-filesystem

http://unix.stackexchange.com/questions/123963/how-to-force-fsck-at-every-boot-all-relevant-filesystems