|Summary:||Missing important feature: libata bus shutdown and device shutdown|
|Product:||IO/Storage||Reporter:||Henrique de Moraes Holschuh (hmh)|
|Component:||Serial ATA||Assignee:||Tejun Heo (htejun)|
|Kernel Version:||2.6.x mainline||Tree:||Mainline|
Description Henrique de Moraes Holschuh 2007-01-16 17:27:56 UTC
Problem Description: The Linux kernel does not handle device_shutdown() for the libata buses. The bus has no .shutdown method. The devices have no .shutdown handlers. Therefore, device caches are not flushed prior to a system restart, system sleep or system powerdown, and device heads are not unloaded before a system sleep or powerdown, be it a transition to ACPI S3, S4 or S5 state, APM sleep/poweroff, or non-PM-poweroff (the one that just asks the user to powerdown the machine). This causes: * Potential data loss if userspace does not manage to flush caches using the passthrough * The need for the userspace halt(8) utility to try to avoid the above mentioned data loss (incidently, the lack of a proper high-level kernel interface to do device cache flushes or device head unloads also means the userspace code to do it is disgusting) * Uneeded device wear (disk head assembly) if userspace or ACPI BIOS firmware does not unload heads prior to power off. The disk's "head auto-park" on power off is actually called "emergency head unload" on modern disk drives by the HD manufacturers for a reason, and wears down the drive mechanics a lot more, causing the drive's lifetime to be drastically reduced. Hitachi documentation for laptop HDs mention an emergency unload as being from 20 to 100 times more stressfull to the drive mechanics, and makes it clear it should be avoided. Note that it is important to avoid unloading disk heads without a reason (especially on non-laptop drives), so the shutdown handler must differentiate a system restart (which should just sync caches) from a system power off (which needs to sync caches and also to unload heads). The fact that userspace will likely try to unload heads if it can before a shutdown means that either the kernel needs to snoop the few ATA commands that cause head unloads and spin-downs, to avoid reissuing such commands needlessly, or it needs to be easy for userspace to detect that the running kernel is one that can handle disk device shutdowns by itself. Otherwise, during a system shutdown sequence, the HDs are likely to be spun down (due to userspace command), only to be spun up and down again when the kernel attempts to shutdown the device. Also note that it is basically impossible for userspace to cover up for the lack of disk device shutdown handling by the kernel for ACPI S4 (suspend-to-disk), and that userspace code may fail to do it even for the normal shutdown if it doesn't find the devices, or doesn't know how to do it or that it should do it (e.g. Debian Sarge cannot properly shutdown libata devices). Steps to reproduce: * Patch halt(8) or the shutdown scripts to not attempt to issue commands to park heands and sync caches, and listen closely to the ammount of noise the HD will make when the system powers down. Note: this could conceivably kill the HD, and even if it doesn't, it will most surely reduce its working lifetime as much as pulling the plug with the machine running would. * Try to suspend-to-disk using ACPI S4 a ThinkPad T43 (other models might also have the same issue, but I have personally verified that a T43 BIOS does not unload disc heads for ACPI S4).
Comment 1 Tejun Heo 2007-01-16 20:51:35 UTC
Cache *is* synchronized prior to shutting down. It happens via sd->shutdown issuing SCSI SYNCHRONIZE_CACHE which gets translated into ATA_CMD_FLUSH[_EXT] as appropriate. Heads are unloaded properly on sleep to memory too. IIRC, sd_shutdown used to issue SCSI START_STOP command to stop (unload) drives. The command is translated into ATA_STANDBY which unloads the head. However, this behavior changed because of multipath SCSI devices. SCSI devices can be connected to multiple hosts and unloading heads when one host is going down causes problems to other hosts. I'll think about how to solve this. So, THERE IS *NO* DATA LOSS. CACHE IS *ALWAYS* SYNCHRONIZED BEFORE SHUTTING DOWN.
Comment 2 Henrique de Moraes Holschuh 2007-01-17 05:44:08 UTC
I am *very* happy to hear this, and I apologise for the "lack of sync cache" misreport. I screwed up and paid too much attention to the libata side of things, and failed to check the scsi infrastructure properly, thus missing sd.c. I can also definately verify that indeed heads are being unloaded on S3. I assumed the ACPI BIOS was doing this, as S4 and S5 are not being properly handled [from a non-multipath device point of view]... another one to apologise for, I should never "assume" anything. The other points still stand, though: it would be nice to special case commands that unload heads/spin down devices/detach(ATA sleep) devices, so that we avoid spinning up devices to issue these commands, or worse, doing full EH and a ATA bus reset to wake up a detached/sleeping drive only to issue a unload head/spin down/sleep command to it... Also, doing a head unload on reboot would not be nice, as minimizing head unloads will increase device lifespan (especially on desktop drives), therefore whatever is done to fix the need for head unloads on poweroff issue should not cause head unloads or platter spindown on system restart, just on power off/halt. Indeed multipath causes several problems for the above. I'd say the only way to deal with multipath sanely would be to always assume a single path by default for the purposes of the above problems, and to have userspace interfaces capable of telling the kernel that a device is multipath, all the local paths (so that the kernel knows that all disc-specific device state for these paths is shared, even if bus states are not), and whether there are external paths or not. For all I know, most of this is already done by the multipath suite (I have no experience with multipath). Still, for the issues related to this bug report, multipath devices without external paths should probably have all the handling of a single path device, except that the kernel would need to avoid issuing the same set of commands to the device over the other local paths. It would be also EXTREMELY nice to know from userspace whether we need to attempt to issue cache flushes and disk standby commands on the halt(8) command or not (because the running kernel will do it). I'd would *very* much like to remove the utter gross crap in halt.c that deals with it in two or three years... is there a way to query the kernel, directly or indirectly, for this capability?
Comment 3 Tejun Heo 2007-01-18 21:17:07 UTC
My memory about sd used to stopping disks seems to be fabricated. I'm probably confusing with a discussion thread on linux-scsi. Anyways, I'm attaching a patch. It's against 2.6.20-rc5 but should also work with 2.6.19. The followings are added by the patch. * /sys/module/sd_mod/parameters/stop_on_shutdown_default * /sys/class/scsi_disk/h:c:i:l/stop_on_shutdown stop_on_shutdown_default defaults to zero and stop_on_shutdown's are initialized from it. To enable stop_on_shutdown globally, you first set stop_on_shutdown_default to 1 then set all present stop_on_shutdown to 1. If stop_on_shutdown for a disk is 1, sd will issue SCSI START_STOP command with START_VALID and START == 0 which tells the disk to stop on shutdown except for restart. As before, SYNCHRONIZE_CACHE is issued unconditionally which does not usually cause spinup on a spun down device. Would this be enough? If you ACK this, I'll try to push this mainline. Thanks.
Comment 4 Tejun Heo 2007-01-18 21:17:41 UTC
Created attachment 10122 [details] implement-sd-stop-on-shutdown
Comment 5 Henrique de Moraes Holschuh 2007-01-19 04:25:22 UTC
Yes, it is enough. I would still prefer to have /sys/module/sd_mod/parameters/stop_on_shutdown_default set to 1 by default, as multipath is the less common setup (it is, in fact, rare when compared with non-multipath setups now that scsi also means SATA, and soon, PATA), and stop_on_shutdown_default=1 is the correct configuration for anything *but* multi-host multipath setups. But we can certainly set /sys/module/sd_mod/parameters/stop_on_shutdown_default properly in userspace, and it is *extremely* better than what we have now. Acked-by: Henrique de Moraes Holschuh <firstname.lastname@example.org>
Comment 6 Henrique de Moraes Holschuh 2007-01-19 04:26:14 UTC
BTW: Thank you very much for the fast response to this bug :-)
Comment 7 Tejun Heo 2007-01-19 08:37:41 UTC
You're welcome. :-) I agree that it would be nicer to have 1 as the default 'default' value but I'm afraid we can't do that as that would change kernel behavior which userland might be relying on. Well, it's not like configuring this during boot is difficult. I'll submit the patch. Thanks.
Comment 8 Tejun Heo 2007-01-19 08:44:43 UTC
Oh.. BTW, s/multipath/multi initiator/.
Comment 9 NotTheEvilOne 2007-04-12 23:41:45 UTC
One question: What happened to this patch? I have a T60 here (AHCI) and needed to include this patch manually to be able to switch the device off cleanly (without SMART values disk_shift and power-off_retract_count increased).
Comment 10 Tejun Heo 2007-04-12 23:47:07 UTC
SCSI part is committed into mainline. libata part is still pending. This can appear in mainline 2.6.22 at the earliest.
Comment 11 NotTheEvilOne 2007-04-12 23:52:48 UTC
Thanks for your quick answer. Hope I don't bother you if I ask: I use linux-2.6.21-rc6-git4 here and had to apply the scsi part manually. When will this patch be applied to the source code (I know that I don't know the exact procedure for patch inclusion - sorry for that)
Comment 12 Tejun Heo 2007-04-12 23:56:45 UTC
It's in scsi-misc-2.6 tree that the SCSI maintainer maintains. It was late for 2.6.21 inclusion and will be merged into mainline during 2.6.22-rc1 cycle.
Comment 13 Henrique de Moraes Holschuh 2008-10-06 10:46:55 UTC
This issue has been dealt with for a long time now. Any remaining fallout is being tracked separately on its own bug entries, so I am marking this bug closed.