1e513de99SMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0 2e513de99SMauro Carvalho Chehab 3e513de99SMauro Carvalho Chehab======= 4e513de99SMauro Carvalho ChehabSCSI EH 5e513de99SMauro Carvalho Chehab======= 6e513de99SMauro Carvalho Chehab 7e513de99SMauro Carvalho ChehabThis document describes SCSI midlayer error handling infrastructure. 8ce5c5d65SMauro Carvalho ChehabPlease refer to Documentation/scsi/scsi_mid_low_api.rst for more 9e513de99SMauro Carvalho Chehabinformation regarding SCSI midlayer. 10e513de99SMauro Carvalho Chehab 11e513de99SMauro Carvalho Chehab.. TABLE OF CONTENTS 12e513de99SMauro Carvalho Chehab 13e513de99SMauro Carvalho Chehab [1] How SCSI commands travel through the midlayer and to EH 14e513de99SMauro Carvalho Chehab [1-1] struct scsi_cmnd 15e513de99SMauro Carvalho Chehab [1-2] How do scmd's get completed? 16e513de99SMauro Carvalho Chehab [1-2-1] Completing a scmd w/ scsi_done 17e513de99SMauro Carvalho Chehab [1-2-2] Completing a scmd w/ timeout 18e513de99SMauro Carvalho Chehab [1-3] How EH takes over 19e513de99SMauro Carvalho Chehab [2] How SCSI EH works 20e513de99SMauro Carvalho Chehab [2-1] EH through fine-grained callbacks 21e513de99SMauro Carvalho Chehab [2-1-1] Overview 22e513de99SMauro Carvalho Chehab [2-1-2] Flow of scmds through EH 23e513de99SMauro Carvalho Chehab [2-1-3] Flow of control 24e513de99SMauro Carvalho Chehab [2-2] EH through transportt->eh_strategy_handler() 25e513de99SMauro Carvalho Chehab [2-2-1] Pre transportt->eh_strategy_handler() SCSI midlayer conditions 26e513de99SMauro Carvalho Chehab [2-2-2] Post transportt->eh_strategy_handler() SCSI midlayer conditions 27e513de99SMauro Carvalho Chehab [2-2-3] Things to consider 28e513de99SMauro Carvalho Chehab 29e513de99SMauro Carvalho Chehab 30e513de99SMauro Carvalho Chehab1. How SCSI commands travel through the midlayer and to EH 31e513de99SMauro Carvalho Chehab========================================================== 32e513de99SMauro Carvalho Chehab 33e513de99SMauro Carvalho Chehab1.1 struct scsi_cmnd 34e513de99SMauro Carvalho Chehab-------------------- 35e513de99SMauro Carvalho Chehab 36e513de99SMauro Carvalho ChehabEach SCSI command is represented with struct scsi_cmnd (== scmd). A 37e513de99SMauro Carvalho Chehabscmd has two list_head's to link itself into lists. The two are 38e513de99SMauro Carvalho Chehabscmd->list and scmd->eh_entry. The former is used for free list or 39e513de99SMauro Carvalho Chehabper-device allocated scmd list and not of much interest to this EH 40e513de99SMauro Carvalho Chehabdiscussion. The latter is used for completion and EH lists and unless 41e513de99SMauro Carvalho Chehabotherwise stated scmds are always linked using scmd->eh_entry in this 42e513de99SMauro Carvalho Chehabdiscussion. 43e513de99SMauro Carvalho Chehab 44e513de99SMauro Carvalho Chehab 45e513de99SMauro Carvalho Chehab1.2 How do scmd's get completed? 46e513de99SMauro Carvalho Chehab-------------------------------- 47e513de99SMauro Carvalho Chehab 48e513de99SMauro Carvalho ChehabOnce LLDD gets hold of a scmd, either the LLDD will complete the 49e513de99SMauro Carvalho Chehabcommand by calling scsi_done callback passed from midlayer when 50e513de99SMauro Carvalho Chehabinvoking hostt->queuecommand() or the block layer will time it out. 51e513de99SMauro Carvalho Chehab 52e513de99SMauro Carvalho Chehab 53e513de99SMauro Carvalho Chehab1.2.1 Completing a scmd w/ scsi_done 54e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 55e513de99SMauro Carvalho Chehab 56e513de99SMauro Carvalho ChehabFor all non-EH commands, scsi_done() is the completion callback. It 57e513de99SMauro Carvalho Chehabjust calls blk_complete_request() to delete the block layer timer and 58e513de99SMauro Carvalho Chehabraise SCSI_SOFTIRQ 59e513de99SMauro Carvalho Chehab 60e513de99SMauro Carvalho ChehabSCSI_SOFTIRQ handler scsi_softirq calls scsi_decide_disposition() to 61e513de99SMauro Carvalho Chehabdetermine what to do with the command. scsi_decide_disposition() 62e513de99SMauro Carvalho Chehablooks at the scmd->result value and sense data to determine what to do 63e513de99SMauro Carvalho Chehabwith the command. 64e513de99SMauro Carvalho Chehab 65e513de99SMauro Carvalho Chehab - SUCCESS 66e513de99SMauro Carvalho Chehab 67e513de99SMauro Carvalho Chehab scsi_finish_command() is invoked for the command. The 68e513de99SMauro Carvalho Chehab function does some maintenance chores and then calls 69e513de99SMauro Carvalho Chehab scsi_io_completion() to finish the I/O. 70e513de99SMauro Carvalho Chehab scsi_io_completion() then notifies the block layer on 71e513de99SMauro Carvalho Chehab the completed request by calling blk_end_request and 72e513de99SMauro Carvalho Chehab friends or figures out what to do with the remainder 73e513de99SMauro Carvalho Chehab of the data in case of an error. 74e513de99SMauro Carvalho Chehab 75e513de99SMauro Carvalho Chehab - NEEDS_RETRY 76e513de99SMauro Carvalho Chehab 77e513de99SMauro Carvalho Chehab - ADD_TO_MLQUEUE 78e513de99SMauro Carvalho Chehab 79e513de99SMauro Carvalho Chehab scmd is requeued to blk queue. 80e513de99SMauro Carvalho Chehab 81e513de99SMauro Carvalho Chehab - otherwise 82e513de99SMauro Carvalho Chehab 83e513de99SMauro Carvalho Chehab scsi_eh_scmd_add(scmd) is invoked for the command. See 84e513de99SMauro Carvalho Chehab [1-3] for details of this function. 85e513de99SMauro Carvalho Chehab 86e513de99SMauro Carvalho Chehab 87e513de99SMauro Carvalho Chehab1.2.2 Completing a scmd w/ timeout 88e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 89e513de99SMauro Carvalho Chehab 90*deef1be1SJohn GarryThe timeout handler is scsi_timeout(). When a timeout occurs, this function 91e513de99SMauro Carvalho Chehab 92e513de99SMauro Carvalho Chehab 1. invokes optional hostt->eh_timed_out() callback. Return value can 93e513de99SMauro Carvalho Chehab be one of 94e513de99SMauro Carvalho Chehab 95e513de99SMauro Carvalho Chehab - BLK_EH_RESET_TIMER 96e513de99SMauro Carvalho Chehab This indicates that more time is required to finish the 975c139ce9SKhazhismel Kumykov command. Timer is restarted. 98e513de99SMauro Carvalho Chehab 99e513de99SMauro Carvalho Chehab - BLK_EH_DONE 100e513de99SMauro Carvalho Chehab eh_timed_out() callback did not handle the command. 101e513de99SMauro Carvalho Chehab Step #2 is taken. 102e513de99SMauro Carvalho Chehab 1035c139ce9SKhazhismel Kumykov 2. scsi_abort_command() is invoked to schedule an asynchronous abort which may 1045c139ce9SKhazhismel Kumykov issue a retry scmd->allowed + 1 times. Asynchronous aborts are not invoked 1055c139ce9SKhazhismel Kumykov for commands for which the SCSI_EH_ABORT_SCHEDULED flag is set (this 1065c139ce9SKhazhismel Kumykov indicates that the command already had been aborted once, and this is a 1075c139ce9SKhazhismel Kumykov retry which failed), when retries are exceeded, or when the EH deadline is 1085c139ce9SKhazhismel Kumykov expired. In these cases Step #3 is taken. 109e513de99SMauro Carvalho Chehab 110e513de99SMauro Carvalho Chehab 3. scsi_eh_scmd_add(scmd, SCSI_EH_CANCEL_CMD) is invoked for the 111e513de99SMauro Carvalho Chehab command. See [1-4] for more information. 112e513de99SMauro Carvalho Chehab 113e513de99SMauro Carvalho Chehab1.3 Asynchronous command aborts 114e513de99SMauro Carvalho Chehab------------------------------- 115e513de99SMauro Carvalho Chehab 116e513de99SMauro Carvalho Chehab After a timeout occurs a command abort is scheduled from 117e513de99SMauro Carvalho Chehab scsi_abort_command(). If the abort is successful the command 118e513de99SMauro Carvalho Chehab will either be retried (if the number of retries is not exhausted) 119e513de99SMauro Carvalho Chehab or terminated with DID_TIME_OUT. 120e513de99SMauro Carvalho Chehab 121e513de99SMauro Carvalho Chehab Otherwise scsi_eh_scmd_add() is invoked for the command. 122e513de99SMauro Carvalho Chehab See [1-4] for more information. 123e513de99SMauro Carvalho Chehab 124e513de99SMauro Carvalho Chehab1.4 How EH takes over 125e513de99SMauro Carvalho Chehab--------------------- 126e513de99SMauro Carvalho Chehab 127e513de99SMauro Carvalho Chehabscmds enter EH via scsi_eh_scmd_add(), which does the following. 128e513de99SMauro Carvalho Chehab 129e513de99SMauro Carvalho Chehab 1. Links scmd->eh_entry to shost->eh_cmd_q 130e513de99SMauro Carvalho Chehab 131e513de99SMauro Carvalho Chehab 2. Sets SHOST_RECOVERY bit in shost->shost_state 132e513de99SMauro Carvalho Chehab 133e513de99SMauro Carvalho Chehab 3. Increments shost->host_failed 134e513de99SMauro Carvalho Chehab 135e513de99SMauro Carvalho Chehab 4. Wakes up SCSI EH thread if shost->host_busy == shost->host_failed 136e513de99SMauro Carvalho Chehab 137e513de99SMauro Carvalho ChehabAs can be seen above, once any scmd is added to shost->eh_cmd_q, 138e513de99SMauro Carvalho ChehabSHOST_RECOVERY shost_state bit is turned on. This prevents any new 139e513de99SMauro Carvalho Chehabscmd to be issued from blk queue to the host; eventually, all scmds on 140e513de99SMauro Carvalho Chehabthe host either complete normally, fail and get added to eh_cmd_q, or 141e513de99SMauro Carvalho Chehabtime out and get added to shost->eh_cmd_q. 142e513de99SMauro Carvalho Chehab 143e513de99SMauro Carvalho ChehabIf all scmds either complete or fail, the number of in-flight scmds 144e513de99SMauro Carvalho Chehabbecomes equal to the number of failed scmds - i.e. shost->host_busy == 145e513de99SMauro Carvalho Chehabshost->host_failed. This wakes up SCSI EH thread. So, once woken up, 146e513de99SMauro Carvalho ChehabSCSI EH thread can expect that all in-flight commands have failed and 147e513de99SMauro Carvalho Chehabare linked on shost->eh_cmd_q. 148e513de99SMauro Carvalho Chehab 149e513de99SMauro Carvalho ChehabNote that this does not mean lower layers are quiescent. If a LLDD 150e513de99SMauro Carvalho Chehabcompleted a scmd with error status, the LLDD and lower layers are 151e513de99SMauro Carvalho Chehabassumed to forget about the scmd at that point. However, if a scmd 152e513de99SMauro Carvalho Chehabhas timed out, unless hostt->eh_timed_out() made lower layers forget 153e513de99SMauro Carvalho Chehababout the scmd, which currently no LLDD does, the command is still 154e513de99SMauro Carvalho Chehabactive as long as lower layers are concerned and completion could 155e513de99SMauro Carvalho Chehaboccur at any time. Of course, all such completions are ignored as the 156e513de99SMauro Carvalho Chehabtimer has already expired. 157e513de99SMauro Carvalho Chehab 158e513de99SMauro Carvalho ChehabWe'll talk about how SCSI EH takes actions to abort - make LLDD 159e513de99SMauro Carvalho Chehabforget about - timed out scmds later. 160e513de99SMauro Carvalho Chehab 161e513de99SMauro Carvalho Chehab 162e513de99SMauro Carvalho Chehab2. How SCSI EH works 163e513de99SMauro Carvalho Chehab==================== 164e513de99SMauro Carvalho Chehab 165e513de99SMauro Carvalho ChehabLLDD's can implement SCSI EH actions in one of the following two 166e513de99SMauro Carvalho Chehabways. 167e513de99SMauro Carvalho Chehab 168e513de99SMauro Carvalho Chehab - Fine-grained EH callbacks 169e513de99SMauro Carvalho Chehab LLDD can implement fine-grained EH callbacks and let SCSI 170e513de99SMauro Carvalho Chehab midlayer drive error handling and call appropriate callbacks. 171e513de99SMauro Carvalho Chehab This will be discussed further in [2-1]. 172e513de99SMauro Carvalho Chehab 173e513de99SMauro Carvalho Chehab - eh_strategy_handler() callback 174e513de99SMauro Carvalho Chehab This is one big callback which should perform whole error 175e513de99SMauro Carvalho Chehab handling. As such, it should do all chores the SCSI midlayer 176e513de99SMauro Carvalho Chehab performs during recovery. This will be discussed in [2-2]. 177e513de99SMauro Carvalho Chehab 178e513de99SMauro Carvalho ChehabOnce recovery is complete, SCSI EH resumes normal operation by 179e513de99SMauro Carvalho Chehabcalling scsi_restart_operations(), which 180e513de99SMauro Carvalho Chehab 181e513de99SMauro Carvalho Chehab 1. Checks if door locking is needed and locks door. 182e513de99SMauro Carvalho Chehab 183e513de99SMauro Carvalho Chehab 2. Clears SHOST_RECOVERY shost_state bit 184e513de99SMauro Carvalho Chehab 185e513de99SMauro Carvalho Chehab 3. Wakes up waiters on shost->host_wait. This occurs if someone 186e513de99SMauro Carvalho Chehab calls scsi_block_when_processing_errors() on the host. 187e513de99SMauro Carvalho Chehab (*QUESTION* why is it needed? All operations will be blocked 188e513de99SMauro Carvalho Chehab anyway after it reaches blk queue.) 189e513de99SMauro Carvalho Chehab 190e513de99SMauro Carvalho Chehab 4. Kicks queues in all devices on the host in the asses 191e513de99SMauro Carvalho Chehab 192e513de99SMauro Carvalho Chehab 193e513de99SMauro Carvalho Chehab2.1 EH through fine-grained callbacks 194e513de99SMauro Carvalho Chehab------------------------------------- 195e513de99SMauro Carvalho Chehab 196e513de99SMauro Carvalho Chehab2.1.1 Overview 197e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^ 198e513de99SMauro Carvalho Chehab 199e513de99SMauro Carvalho ChehabIf eh_strategy_handler() is not present, SCSI midlayer takes charge 200e513de99SMauro Carvalho Chehabof driving error handling. EH's goals are two - make LLDD, host and 201e513de99SMauro Carvalho Chehabdevice forget about timed out scmds and make them ready for new 202e513de99SMauro Carvalho Chehabcommands. A scmd is said to be recovered if the scmd is forgotten by 203e513de99SMauro Carvalho Chehablower layers and lower layers are ready to process or fail the scmd 204e513de99SMauro Carvalho Chehabagain. 205e513de99SMauro Carvalho Chehab 206e513de99SMauro Carvalho ChehabTo achieve these goals, EH performs recovery actions with increasing 207e513de99SMauro Carvalho Chehabseverity. Some actions are performed by issuing SCSI commands and 208e513de99SMauro Carvalho Chehabothers are performed by invoking one of the following fine-grained 209e513de99SMauro Carvalho Chehabhostt EH callbacks. Callbacks may be omitted and omitted ones are 210e513de99SMauro Carvalho Chehabconsidered to fail always. 211e513de99SMauro Carvalho Chehab 212e513de99SMauro Carvalho Chehab:: 213e513de99SMauro Carvalho Chehab 214e513de99SMauro Carvalho Chehab int (* eh_abort_handler)(struct scsi_cmnd *); 215e513de99SMauro Carvalho Chehab int (* eh_device_reset_handler)(struct scsi_cmnd *); 216e513de99SMauro Carvalho Chehab int (* eh_bus_reset_handler)(struct scsi_cmnd *); 217e513de99SMauro Carvalho Chehab int (* eh_host_reset_handler)(struct scsi_cmnd *); 218e513de99SMauro Carvalho Chehab 219e513de99SMauro Carvalho ChehabHigher-severity actions are taken only when lower-severity actions 220e513de99SMauro Carvalho Chehabcannot recover some of failed scmds. Also, note that failure of the 221e513de99SMauro Carvalho Chehabhighest-severity action means EH failure and results in offlining of 222e513de99SMauro Carvalho Chehaball unrecovered devices. 223e513de99SMauro Carvalho Chehab 224e513de99SMauro Carvalho ChehabDuring recovery, the following rules are followed 225e513de99SMauro Carvalho Chehab 226e513de99SMauro Carvalho Chehab - Recovery actions are performed on failed scmds on the to do list, 227e513de99SMauro Carvalho Chehab eh_work_q. If a recovery action succeeds for a scmd, recovered 228e513de99SMauro Carvalho Chehab scmds are removed from eh_work_q. 229e513de99SMauro Carvalho Chehab 230e513de99SMauro Carvalho Chehab Note that single recovery action on a scmd can recover multiple 231e513de99SMauro Carvalho Chehab scmds. e.g. resetting a device recovers all failed scmds on the 232e513de99SMauro Carvalho Chehab device. 233e513de99SMauro Carvalho Chehab 234e513de99SMauro Carvalho Chehab - Higher severity actions are taken iff eh_work_q is not empty after 235e513de99SMauro Carvalho Chehab lower severity actions are complete. 236e513de99SMauro Carvalho Chehab 237e513de99SMauro Carvalho Chehab - EH reuses failed scmds to issue commands for recovery. For 238e513de99SMauro Carvalho Chehab timed-out scmds, SCSI EH ensures that LLDD forgets about a scmd 239e513de99SMauro Carvalho Chehab before reusing it for EH commands. 240e513de99SMauro Carvalho Chehab 241e513de99SMauro Carvalho ChehabWhen a scmd is recovered, the scmd is moved from eh_work_q to EH 242e513de99SMauro Carvalho Chehablocal eh_done_q using scsi_eh_finish_cmd(). After all scmds are 243e513de99SMauro Carvalho Chehabrecovered (eh_work_q is empty), scsi_eh_flush_done_q() is invoked to 244e513de99SMauro Carvalho Chehabeither retry or error-finish (notify upper layer of failure) recovered 245e513de99SMauro Carvalho Chehabscmds. 246e513de99SMauro Carvalho Chehab 247e513de99SMauro Carvalho Chehabscmds are retried iff its sdev is still online (not offlined during 248e513de99SMauro Carvalho ChehabEH), REQ_FAILFAST is not set and ++scmd->retries is less than 249e513de99SMauro Carvalho Chehabscmd->allowed. 250e513de99SMauro Carvalho Chehab 251e513de99SMauro Carvalho Chehab 252e513de99SMauro Carvalho Chehab2.1.2 Flow of scmds through EH 253e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 254e513de99SMauro Carvalho Chehab 255e513de99SMauro Carvalho Chehab 1. Error completion / time out 256e513de99SMauro Carvalho Chehab 257e513de99SMauro Carvalho Chehab :ACTION: scsi_eh_scmd_add() is invoked for scmd 258e513de99SMauro Carvalho Chehab 259e513de99SMauro Carvalho Chehab - add scmd to shost->eh_cmd_q 260e513de99SMauro Carvalho Chehab - set SHOST_RECOVERY 261e513de99SMauro Carvalho Chehab - shost->host_failed++ 262e513de99SMauro Carvalho Chehab 263e513de99SMauro Carvalho Chehab :LOCKING: shost->host_lock 264e513de99SMauro Carvalho Chehab 265e513de99SMauro Carvalho Chehab 2. EH starts 266e513de99SMauro Carvalho Chehab 267e513de99SMauro Carvalho Chehab :ACTION: move all scmds to EH's local eh_work_q. shost->eh_cmd_q 268e513de99SMauro Carvalho Chehab is cleared. 269e513de99SMauro Carvalho Chehab 270e513de99SMauro Carvalho Chehab :LOCKING: shost->host_lock (not strictly necessary, just for 271e513de99SMauro Carvalho Chehab consistency) 272e513de99SMauro Carvalho Chehab 273e513de99SMauro Carvalho Chehab 3. scmd recovered 274e513de99SMauro Carvalho Chehab 275e513de99SMauro Carvalho Chehab :ACTION: scsi_eh_finish_cmd() is invoked to EH-finish scmd 276e513de99SMauro Carvalho Chehab 277e513de99SMauro Carvalho Chehab - scsi_setup_cmd_retry() 278e513de99SMauro Carvalho Chehab - move from local eh_work_q to local eh_done_q 279e513de99SMauro Carvalho Chehab 280e513de99SMauro Carvalho Chehab :LOCKING: none 281e513de99SMauro Carvalho Chehab 282e513de99SMauro Carvalho Chehab :CONCURRENCY: at most one thread per separate eh_work_q to 283e513de99SMauro Carvalho Chehab keep queue manipulation lockless 284e513de99SMauro Carvalho Chehab 285e513de99SMauro Carvalho Chehab 4. EH completes 286e513de99SMauro Carvalho Chehab 287e513de99SMauro Carvalho Chehab :ACTION: scsi_eh_flush_done_q() retries scmds or notifies upper 288e513de99SMauro Carvalho Chehab layer of failure. May be called concurrently but must have 289e513de99SMauro Carvalho Chehab a no more than one thread per separate eh_work_q to 290e513de99SMauro Carvalho Chehab manipulate the queue locklessly 291e513de99SMauro Carvalho Chehab 292e513de99SMauro Carvalho Chehab - scmd is removed from eh_done_q and scmd->eh_entry is cleared 293e513de99SMauro Carvalho Chehab - if retry is necessary, scmd is requeued using 294e513de99SMauro Carvalho Chehab scsi_queue_insert() 295e513de99SMauro Carvalho Chehab - otherwise, scsi_finish_command() is invoked for scmd 296e513de99SMauro Carvalho Chehab - zero shost->host_failed 297e513de99SMauro Carvalho Chehab 298e513de99SMauro Carvalho Chehab :LOCKING: queue or finish function performs appropriate locking 299e513de99SMauro Carvalho Chehab 300e513de99SMauro Carvalho Chehab 301e513de99SMauro Carvalho Chehab2.1.3 Flow of control 302e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^ 303e513de99SMauro Carvalho Chehab 304e513de99SMauro Carvalho Chehab EH through fine-grained callbacks start from scsi_unjam_host(). 305e513de99SMauro Carvalho Chehab 306e513de99SMauro Carvalho Chehab``scsi_unjam_host`` 307e513de99SMauro Carvalho Chehab 308e513de99SMauro Carvalho Chehab 1. Lock shost->host_lock, splice_init shost->eh_cmd_q into local 309e513de99SMauro Carvalho Chehab eh_work_q and unlock host_lock. Note that shost->eh_cmd_q is 310e513de99SMauro Carvalho Chehab cleared by this action. 311e513de99SMauro Carvalho Chehab 312e513de99SMauro Carvalho Chehab 2. Invoke scsi_eh_get_sense. 313e513de99SMauro Carvalho Chehab 314e513de99SMauro Carvalho Chehab ``scsi_eh_get_sense`` 315e513de99SMauro Carvalho Chehab 316e513de99SMauro Carvalho Chehab This action is taken for each error-completed 317e513de99SMauro Carvalho Chehab (!SCSI_EH_CANCEL_CMD) commands without valid sense data. Most 318e513de99SMauro Carvalho Chehab SCSI transports/LLDDs automatically acquire sense data on 319e513de99SMauro Carvalho Chehab command failures (autosense). Autosense is recommended for 320e513de99SMauro Carvalho Chehab performance reasons and as sense information could get out of 321e513de99SMauro Carvalho Chehab sync between occurrence of CHECK CONDITION and this action. 322e513de99SMauro Carvalho Chehab 323e513de99SMauro Carvalho Chehab Note that if autosense is not supported, scmd->sense_buffer 324e513de99SMauro Carvalho Chehab contains invalid sense data when error-completing the scmd 325e513de99SMauro Carvalho Chehab with scsi_done(). scsi_decide_disposition() always returns 326e513de99SMauro Carvalho Chehab FAILED in such cases thus invoking SCSI EH. When the scmd 327e513de99SMauro Carvalho Chehab reaches here, sense data is acquired and 328e513de99SMauro Carvalho Chehab scsi_decide_disposition() is called again. 329e513de99SMauro Carvalho Chehab 330e513de99SMauro Carvalho Chehab 1. Invoke scsi_request_sense() which issues REQUEST_SENSE 331e513de99SMauro Carvalho Chehab command. If fails, no action. Note that taking no action 332e513de99SMauro Carvalho Chehab causes higher-severity recovery to be taken for the scmd. 333e513de99SMauro Carvalho Chehab 334e513de99SMauro Carvalho Chehab 2. Invoke scsi_decide_disposition() on the scmd 335e513de99SMauro Carvalho Chehab 336e513de99SMauro Carvalho Chehab - SUCCESS 337e513de99SMauro Carvalho Chehab scmd->retries is set to scmd->allowed preventing 338e513de99SMauro Carvalho Chehab scsi_eh_flush_done_q() from retrying the scmd and 339e513de99SMauro Carvalho Chehab scsi_eh_finish_cmd() is invoked. 340e513de99SMauro Carvalho Chehab 341e513de99SMauro Carvalho Chehab - NEEDS_RETRY 342e513de99SMauro Carvalho Chehab scsi_eh_finish_cmd() invoked 343e513de99SMauro Carvalho Chehab 344e513de99SMauro Carvalho Chehab - otherwise 345e513de99SMauro Carvalho Chehab No action. 346e513de99SMauro Carvalho Chehab 347e513de99SMauro Carvalho Chehab 3. If !list_empty(&eh_work_q), invoke scsi_eh_abort_cmds(). 348e513de99SMauro Carvalho Chehab 349e513de99SMauro Carvalho Chehab ``scsi_eh_abort_cmds`` 350e513de99SMauro Carvalho Chehab 351e513de99SMauro Carvalho Chehab This action is taken for each timed out command when 352e513de99SMauro Carvalho Chehab no_async_abort is enabled in the host template. 353e513de99SMauro Carvalho Chehab hostt->eh_abort_handler() is invoked for each scmd. The 354e513de99SMauro Carvalho Chehab handler returns SUCCESS if it has succeeded to make LLDD and 355e513de99SMauro Carvalho Chehab all related hardware forget about the scmd. 356e513de99SMauro Carvalho Chehab 357e513de99SMauro Carvalho Chehab If a timedout scmd is successfully aborted and the sdev is 358e513de99SMauro Carvalho Chehab either offline or ready, scsi_eh_finish_cmd() is invoked for 359e513de99SMauro Carvalho Chehab the scmd. Otherwise, the scmd is left in eh_work_q for 360e513de99SMauro Carvalho Chehab higher-severity actions. 361e513de99SMauro Carvalho Chehab 362e513de99SMauro Carvalho Chehab Note that both offline and ready status mean that the sdev is 363e513de99SMauro Carvalho Chehab ready to process new scmds, where processing also implies 364e513de99SMauro Carvalho Chehab immediate failing; thus, if a sdev is in one of the two 365e513de99SMauro Carvalho Chehab states, no further recovery action is needed. 366e513de99SMauro Carvalho Chehab 367e513de99SMauro Carvalho Chehab Device readiness is tested using scsi_eh_tur() which issues 368e513de99SMauro Carvalho Chehab TEST_UNIT_READY command. Note that the scmd must have been 369e513de99SMauro Carvalho Chehab aborted successfully before reusing it for TEST_UNIT_READY. 370e513de99SMauro Carvalho Chehab 371e513de99SMauro Carvalho Chehab 4. If !list_empty(&eh_work_q), invoke scsi_eh_ready_devs() 372e513de99SMauro Carvalho Chehab 373e513de99SMauro Carvalho Chehab ``scsi_eh_ready_devs`` 374e513de99SMauro Carvalho Chehab 375e513de99SMauro Carvalho Chehab This function takes four increasingly more severe measures to 376e513de99SMauro Carvalho Chehab make failed sdevs ready for new commands. 377e513de99SMauro Carvalho Chehab 378e513de99SMauro Carvalho Chehab 1. Invoke scsi_eh_stu() 379e513de99SMauro Carvalho Chehab 380e513de99SMauro Carvalho Chehab ``scsi_eh_stu`` 381e513de99SMauro Carvalho Chehab 382e513de99SMauro Carvalho Chehab For each sdev which has failed scmds with valid sense data 383e513de99SMauro Carvalho Chehab of which scsi_check_sense()'s verdict is FAILED, 384e513de99SMauro Carvalho Chehab START_STOP_UNIT command is issued w/ start=1. Note that 385e513de99SMauro Carvalho Chehab as we explicitly choose error-completed scmds, it is known 386e513de99SMauro Carvalho Chehab that lower layers have forgotten about the scmd and we can 387e513de99SMauro Carvalho Chehab reuse it for STU. 388e513de99SMauro Carvalho Chehab 389e513de99SMauro Carvalho Chehab If STU succeeds and the sdev is either offline or ready, 390e513de99SMauro Carvalho Chehab all failed scmds on the sdev are EH-finished with 391e513de99SMauro Carvalho Chehab scsi_eh_finish_cmd(). 392e513de99SMauro Carvalho Chehab 393e513de99SMauro Carvalho Chehab *NOTE* If hostt->eh_abort_handler() isn't implemented or 394e513de99SMauro Carvalho Chehab failed, we may still have timed out scmds at this point 395e513de99SMauro Carvalho Chehab and STU doesn't make lower layers forget about those 396e513de99SMauro Carvalho Chehab scmds. Yet, this function EH-finish all scmds on the sdev 397e513de99SMauro Carvalho Chehab if STU succeeds leaving lower layers in an inconsistent 398e513de99SMauro Carvalho Chehab state. It seems that STU action should be taken only when 399e513de99SMauro Carvalho Chehab a sdev has no timed out scmd. 400e513de99SMauro Carvalho Chehab 401e513de99SMauro Carvalho Chehab 2. If !list_empty(&eh_work_q), invoke scsi_eh_bus_device_reset(). 402e513de99SMauro Carvalho Chehab 403e513de99SMauro Carvalho Chehab ``scsi_eh_bus_device_reset`` 404e513de99SMauro Carvalho Chehab 405e513de99SMauro Carvalho Chehab This action is very similar to scsi_eh_stu() except that, 406e513de99SMauro Carvalho Chehab instead of issuing STU, hostt->eh_device_reset_handler() 407e513de99SMauro Carvalho Chehab is used. Also, as we're not issuing SCSI commands and 408e513de99SMauro Carvalho Chehab resetting clears all scmds on the sdev, there is no need 409e513de99SMauro Carvalho Chehab to choose error-completed scmds. 410e513de99SMauro Carvalho Chehab 411e513de99SMauro Carvalho Chehab 3. If !list_empty(&eh_work_q), invoke scsi_eh_bus_reset() 412e513de99SMauro Carvalho Chehab 413e513de99SMauro Carvalho Chehab ``scsi_eh_bus_reset`` 414e513de99SMauro Carvalho Chehab 415e513de99SMauro Carvalho Chehab hostt->eh_bus_reset_handler() is invoked for each channel 416e513de99SMauro Carvalho Chehab with failed scmds. If bus reset succeeds, all failed 417e513de99SMauro Carvalho Chehab scmds on all ready or offline sdevs on the channel are 418e513de99SMauro Carvalho Chehab EH-finished. 419e513de99SMauro Carvalho Chehab 420e513de99SMauro Carvalho Chehab 4. If !list_empty(&eh_work_q), invoke scsi_eh_host_reset() 421e513de99SMauro Carvalho Chehab 422e513de99SMauro Carvalho Chehab ``scsi_eh_host_reset`` 423e513de99SMauro Carvalho Chehab 424e513de99SMauro Carvalho Chehab This is the last resort. hostt->eh_host_reset_handler() 425e513de99SMauro Carvalho Chehab is invoked. If host reset succeeds, all failed scmds on 426e513de99SMauro Carvalho Chehab all ready or offline sdevs on the host are EH-finished. 427e513de99SMauro Carvalho Chehab 428e513de99SMauro Carvalho Chehab 5. If !list_empty(&eh_work_q), invoke scsi_eh_offline_sdevs() 429e513de99SMauro Carvalho Chehab 430e513de99SMauro Carvalho Chehab ``scsi_eh_offline_sdevs`` 431e513de99SMauro Carvalho Chehab 432e513de99SMauro Carvalho Chehab Take all sdevs which still have unrecovered scmds offline 433e513de99SMauro Carvalho Chehab and EH-finish the scmds. 434e513de99SMauro Carvalho Chehab 435e513de99SMauro Carvalho Chehab 5. Invoke scsi_eh_flush_done_q(). 436e513de99SMauro Carvalho Chehab 437e513de99SMauro Carvalho Chehab ``scsi_eh_flush_done_q`` 438e513de99SMauro Carvalho Chehab 439e513de99SMauro Carvalho Chehab At this point all scmds are recovered (or given up) and 440e513de99SMauro Carvalho Chehab put on eh_done_q by scsi_eh_finish_cmd(). This function 441e513de99SMauro Carvalho Chehab flushes eh_done_q by either retrying or notifying upper 442e513de99SMauro Carvalho Chehab layer of failure of the scmds. 443e513de99SMauro Carvalho Chehab 444e513de99SMauro Carvalho Chehab 445e513de99SMauro Carvalho Chehab2.2 EH through transportt->eh_strategy_handler() 446e513de99SMauro Carvalho Chehab------------------------------------------------ 447e513de99SMauro Carvalho Chehab 448e513de99SMauro Carvalho Chehabtransportt->eh_strategy_handler() is invoked in the place of 449e513de99SMauro Carvalho Chehabscsi_unjam_host() and it is responsible for whole recovery process. 450e513de99SMauro Carvalho ChehabOn completion, the handler should have made lower layers forget about 451e513de99SMauro Carvalho Chehaball failed scmds and either ready for new commands or offline. Also, 452e513de99SMauro Carvalho Chehabit should perform SCSI EH maintenance chores to maintain integrity of 453e513de99SMauro Carvalho ChehabSCSI midlayer. IOW, of the steps described in [2-1-2], all steps 454e513de99SMauro Carvalho Chehabexcept for #1 must be implemented by eh_strategy_handler(). 455e513de99SMauro Carvalho Chehab 456e513de99SMauro Carvalho Chehab 457e513de99SMauro Carvalho Chehab2.2.1 Pre transportt->eh_strategy_handler() SCSI midlayer conditions 458e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 459e513de99SMauro Carvalho Chehab 460e513de99SMauro Carvalho Chehab The following conditions are true on entry to the handler. 461e513de99SMauro Carvalho Chehab 462e513de99SMauro Carvalho Chehab - Each failed scmd's eh_flags field is set appropriately. 463e513de99SMauro Carvalho Chehab 464e513de99SMauro Carvalho Chehab - Each failed scmd is linked on scmd->eh_cmd_q by scmd->eh_entry. 465e513de99SMauro Carvalho Chehab 466e513de99SMauro Carvalho Chehab - SHOST_RECOVERY is set. 467e513de99SMauro Carvalho Chehab 468e513de99SMauro Carvalho Chehab - shost->host_failed == shost->host_busy 469e513de99SMauro Carvalho Chehab 470e513de99SMauro Carvalho Chehab 471e513de99SMauro Carvalho Chehab2.2.2 Post transportt->eh_strategy_handler() SCSI midlayer conditions 472e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 473e513de99SMauro Carvalho Chehab 474e513de99SMauro Carvalho Chehab The following conditions must be true on exit from the handler. 475e513de99SMauro Carvalho Chehab 476e513de99SMauro Carvalho Chehab - shost->host_failed is zero. 477e513de99SMauro Carvalho Chehab 478e513de99SMauro Carvalho Chehab - Each scmd is in such a state that scsi_setup_cmd_retry() on the 479e513de99SMauro Carvalho Chehab scmd doesn't make any difference. 480e513de99SMauro Carvalho Chehab 481e513de99SMauro Carvalho Chehab - shost->eh_cmd_q is cleared. 482e513de99SMauro Carvalho Chehab 483e513de99SMauro Carvalho Chehab - Each scmd->eh_entry is cleared. 484e513de99SMauro Carvalho Chehab 485e513de99SMauro Carvalho Chehab - Either scsi_queue_insert() or scsi_finish_command() is called on 486e513de99SMauro Carvalho Chehab each scmd. Note that the handler is free to use scmd->retries and 487e513de99SMauro Carvalho Chehab ->allowed to limit the number of retries. 488e513de99SMauro Carvalho Chehab 489e513de99SMauro Carvalho Chehab 490e513de99SMauro Carvalho Chehab2.2.3 Things to consider 491e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^ 492e513de99SMauro Carvalho Chehab 493e513de99SMauro Carvalho Chehab - Know that timed out scmds are still active on lower layers. Make 494e513de99SMauro Carvalho Chehab lower layers forget about them before doing anything else with 495e513de99SMauro Carvalho Chehab those scmds. 496e513de99SMauro Carvalho Chehab 497e513de99SMauro Carvalho Chehab - For consistency, when accessing/modifying shost data structure, 498e513de99SMauro Carvalho Chehab grab shost->host_lock. 499e513de99SMauro Carvalho Chehab 500e513de99SMauro Carvalho Chehab - On completion, each failed sdev must have forgotten about all 501e513de99SMauro Carvalho Chehab active scmds. 502e513de99SMauro Carvalho Chehab 503e513de99SMauro Carvalho Chehab - On completion, each failed sdev must be ready for new commands or 504e513de99SMauro Carvalho Chehab offline. 505e513de99SMauro Carvalho Chehab 506e513de99SMauro Carvalho Chehab 507e513de99SMauro Carvalho ChehabTejun Heo 508e513de99SMauro Carvalho Chehabhtejun@gmail.com 509e513de99SMauro Carvalho Chehab 510e513de99SMauro Carvalho Chehab11th September 2005 511