1e513de99SMauro Carvalho Chehab.. SPDX-License-Identifier: GPL-2.0 2e513de99SMauro Carvalho Chehab 3e513de99SMauro Carvalho Chehab======= 4e513de99SMauro Carvalho ChehabSCSI EH 5e513de99SMauro Carvalho Chehab======= 6e513de99SMauro Carvalho Chehab 7e513de99SMauro Carvalho ChehabThis document describes SCSI midlayer error handling infrastructure. 8ce5c5d65SMauro Carvalho ChehabPlease refer to Documentation/scsi/scsi_mid_low_api.rst for more 9e513de99SMauro Carvalho Chehabinformation regarding SCSI midlayer. 10e513de99SMauro Carvalho Chehab 11e513de99SMauro Carvalho Chehab.. TABLE OF CONTENTS 12e513de99SMauro Carvalho Chehab 13e513de99SMauro Carvalho Chehab [1] How SCSI commands travel through the midlayer and to EH 14e513de99SMauro Carvalho Chehab [1-1] struct scsi_cmnd 15e513de99SMauro Carvalho Chehab [1-2] How do scmd's get completed? 16e513de99SMauro Carvalho Chehab [1-2-1] Completing a scmd w/ scsi_done 17e513de99SMauro Carvalho Chehab [1-2-2] Completing a scmd w/ timeout 18e513de99SMauro Carvalho Chehab [1-3] How EH takes over 19e513de99SMauro Carvalho Chehab [2] How SCSI EH works 20e513de99SMauro Carvalho Chehab [2-1] EH through fine-grained callbacks 21e513de99SMauro Carvalho Chehab [2-1-1] Overview 22e513de99SMauro Carvalho Chehab [2-1-2] Flow of scmds through EH 23e513de99SMauro Carvalho Chehab [2-1-3] Flow of control 24e513de99SMauro Carvalho Chehab [2-2] EH through transportt->eh_strategy_handler() 25e513de99SMauro Carvalho Chehab [2-2-1] Pre transportt->eh_strategy_handler() SCSI midlayer conditions 26e513de99SMauro Carvalho Chehab [2-2-2] Post transportt->eh_strategy_handler() SCSI midlayer conditions 27e513de99SMauro Carvalho Chehab [2-2-3] Things to consider 28e513de99SMauro Carvalho Chehab 29e513de99SMauro Carvalho Chehab 30e513de99SMauro Carvalho Chehab1. How SCSI commands travel through the midlayer and to EH 31e513de99SMauro Carvalho Chehab========================================================== 32e513de99SMauro Carvalho Chehab 33e513de99SMauro Carvalho Chehab1.1 struct scsi_cmnd 34e513de99SMauro Carvalho Chehab-------------------- 35e513de99SMauro Carvalho Chehab 36e513de99SMauro Carvalho ChehabEach SCSI command is represented with struct scsi_cmnd (== scmd). A 37e513de99SMauro Carvalho Chehabscmd has two list_head's to link itself into lists. The two are 38e513de99SMauro Carvalho Chehabscmd->list and scmd->eh_entry. The former is used for free list or 39e513de99SMauro Carvalho Chehabper-device allocated scmd list and not of much interest to this EH 40e513de99SMauro Carvalho Chehabdiscussion. The latter is used for completion and EH lists and unless 41e513de99SMauro Carvalho Chehabotherwise stated scmds are always linked using scmd->eh_entry in this 42e513de99SMauro Carvalho Chehabdiscussion. 43e513de99SMauro Carvalho Chehab 44e513de99SMauro Carvalho Chehab 45e513de99SMauro Carvalho Chehab1.2 How do scmd's get completed? 46e513de99SMauro Carvalho Chehab-------------------------------- 47e513de99SMauro Carvalho Chehab 48e513de99SMauro Carvalho ChehabOnce LLDD gets hold of a scmd, either the LLDD will complete the 49e513de99SMauro Carvalho Chehabcommand by calling scsi_done callback passed from midlayer when 50e513de99SMauro Carvalho Chehabinvoking hostt->queuecommand() or the block layer will time it out. 51e513de99SMauro Carvalho Chehab 52e513de99SMauro Carvalho Chehab 53e513de99SMauro Carvalho Chehab1.2.1 Completing a scmd w/ scsi_done 54e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 55e513de99SMauro Carvalho Chehab 56e513de99SMauro Carvalho ChehabFor all non-EH commands, scsi_done() is the completion callback. It 57e513de99SMauro Carvalho Chehabjust calls blk_complete_request() to delete the block layer timer and 58e513de99SMauro Carvalho Chehabraise SCSI_SOFTIRQ 59e513de99SMauro Carvalho Chehab 60e513de99SMauro Carvalho ChehabSCSI_SOFTIRQ handler scsi_softirq calls scsi_decide_disposition() to 61e513de99SMauro Carvalho Chehabdetermine what to do with the command. scsi_decide_disposition() 62e513de99SMauro Carvalho Chehablooks at the scmd->result value and sense data to determine what to do 63e513de99SMauro Carvalho Chehabwith the command. 64e513de99SMauro Carvalho Chehab 65e513de99SMauro Carvalho Chehab - SUCCESS 66e513de99SMauro Carvalho Chehab 67e513de99SMauro Carvalho Chehab scsi_finish_command() is invoked for the command. The 68e513de99SMauro Carvalho Chehab function does some maintenance chores and then calls 69e513de99SMauro Carvalho Chehab scsi_io_completion() to finish the I/O. 70e513de99SMauro Carvalho Chehab scsi_io_completion() then notifies the block layer on 71e513de99SMauro Carvalho Chehab the completed request by calling blk_end_request and 72e513de99SMauro Carvalho Chehab friends or figures out what to do with the remainder 73e513de99SMauro Carvalho Chehab of the data in case of an error. 74e513de99SMauro Carvalho Chehab 75e513de99SMauro Carvalho Chehab - NEEDS_RETRY 76e513de99SMauro Carvalho Chehab 77e513de99SMauro Carvalho Chehab - ADD_TO_MLQUEUE 78e513de99SMauro Carvalho Chehab 79e513de99SMauro Carvalho Chehab scmd is requeued to blk queue. 80e513de99SMauro Carvalho Chehab 81e513de99SMauro Carvalho Chehab - otherwise 82e513de99SMauro Carvalho Chehab 83e513de99SMauro Carvalho Chehab scsi_eh_scmd_add(scmd) is invoked for the command. See 84e513de99SMauro Carvalho Chehab [1-3] for details of this function. 85e513de99SMauro Carvalho Chehab 86e513de99SMauro Carvalho Chehab 87e513de99SMauro Carvalho Chehab1.2.2 Completing a scmd w/ timeout 88e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 89e513de99SMauro Carvalho Chehab 90deef1be1SJohn GarryThe timeout handler is scsi_timeout(). When a timeout occurs, this function 91e513de99SMauro Carvalho Chehab 92e513de99SMauro Carvalho Chehab 1. invokes optional hostt->eh_timed_out() callback. Return value can 93e513de99SMauro Carvalho Chehab be one of 94e513de99SMauro Carvalho Chehab 95*dee7121eSBart Van Assche - SCSI_EH_RESET_TIMER 96e513de99SMauro Carvalho Chehab This indicates that more time is required to finish the 975c139ce9SKhazhismel Kumykov command. Timer is restarted. 98e513de99SMauro Carvalho Chehab 99*dee7121eSBart Van Assche - SCSI_EH_NOT_HANDLED 100e513de99SMauro Carvalho Chehab eh_timed_out() callback did not handle the command. 101e513de99SMauro Carvalho Chehab Step #2 is taken. 102e513de99SMauro Carvalho Chehab 103*dee7121eSBart Van Assche - SCSI_EH_DONE 104*dee7121eSBart Van Assche eh_timed_out() completed the command. 105*dee7121eSBart Van Assche 1065c139ce9SKhazhismel Kumykov 2. scsi_abort_command() is invoked to schedule an asynchronous abort which may 1075c139ce9SKhazhismel Kumykov issue a retry scmd->allowed + 1 times. Asynchronous aborts are not invoked 1085c139ce9SKhazhismel Kumykov for commands for which the SCSI_EH_ABORT_SCHEDULED flag is set (this 1095c139ce9SKhazhismel Kumykov indicates that the command already had been aborted once, and this is a 1105c139ce9SKhazhismel Kumykov retry which failed), when retries are exceeded, or when the EH deadline is 1115c139ce9SKhazhismel Kumykov expired. In these cases Step #3 is taken. 112e513de99SMauro Carvalho Chehab 113e513de99SMauro Carvalho Chehab 3. scsi_eh_scmd_add(scmd, SCSI_EH_CANCEL_CMD) is invoked for the 114e513de99SMauro Carvalho Chehab command. See [1-4] for more information. 115e513de99SMauro Carvalho Chehab 116e513de99SMauro Carvalho Chehab1.3 Asynchronous command aborts 117e513de99SMauro Carvalho Chehab------------------------------- 118e513de99SMauro Carvalho Chehab 119e513de99SMauro Carvalho Chehab After a timeout occurs a command abort is scheduled from 120e513de99SMauro Carvalho Chehab scsi_abort_command(). If the abort is successful the command 121e513de99SMauro Carvalho Chehab will either be retried (if the number of retries is not exhausted) 122e513de99SMauro Carvalho Chehab or terminated with DID_TIME_OUT. 123e513de99SMauro Carvalho Chehab 124e513de99SMauro Carvalho Chehab Otherwise scsi_eh_scmd_add() is invoked for the command. 125e513de99SMauro Carvalho Chehab See [1-4] for more information. 126e513de99SMauro Carvalho Chehab 127e513de99SMauro Carvalho Chehab1.4 How EH takes over 128e513de99SMauro Carvalho Chehab--------------------- 129e513de99SMauro Carvalho Chehab 130e513de99SMauro Carvalho Chehabscmds enter EH via scsi_eh_scmd_add(), which does the following. 131e513de99SMauro Carvalho Chehab 132e513de99SMauro Carvalho Chehab 1. Links scmd->eh_entry to shost->eh_cmd_q 133e513de99SMauro Carvalho Chehab 134e513de99SMauro Carvalho Chehab 2. Sets SHOST_RECOVERY bit in shost->shost_state 135e513de99SMauro Carvalho Chehab 136e513de99SMauro Carvalho Chehab 3. Increments shost->host_failed 137e513de99SMauro Carvalho Chehab 138e513de99SMauro Carvalho Chehab 4. Wakes up SCSI EH thread if shost->host_busy == shost->host_failed 139e513de99SMauro Carvalho Chehab 140e513de99SMauro Carvalho ChehabAs can be seen above, once any scmd is added to shost->eh_cmd_q, 141e513de99SMauro Carvalho ChehabSHOST_RECOVERY shost_state bit is turned on. This prevents any new 142e513de99SMauro Carvalho Chehabscmd to be issued from blk queue to the host; eventually, all scmds on 143e513de99SMauro Carvalho Chehabthe host either complete normally, fail and get added to eh_cmd_q, or 144e513de99SMauro Carvalho Chehabtime out and get added to shost->eh_cmd_q. 145e513de99SMauro Carvalho Chehab 146e513de99SMauro Carvalho ChehabIf all scmds either complete or fail, the number of in-flight scmds 147e513de99SMauro Carvalho Chehabbecomes equal to the number of failed scmds - i.e. shost->host_busy == 148e513de99SMauro Carvalho Chehabshost->host_failed. This wakes up SCSI EH thread. So, once woken up, 149e513de99SMauro Carvalho ChehabSCSI EH thread can expect that all in-flight commands have failed and 150e513de99SMauro Carvalho Chehabare linked on shost->eh_cmd_q. 151e513de99SMauro Carvalho Chehab 152e513de99SMauro Carvalho ChehabNote that this does not mean lower layers are quiescent. If a LLDD 153e513de99SMauro Carvalho Chehabcompleted a scmd with error status, the LLDD and lower layers are 154e513de99SMauro Carvalho Chehabassumed to forget about the scmd at that point. However, if a scmd 155e513de99SMauro Carvalho Chehabhas timed out, unless hostt->eh_timed_out() made lower layers forget 156e513de99SMauro Carvalho Chehababout the scmd, which currently no LLDD does, the command is still 157e513de99SMauro Carvalho Chehabactive as long as lower layers are concerned and completion could 158e513de99SMauro Carvalho Chehaboccur at any time. Of course, all such completions are ignored as the 159e513de99SMauro Carvalho Chehabtimer has already expired. 160e513de99SMauro Carvalho Chehab 161e513de99SMauro Carvalho ChehabWe'll talk about how SCSI EH takes actions to abort - make LLDD 162e513de99SMauro Carvalho Chehabforget about - timed out scmds later. 163e513de99SMauro Carvalho Chehab 164e513de99SMauro Carvalho Chehab 165e513de99SMauro Carvalho Chehab2. How SCSI EH works 166e513de99SMauro Carvalho Chehab==================== 167e513de99SMauro Carvalho Chehab 168e513de99SMauro Carvalho ChehabLLDD's can implement SCSI EH actions in one of the following two 169e513de99SMauro Carvalho Chehabways. 170e513de99SMauro Carvalho Chehab 171e513de99SMauro Carvalho Chehab - Fine-grained EH callbacks 172e513de99SMauro Carvalho Chehab LLDD can implement fine-grained EH callbacks and let SCSI 173e513de99SMauro Carvalho Chehab midlayer drive error handling and call appropriate callbacks. 174e513de99SMauro Carvalho Chehab This will be discussed further in [2-1]. 175e513de99SMauro Carvalho Chehab 176e513de99SMauro Carvalho Chehab - eh_strategy_handler() callback 177e513de99SMauro Carvalho Chehab This is one big callback which should perform whole error 178e513de99SMauro Carvalho Chehab handling. As such, it should do all chores the SCSI midlayer 179e513de99SMauro Carvalho Chehab performs during recovery. This will be discussed in [2-2]. 180e513de99SMauro Carvalho Chehab 181e513de99SMauro Carvalho ChehabOnce recovery is complete, SCSI EH resumes normal operation by 182e513de99SMauro Carvalho Chehabcalling scsi_restart_operations(), which 183e513de99SMauro Carvalho Chehab 184e513de99SMauro Carvalho Chehab 1. Checks if door locking is needed and locks door. 185e513de99SMauro Carvalho Chehab 186e513de99SMauro Carvalho Chehab 2. Clears SHOST_RECOVERY shost_state bit 187e513de99SMauro Carvalho Chehab 188e513de99SMauro Carvalho Chehab 3. Wakes up waiters on shost->host_wait. This occurs if someone 189e513de99SMauro Carvalho Chehab calls scsi_block_when_processing_errors() on the host. 190e513de99SMauro Carvalho Chehab (*QUESTION* why is it needed? All operations will be blocked 191e513de99SMauro Carvalho Chehab anyway after it reaches blk queue.) 192e513de99SMauro Carvalho Chehab 193e513de99SMauro Carvalho Chehab 4. Kicks queues in all devices on the host in the asses 194e513de99SMauro Carvalho Chehab 195e513de99SMauro Carvalho Chehab 196e513de99SMauro Carvalho Chehab2.1 EH through fine-grained callbacks 197e513de99SMauro Carvalho Chehab------------------------------------- 198e513de99SMauro Carvalho Chehab 199e513de99SMauro Carvalho Chehab2.1.1 Overview 200e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^ 201e513de99SMauro Carvalho Chehab 202e513de99SMauro Carvalho ChehabIf eh_strategy_handler() is not present, SCSI midlayer takes charge 203e513de99SMauro Carvalho Chehabof driving error handling. EH's goals are two - make LLDD, host and 204e513de99SMauro Carvalho Chehabdevice forget about timed out scmds and make them ready for new 205e513de99SMauro Carvalho Chehabcommands. A scmd is said to be recovered if the scmd is forgotten by 206e513de99SMauro Carvalho Chehablower layers and lower layers are ready to process or fail the scmd 207e513de99SMauro Carvalho Chehabagain. 208e513de99SMauro Carvalho Chehab 209e513de99SMauro Carvalho ChehabTo achieve these goals, EH performs recovery actions with increasing 210e513de99SMauro Carvalho Chehabseverity. Some actions are performed by issuing SCSI commands and 211e513de99SMauro Carvalho Chehabothers are performed by invoking one of the following fine-grained 212e513de99SMauro Carvalho Chehabhostt EH callbacks. Callbacks may be omitted and omitted ones are 213e513de99SMauro Carvalho Chehabconsidered to fail always. 214e513de99SMauro Carvalho Chehab 215e513de99SMauro Carvalho Chehab:: 216e513de99SMauro Carvalho Chehab 217e513de99SMauro Carvalho Chehab int (* eh_abort_handler)(struct scsi_cmnd *); 218e513de99SMauro Carvalho Chehab int (* eh_device_reset_handler)(struct scsi_cmnd *); 219e513de99SMauro Carvalho Chehab int (* eh_bus_reset_handler)(struct scsi_cmnd *); 220e513de99SMauro Carvalho Chehab int (* eh_host_reset_handler)(struct scsi_cmnd *); 221e513de99SMauro Carvalho Chehab 222e513de99SMauro Carvalho ChehabHigher-severity actions are taken only when lower-severity actions 223e513de99SMauro Carvalho Chehabcannot recover some of failed scmds. Also, note that failure of the 224e513de99SMauro Carvalho Chehabhighest-severity action means EH failure and results in offlining of 225e513de99SMauro Carvalho Chehaball unrecovered devices. 226e513de99SMauro Carvalho Chehab 227e513de99SMauro Carvalho ChehabDuring recovery, the following rules are followed 228e513de99SMauro Carvalho Chehab 229e513de99SMauro Carvalho Chehab - Recovery actions are performed on failed scmds on the to do list, 230e513de99SMauro Carvalho Chehab eh_work_q. If a recovery action succeeds for a scmd, recovered 231e513de99SMauro Carvalho Chehab scmds are removed from eh_work_q. 232e513de99SMauro Carvalho Chehab 233e513de99SMauro Carvalho Chehab Note that single recovery action on a scmd can recover multiple 234e513de99SMauro Carvalho Chehab scmds. e.g. resetting a device recovers all failed scmds on the 235e513de99SMauro Carvalho Chehab device. 236e513de99SMauro Carvalho Chehab 237e513de99SMauro Carvalho Chehab - Higher severity actions are taken iff eh_work_q is not empty after 238e513de99SMauro Carvalho Chehab lower severity actions are complete. 239e513de99SMauro Carvalho Chehab 240e513de99SMauro Carvalho Chehab - EH reuses failed scmds to issue commands for recovery. For 241e513de99SMauro Carvalho Chehab timed-out scmds, SCSI EH ensures that LLDD forgets about a scmd 242e513de99SMauro Carvalho Chehab before reusing it for EH commands. 243e513de99SMauro Carvalho Chehab 244e513de99SMauro Carvalho ChehabWhen a scmd is recovered, the scmd is moved from eh_work_q to EH 245e513de99SMauro Carvalho Chehablocal eh_done_q using scsi_eh_finish_cmd(). After all scmds are 246e513de99SMauro Carvalho Chehabrecovered (eh_work_q is empty), scsi_eh_flush_done_q() is invoked to 247e513de99SMauro Carvalho Chehabeither retry or error-finish (notify upper layer of failure) recovered 248e513de99SMauro Carvalho Chehabscmds. 249e513de99SMauro Carvalho Chehab 250e513de99SMauro Carvalho Chehabscmds are retried iff its sdev is still online (not offlined during 251e513de99SMauro Carvalho ChehabEH), REQ_FAILFAST is not set and ++scmd->retries is less than 252e513de99SMauro Carvalho Chehabscmd->allowed. 253e513de99SMauro Carvalho Chehab 254e513de99SMauro Carvalho Chehab 255e513de99SMauro Carvalho Chehab2.1.2 Flow of scmds through EH 256e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 257e513de99SMauro Carvalho Chehab 258e513de99SMauro Carvalho Chehab 1. Error completion / time out 259e513de99SMauro Carvalho Chehab 260e513de99SMauro Carvalho Chehab :ACTION: scsi_eh_scmd_add() is invoked for scmd 261e513de99SMauro Carvalho Chehab 262e513de99SMauro Carvalho Chehab - add scmd to shost->eh_cmd_q 263e513de99SMauro Carvalho Chehab - set SHOST_RECOVERY 264e513de99SMauro Carvalho Chehab - shost->host_failed++ 265e513de99SMauro Carvalho Chehab 266e513de99SMauro Carvalho Chehab :LOCKING: shost->host_lock 267e513de99SMauro Carvalho Chehab 268e513de99SMauro Carvalho Chehab 2. EH starts 269e513de99SMauro Carvalho Chehab 270e513de99SMauro Carvalho Chehab :ACTION: move all scmds to EH's local eh_work_q. shost->eh_cmd_q 271e513de99SMauro Carvalho Chehab is cleared. 272e513de99SMauro Carvalho Chehab 273e513de99SMauro Carvalho Chehab :LOCKING: shost->host_lock (not strictly necessary, just for 274e513de99SMauro Carvalho Chehab consistency) 275e513de99SMauro Carvalho Chehab 276e513de99SMauro Carvalho Chehab 3. scmd recovered 277e513de99SMauro Carvalho Chehab 278e513de99SMauro Carvalho Chehab :ACTION: scsi_eh_finish_cmd() is invoked to EH-finish scmd 279e513de99SMauro Carvalho Chehab 280e513de99SMauro Carvalho Chehab - scsi_setup_cmd_retry() 281e513de99SMauro Carvalho Chehab - move from local eh_work_q to local eh_done_q 282e513de99SMauro Carvalho Chehab 283e513de99SMauro Carvalho Chehab :LOCKING: none 284e513de99SMauro Carvalho Chehab 285e513de99SMauro Carvalho Chehab :CONCURRENCY: at most one thread per separate eh_work_q to 286e513de99SMauro Carvalho Chehab keep queue manipulation lockless 287e513de99SMauro Carvalho Chehab 288e513de99SMauro Carvalho Chehab 4. EH completes 289e513de99SMauro Carvalho Chehab 290e513de99SMauro Carvalho Chehab :ACTION: scsi_eh_flush_done_q() retries scmds or notifies upper 291e513de99SMauro Carvalho Chehab layer of failure. May be called concurrently but must have 292e513de99SMauro Carvalho Chehab a no more than one thread per separate eh_work_q to 293e513de99SMauro Carvalho Chehab manipulate the queue locklessly 294e513de99SMauro Carvalho Chehab 295e513de99SMauro Carvalho Chehab - scmd is removed from eh_done_q and scmd->eh_entry is cleared 296e513de99SMauro Carvalho Chehab - if retry is necessary, scmd is requeued using 297e513de99SMauro Carvalho Chehab scsi_queue_insert() 298e513de99SMauro Carvalho Chehab - otherwise, scsi_finish_command() is invoked for scmd 299e513de99SMauro Carvalho Chehab - zero shost->host_failed 300e513de99SMauro Carvalho Chehab 301e513de99SMauro Carvalho Chehab :LOCKING: queue or finish function performs appropriate locking 302e513de99SMauro Carvalho Chehab 303e513de99SMauro Carvalho Chehab 304e513de99SMauro Carvalho Chehab2.1.3 Flow of control 305e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^ 306e513de99SMauro Carvalho Chehab 307e513de99SMauro Carvalho Chehab EH through fine-grained callbacks start from scsi_unjam_host(). 308e513de99SMauro Carvalho Chehab 309e513de99SMauro Carvalho Chehab``scsi_unjam_host`` 310e513de99SMauro Carvalho Chehab 311e513de99SMauro Carvalho Chehab 1. Lock shost->host_lock, splice_init shost->eh_cmd_q into local 312e513de99SMauro Carvalho Chehab eh_work_q and unlock host_lock. Note that shost->eh_cmd_q is 313e513de99SMauro Carvalho Chehab cleared by this action. 314e513de99SMauro Carvalho Chehab 315e513de99SMauro Carvalho Chehab 2. Invoke scsi_eh_get_sense. 316e513de99SMauro Carvalho Chehab 317e513de99SMauro Carvalho Chehab ``scsi_eh_get_sense`` 318e513de99SMauro Carvalho Chehab 319e513de99SMauro Carvalho Chehab This action is taken for each error-completed 320e513de99SMauro Carvalho Chehab (!SCSI_EH_CANCEL_CMD) commands without valid sense data. Most 321e513de99SMauro Carvalho Chehab SCSI transports/LLDDs automatically acquire sense data on 322e513de99SMauro Carvalho Chehab command failures (autosense). Autosense is recommended for 323e513de99SMauro Carvalho Chehab performance reasons and as sense information could get out of 324e513de99SMauro Carvalho Chehab sync between occurrence of CHECK CONDITION and this action. 325e513de99SMauro Carvalho Chehab 326e513de99SMauro Carvalho Chehab Note that if autosense is not supported, scmd->sense_buffer 327e513de99SMauro Carvalho Chehab contains invalid sense data when error-completing the scmd 328e513de99SMauro Carvalho Chehab with scsi_done(). scsi_decide_disposition() always returns 329e513de99SMauro Carvalho Chehab FAILED in such cases thus invoking SCSI EH. When the scmd 330e513de99SMauro Carvalho Chehab reaches here, sense data is acquired and 331e513de99SMauro Carvalho Chehab scsi_decide_disposition() is called again. 332e513de99SMauro Carvalho Chehab 333e513de99SMauro Carvalho Chehab 1. Invoke scsi_request_sense() which issues REQUEST_SENSE 334e513de99SMauro Carvalho Chehab command. If fails, no action. Note that taking no action 335e513de99SMauro Carvalho Chehab causes higher-severity recovery to be taken for the scmd. 336e513de99SMauro Carvalho Chehab 337e513de99SMauro Carvalho Chehab 2. Invoke scsi_decide_disposition() on the scmd 338e513de99SMauro Carvalho Chehab 339e513de99SMauro Carvalho Chehab - SUCCESS 340e513de99SMauro Carvalho Chehab scmd->retries is set to scmd->allowed preventing 341e513de99SMauro Carvalho Chehab scsi_eh_flush_done_q() from retrying the scmd and 342e513de99SMauro Carvalho Chehab scsi_eh_finish_cmd() is invoked. 343e513de99SMauro Carvalho Chehab 344e513de99SMauro Carvalho Chehab - NEEDS_RETRY 345e513de99SMauro Carvalho Chehab scsi_eh_finish_cmd() invoked 346e513de99SMauro Carvalho Chehab 347e513de99SMauro Carvalho Chehab - otherwise 348e513de99SMauro Carvalho Chehab No action. 349e513de99SMauro Carvalho Chehab 350e513de99SMauro Carvalho Chehab 3. If !list_empty(&eh_work_q), invoke scsi_eh_abort_cmds(). 351e513de99SMauro Carvalho Chehab 352e513de99SMauro Carvalho Chehab ``scsi_eh_abort_cmds`` 353e513de99SMauro Carvalho Chehab 354e513de99SMauro Carvalho Chehab This action is taken for each timed out command when 355e513de99SMauro Carvalho Chehab no_async_abort is enabled in the host template. 356e513de99SMauro Carvalho Chehab hostt->eh_abort_handler() is invoked for each scmd. The 357e513de99SMauro Carvalho Chehab handler returns SUCCESS if it has succeeded to make LLDD and 358e513de99SMauro Carvalho Chehab all related hardware forget about the scmd. 359e513de99SMauro Carvalho Chehab 360e513de99SMauro Carvalho Chehab If a timedout scmd is successfully aborted and the sdev is 361e513de99SMauro Carvalho Chehab either offline or ready, scsi_eh_finish_cmd() is invoked for 362e513de99SMauro Carvalho Chehab the scmd. Otherwise, the scmd is left in eh_work_q for 363e513de99SMauro Carvalho Chehab higher-severity actions. 364e513de99SMauro Carvalho Chehab 365e513de99SMauro Carvalho Chehab Note that both offline and ready status mean that the sdev is 366e513de99SMauro Carvalho Chehab ready to process new scmds, where processing also implies 367e513de99SMauro Carvalho Chehab immediate failing; thus, if a sdev is in one of the two 368e513de99SMauro Carvalho Chehab states, no further recovery action is needed. 369e513de99SMauro Carvalho Chehab 370e513de99SMauro Carvalho Chehab Device readiness is tested using scsi_eh_tur() which issues 371e513de99SMauro Carvalho Chehab TEST_UNIT_READY command. Note that the scmd must have been 372e513de99SMauro Carvalho Chehab aborted successfully before reusing it for TEST_UNIT_READY. 373e513de99SMauro Carvalho Chehab 374e513de99SMauro Carvalho Chehab 4. If !list_empty(&eh_work_q), invoke scsi_eh_ready_devs() 375e513de99SMauro Carvalho Chehab 376e513de99SMauro Carvalho Chehab ``scsi_eh_ready_devs`` 377e513de99SMauro Carvalho Chehab 378e513de99SMauro Carvalho Chehab This function takes four increasingly more severe measures to 379e513de99SMauro Carvalho Chehab make failed sdevs ready for new commands. 380e513de99SMauro Carvalho Chehab 381e513de99SMauro Carvalho Chehab 1. Invoke scsi_eh_stu() 382e513de99SMauro Carvalho Chehab 383e513de99SMauro Carvalho Chehab ``scsi_eh_stu`` 384e513de99SMauro Carvalho Chehab 385e513de99SMauro Carvalho Chehab For each sdev which has failed scmds with valid sense data 386e513de99SMauro Carvalho Chehab of which scsi_check_sense()'s verdict is FAILED, 387e513de99SMauro Carvalho Chehab START_STOP_UNIT command is issued w/ start=1. Note that 388e513de99SMauro Carvalho Chehab as we explicitly choose error-completed scmds, it is known 389e513de99SMauro Carvalho Chehab that lower layers have forgotten about the scmd and we can 390e513de99SMauro Carvalho Chehab reuse it for STU. 391e513de99SMauro Carvalho Chehab 392e513de99SMauro Carvalho Chehab If STU succeeds and the sdev is either offline or ready, 393e513de99SMauro Carvalho Chehab all failed scmds on the sdev are EH-finished with 394e513de99SMauro Carvalho Chehab scsi_eh_finish_cmd(). 395e513de99SMauro Carvalho Chehab 396e513de99SMauro Carvalho Chehab *NOTE* If hostt->eh_abort_handler() isn't implemented or 397e513de99SMauro Carvalho Chehab failed, we may still have timed out scmds at this point 398e513de99SMauro Carvalho Chehab and STU doesn't make lower layers forget about those 399e513de99SMauro Carvalho Chehab scmds. Yet, this function EH-finish all scmds on the sdev 400e513de99SMauro Carvalho Chehab if STU succeeds leaving lower layers in an inconsistent 401e513de99SMauro Carvalho Chehab state. It seems that STU action should be taken only when 402e513de99SMauro Carvalho Chehab a sdev has no timed out scmd. 403e513de99SMauro Carvalho Chehab 404e513de99SMauro Carvalho Chehab 2. If !list_empty(&eh_work_q), invoke scsi_eh_bus_device_reset(). 405e513de99SMauro Carvalho Chehab 406e513de99SMauro Carvalho Chehab ``scsi_eh_bus_device_reset`` 407e513de99SMauro Carvalho Chehab 408e513de99SMauro Carvalho Chehab This action is very similar to scsi_eh_stu() except that, 409e513de99SMauro Carvalho Chehab instead of issuing STU, hostt->eh_device_reset_handler() 410e513de99SMauro Carvalho Chehab is used. Also, as we're not issuing SCSI commands and 411e513de99SMauro Carvalho Chehab resetting clears all scmds on the sdev, there is no need 412e513de99SMauro Carvalho Chehab to choose error-completed scmds. 413e513de99SMauro Carvalho Chehab 414e513de99SMauro Carvalho Chehab 3. If !list_empty(&eh_work_q), invoke scsi_eh_bus_reset() 415e513de99SMauro Carvalho Chehab 416e513de99SMauro Carvalho Chehab ``scsi_eh_bus_reset`` 417e513de99SMauro Carvalho Chehab 418e513de99SMauro Carvalho Chehab hostt->eh_bus_reset_handler() is invoked for each channel 419e513de99SMauro Carvalho Chehab with failed scmds. If bus reset succeeds, all failed 420e513de99SMauro Carvalho Chehab scmds on all ready or offline sdevs on the channel are 421e513de99SMauro Carvalho Chehab EH-finished. 422e513de99SMauro Carvalho Chehab 423e513de99SMauro Carvalho Chehab 4. If !list_empty(&eh_work_q), invoke scsi_eh_host_reset() 424e513de99SMauro Carvalho Chehab 425e513de99SMauro Carvalho Chehab ``scsi_eh_host_reset`` 426e513de99SMauro Carvalho Chehab 427e513de99SMauro Carvalho Chehab This is the last resort. hostt->eh_host_reset_handler() 428e513de99SMauro Carvalho Chehab is invoked. If host reset succeeds, all failed scmds on 429e513de99SMauro Carvalho Chehab all ready or offline sdevs on the host are EH-finished. 430e513de99SMauro Carvalho Chehab 431e513de99SMauro Carvalho Chehab 5. If !list_empty(&eh_work_q), invoke scsi_eh_offline_sdevs() 432e513de99SMauro Carvalho Chehab 433e513de99SMauro Carvalho Chehab ``scsi_eh_offline_sdevs`` 434e513de99SMauro Carvalho Chehab 435e513de99SMauro Carvalho Chehab Take all sdevs which still have unrecovered scmds offline 436e513de99SMauro Carvalho Chehab and EH-finish the scmds. 437e513de99SMauro Carvalho Chehab 438e513de99SMauro Carvalho Chehab 5. Invoke scsi_eh_flush_done_q(). 439e513de99SMauro Carvalho Chehab 440e513de99SMauro Carvalho Chehab ``scsi_eh_flush_done_q`` 441e513de99SMauro Carvalho Chehab 442e513de99SMauro Carvalho Chehab At this point all scmds are recovered (or given up) and 443e513de99SMauro Carvalho Chehab put on eh_done_q by scsi_eh_finish_cmd(). This function 444e513de99SMauro Carvalho Chehab flushes eh_done_q by either retrying or notifying upper 445e513de99SMauro Carvalho Chehab layer of failure of the scmds. 446e513de99SMauro Carvalho Chehab 447e513de99SMauro Carvalho Chehab 448e513de99SMauro Carvalho Chehab2.2 EH through transportt->eh_strategy_handler() 449e513de99SMauro Carvalho Chehab------------------------------------------------ 450e513de99SMauro Carvalho Chehab 451e513de99SMauro Carvalho Chehabtransportt->eh_strategy_handler() is invoked in the place of 452e513de99SMauro Carvalho Chehabscsi_unjam_host() and it is responsible for whole recovery process. 453e513de99SMauro Carvalho ChehabOn completion, the handler should have made lower layers forget about 454e513de99SMauro Carvalho Chehaball failed scmds and either ready for new commands or offline. Also, 455e513de99SMauro Carvalho Chehabit should perform SCSI EH maintenance chores to maintain integrity of 456e513de99SMauro Carvalho ChehabSCSI midlayer. IOW, of the steps described in [2-1-2], all steps 457e513de99SMauro Carvalho Chehabexcept for #1 must be implemented by eh_strategy_handler(). 458e513de99SMauro Carvalho Chehab 459e513de99SMauro Carvalho Chehab 460e513de99SMauro Carvalho Chehab2.2.1 Pre transportt->eh_strategy_handler() SCSI midlayer conditions 461e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 462e513de99SMauro Carvalho Chehab 463e513de99SMauro Carvalho Chehab The following conditions are true on entry to the handler. 464e513de99SMauro Carvalho Chehab 465e513de99SMauro Carvalho Chehab - Each failed scmd's eh_flags field is set appropriately. 466e513de99SMauro Carvalho Chehab 467e513de99SMauro Carvalho Chehab - Each failed scmd is linked on scmd->eh_cmd_q by scmd->eh_entry. 468e513de99SMauro Carvalho Chehab 469e513de99SMauro Carvalho Chehab - SHOST_RECOVERY is set. 470e513de99SMauro Carvalho Chehab 471e513de99SMauro Carvalho Chehab - shost->host_failed == shost->host_busy 472e513de99SMauro Carvalho Chehab 473e513de99SMauro Carvalho Chehab 474e513de99SMauro Carvalho Chehab2.2.2 Post transportt->eh_strategy_handler() SCSI midlayer conditions 475e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 476e513de99SMauro Carvalho Chehab 477e513de99SMauro Carvalho Chehab The following conditions must be true on exit from the handler. 478e513de99SMauro Carvalho Chehab 479e513de99SMauro Carvalho Chehab - shost->host_failed is zero. 480e513de99SMauro Carvalho Chehab 481e513de99SMauro Carvalho Chehab - Each scmd is in such a state that scsi_setup_cmd_retry() on the 482e513de99SMauro Carvalho Chehab scmd doesn't make any difference. 483e513de99SMauro Carvalho Chehab 484e513de99SMauro Carvalho Chehab - shost->eh_cmd_q is cleared. 485e513de99SMauro Carvalho Chehab 486e513de99SMauro Carvalho Chehab - Each scmd->eh_entry is cleared. 487e513de99SMauro Carvalho Chehab 488e513de99SMauro Carvalho Chehab - Either scsi_queue_insert() or scsi_finish_command() is called on 489e513de99SMauro Carvalho Chehab each scmd. Note that the handler is free to use scmd->retries and 490e513de99SMauro Carvalho Chehab ->allowed to limit the number of retries. 491e513de99SMauro Carvalho Chehab 492e513de99SMauro Carvalho Chehab 493e513de99SMauro Carvalho Chehab2.2.3 Things to consider 494e513de99SMauro Carvalho Chehab^^^^^^^^^^^^^^^^^^^^^^^^ 495e513de99SMauro Carvalho Chehab 496e513de99SMauro Carvalho Chehab - Know that timed out scmds are still active on lower layers. Make 497e513de99SMauro Carvalho Chehab lower layers forget about them before doing anything else with 498e513de99SMauro Carvalho Chehab those scmds. 499e513de99SMauro Carvalho Chehab 500e513de99SMauro Carvalho Chehab - For consistency, when accessing/modifying shost data structure, 501e513de99SMauro Carvalho Chehab grab shost->host_lock. 502e513de99SMauro Carvalho Chehab 503e513de99SMauro Carvalho Chehab - On completion, each failed sdev must have forgotten about all 504e513de99SMauro Carvalho Chehab active scmds. 505e513de99SMauro Carvalho Chehab 506e513de99SMauro Carvalho Chehab - On completion, each failed sdev must be ready for new commands or 507e513de99SMauro Carvalho Chehab offline. 508e513de99SMauro Carvalho Chehab 509e513de99SMauro Carvalho Chehab 510e513de99SMauro Carvalho ChehabTejun Heo 511e513de99SMauro Carvalho Chehabhtejun@gmail.com 512e513de99SMauro Carvalho Chehab 513e513de99SMauro Carvalho Chehab11th September 2005 514