xref: /openbmc/linux/Documentation/arch/s390/cds.rst (revision 37002bc6)
1===========================
2Linux for S/390 and zSeries
3===========================
4
5Common Device Support (CDS)
6Device Driver I/O Support Routines
7
8Authors:
9	- Ingo Adlung
10	- Cornelia Huck
11
12Copyright, IBM Corp. 1999-2002
13
14Introduction
15============
16
17This document describes the common device support routines for Linux/390.
18Different than other hardware architectures, ESA/390 has defined a unified
19I/O access method. This gives relief to the device drivers as they don't
20have to deal with different bus types, polling versus interrupt
21processing, shared versus non-shared interrupt processing, DMA versus port
22I/O (PIO), and other hardware features more. However, this implies that
23either every single device driver needs to implement the hardware I/O
24attachment functionality itself, or the operating system provides for a
25unified method to access the hardware, providing all the functionality that
26every single device driver would have to provide itself.
27
28The document does not intend to explain the ESA/390 hardware architecture in
29every detail.This information can be obtained from the ESA/390 Principles of
30Operation manual (IBM Form. No. SA22-7201).
31
32In order to build common device support for ESA/390 I/O interfaces, a
33functional layer was introduced that provides generic I/O access methods to
34the hardware.
35
36The common device support layer comprises the I/O support routines defined
37below. Some of them implement common Linux device driver interfaces, while
38some of them are ESA/390 platform specific.
39
40Note:
41  In order to write a driver for S/390, you also need to look into the interface
42  described in Documentation/arch/s390/driver-model.rst.
43
44Note for porting drivers from 2.4:
45
46The major changes are:
47
48* The functions use a ccw_device instead of an irq (subchannel).
49* All drivers must define a ccw_driver (see driver-model.txt) and the associated
50  functions.
51* request_irq() and free_irq() are no longer done by the driver.
52* The oper_handler is (kindof) replaced by the probe() and set_online() functions
53  of the ccw_driver.
54* The not_oper_handler is (kindof) replaced by the remove() and set_offline()
55  functions of the ccw_driver.
56* The channel device layer is gone.
57* The interrupt handlers must be adapted to use a ccw_device as argument.
58  Moreover, they don't return a devstat, but an irb.
59* Before initiating an io, the options must be set via ccw_device_set_options().
60* Instead of calling read_dev_chars()/read_conf_data(), the driver issues
61  the channel program and handles the interrupt itself.
62
63ccw_device_get_ciw()
64   get commands from extended sense data.
65
66ccw_device_start(), ccw_device_start_timeout(), ccw_device_start_key(), ccw_device_start_key_timeout()
67   initiate an I/O request.
68
69ccw_device_resume()
70   resume channel program execution.
71
72ccw_device_halt()
73   terminate the current I/O request processed on the device.
74
75do_IRQ()
76   generic interrupt routine. This function is called by the interrupt entry
77   routine whenever an I/O interrupt is presented to the system. The do_IRQ()
78   routine determines the interrupt status and calls the device specific
79   interrupt handler according to the rules (flags) defined during I/O request
80   initiation with do_IO().
81
82The next chapters describe the functions other than do_IRQ() in more details.
83The do_IRQ() interface is not described, as it is called from the Linux/390
84first level interrupt handler only and does not comprise a device driver
85callable interface. Instead, the functional description of do_IO() also
86describes the input to the device specific interrupt handler.
87
88Note:
89	All explanations apply also to the 64 bit architecture s390x.
90
91
92Common Device Support (CDS) for Linux/390 Device Drivers
93========================================================
94
95General Information
96-------------------
97
98The following chapters describe the I/O related interface routines the
99Linux/390 common device support (CDS) provides to allow for device specific
100driver implementations on the IBM ESA/390 hardware platform. Those interfaces
101intend to provide the functionality required by every device driver
102implementation to allow to drive a specific hardware device on the ESA/390
103platform. Some of the interface routines are specific to Linux/390 and some
104of them can be found on other Linux platforms implementations too.
105Miscellaneous function prototypes, data declarations, and macro definitions
106can be found in the architecture specific C header file
107linux/arch/s390/include/asm/irq.h.
108
109Overview of CDS interface concepts
110----------------------------------
111
112Different to other hardware platforms, the ESA/390 architecture doesn't define
113interrupt lines managed by a specific interrupt controller and bus systems
114that may or may not allow for shared interrupts, DMA processing, etc.. Instead,
115the ESA/390 architecture has implemented a so called channel subsystem, that
116provides a unified view of the devices physically attached to the systems.
117Though the ESA/390 hardware platform knows about a huge variety of different
118peripheral attachments like disk devices (aka. DASDs), tapes, communication
119controllers, etc. they can all be accessed by a well defined access method and
120they are presenting I/O completion a unified way : I/O interruptions. Every
121single device is uniquely identified to the system by a so called subchannel,
122where the ESA/390 architecture allows for 64k devices be attached.
123
124Linux, however, was first built on the Intel PC architecture, with its two
125cascaded 8259 programmable interrupt controllers (PICs), that allow for a
126maximum of 15 different interrupt lines. All devices attached to such a system
127share those 15 interrupt levels. Devices attached to the ISA bus system must
128not share interrupt levels (aka. IRQs), as the ISA bus bases on edge triggered
129interrupts. MCA, EISA, PCI and other bus systems base on level triggered
130interrupts, and therewith allow for shared IRQs. However, if multiple devices
131present their hardware status by the same (shared) IRQ, the operating system
132has to call every single device driver registered on this IRQ in order to
133determine the device driver owning the device that raised the interrupt.
134
135Up to kernel 2.4, Linux/390 used to provide interfaces via the IRQ (subchannel).
136For internal use of the common I/O layer, these are still there. However,
137device drivers should use the new calling interface via the ccw_device only.
138
139During its startup the Linux/390 system checks for peripheral devices. Each
140of those devices is uniquely defined by a so called subchannel by the ESA/390
141channel subsystem. While the subchannel numbers are system generated, each
142subchannel also takes a user defined attribute, the so called device number.
143Both subchannel number and device number cannot exceed 65535. During sysfs
144initialisation, the information about control unit type and device types that
145imply specific I/O commands (channel command words - CCWs) in order to operate
146the device are gathered. Device drivers can retrieve this set of hardware
147information during their initialization step to recognize the devices they
148support using the information saved in the struct ccw_device given to them.
149This methods implies that Linux/390 doesn't require to probe for free (not
150armed) interrupt request lines (IRQs) to drive its devices with. Where
151applicable, the device drivers can use issue the READ DEVICE CHARACTERISTICS
152ccw to retrieve device characteristics in its online routine.
153
154In order to allow for easy I/O initiation the CDS layer provides a
155ccw_device_start() interface that takes a device specific channel program (one
156or more CCWs) as input sets up the required architecture specific control blocks
157and initiates an I/O request on behalf of the device driver. The
158ccw_device_start() routine allows to specify whether it expects the CDS layer
159to notify the device driver for every interrupt it observes, or with final status
160only. See ccw_device_start() for more details. A device driver must never issue
161ESA/390 I/O commands itself, but must use the Linux/390 CDS interfaces instead.
162
163For long running I/O request to be canceled, the CDS layer provides the
164ccw_device_halt() function. Some devices require to initially issue a HALT
165SUBCHANNEL (HSCH) command without having pending I/O requests. This function is
166also covered by ccw_device_halt().
167
168
169get_ciw() - get command information word
170
171This call enables a device driver to get information about supported commands
172from the extended SenseID data.
173
174::
175
176  struct ciw *
177  ccw_device_get_ciw(struct ccw_device *cdev, __u32 cmd);
178
179====  ========================================================
180cdev  The ccw_device for which the command is to be retrieved.
181cmd   The command type to be retrieved.
182====  ========================================================
183
184ccw_device_get_ciw() returns:
185
186=====  ================================================================
187 NULL  No extended data available, invalid device or command not found.
188!NULL  The command requested.
189=====  ================================================================
190
191::
192
193  ccw_device_start() - Initiate I/O Request
194
195The ccw_device_start() routines is the I/O request front-end processor. All
196device driver I/O requests must be issued using this routine. A device driver
197must not issue ESA/390 I/O commands itself. Instead the ccw_device_start()
198routine provides all interfaces required to drive arbitrary devices.
199
200This description also covers the status information passed to the device
201driver's interrupt handler as this is related to the rules (flags) defined
202with the associated I/O request when calling ccw_device_start().
203
204::
205
206  int ccw_device_start(struct ccw_device *cdev,
207		       struct ccw1 *cpa,
208		       unsigned long intparm,
209		       __u8 lpm,
210		       unsigned long flags);
211  int ccw_device_start_timeout(struct ccw_device *cdev,
212			       struct ccw1 *cpa,
213			       unsigned long intparm,
214			       __u8 lpm,
215			       unsigned long flags,
216			       int expires);
217  int ccw_device_start_key(struct ccw_device *cdev,
218			   struct ccw1 *cpa,
219			   unsigned long intparm,
220			   __u8 lpm,
221			   __u8 key,
222			   unsigned long flags);
223  int ccw_device_start_key_timeout(struct ccw_device *cdev,
224				   struct ccw1 *cpa,
225				   unsigned long intparm,
226				   __u8 lpm,
227				   __u8 key,
228				   unsigned long flags,
229				   int expires);
230
231============= =============================================================
232cdev          ccw_device the I/O is destined for
233cpa           logical start address of channel program
234user_intparm  user specific interrupt information; will be presented
235	      back to the device driver's interrupt handler. Allows a
236	      device driver to associate the interrupt with a
237	      particular I/O request.
238lpm           defines the channel path to be used for a specific I/O
239	      request. A value of 0 will make cio use the opm.
240key           the storage key to use for the I/O (useful for operating on a
241	      storage with a storage key != default key)
242flag          defines the action to be performed for I/O processing
243expires       timeout value in jiffies. The common I/O layer will terminate
244	      the running program after this and call the interrupt handler
245	      with ERR_PTR(-ETIMEDOUT) as irb.
246============= =============================================================
247
248Possible flag values are:
249
250========================= =============================================
251DOIO_ALLOW_SUSPEND        channel program may become suspended
252DOIO_DENY_PREFETCH        don't allow for CCW prefetch; usually
253			  this implies the channel program might
254			  become modified
255DOIO_SUPPRESS_INTER       don't call the handler on intermediate status
256========================= =============================================
257
258The cpa parameter points to the first format 1 CCW of a channel program::
259
260  struct ccw1 {
261	__u8  cmd_code;/* command code */
262	__u8  flags;   /* flags, like IDA addressing, etc. */
263	__u16 count;   /* byte count */
264	__u32 cda;     /* data address */
265  } __attribute__ ((packed,aligned(8)));
266
267with the following CCW flags values defined:
268
269=================== =========================
270CCW_FLAG_DC         data chaining
271CCW_FLAG_CC         command chaining
272CCW_FLAG_SLI        suppress incorrect length
273CCW_FLAG_SKIP       skip
274CCW_FLAG_PCI        PCI
275CCW_FLAG_IDA        indirect addressing
276CCW_FLAG_SUSPEND    suspend
277=================== =========================
278
279
280Via ccw_device_set_options(), the device driver may specify the following
281options for the device:
282
283========================= ======================================
284DOIO_EARLY_NOTIFICATION   allow for early interrupt notification
285DOIO_REPORT_ALL           report all interrupt conditions
286========================= ======================================
287
288
289The ccw_device_start() function returns:
290
291======== ======================================================================
292      0  successful completion or request successfully initiated
293 -EBUSY  The device is currently processing a previous I/O request, or there is
294	 a status pending at the device.
295-ENODEV  cdev is invalid, the device is not operational or the ccw_device is
296	 not online.
297======== ======================================================================
298
299When the I/O request completes, the CDS first level interrupt handler will
300accumulate the status in a struct irb and then call the device interrupt handler.
301The intparm field will contain the value the device driver has associated with a
302particular I/O request. If a pending device status was recognized,
303intparm will be set to 0 (zero). This may happen during I/O initiation or delayed
304by an alert status notification. In any case this status is not related to the
305current (last) I/O request. In case of a delayed status notification no special
306interrupt will be presented to indicate I/O completion as the I/O request was
307never started, even though ccw_device_start() returned with successful completion.
308
309The irb may contain an error value, and the device driver should check for this
310first:
311
312========== =================================================================
313-ETIMEDOUT the common I/O layer terminated the request after the specified
314	   timeout value
315-EIO       the common I/O layer terminated the request due to an error state
316========== =================================================================
317
318If the concurrent sense flag in the extended status word (esw) in the irb is
319set, the field erw.scnt in the esw describes the number of device specific
320sense bytes available in the extended control word irb->scsw.ecw[]. No device
321sensing by the device driver itself is required.
322
323The device interrupt handler can use the following definitions to investigate
324the primary unit check source coded in sense byte 0 :
325
326======================= ====
327SNS0_CMD_REJECT         0x80
328SNS0_INTERVENTION_REQ   0x40
329SNS0_BUS_OUT_CHECK      0x20
330SNS0_EQUIPMENT_CHECK    0x10
331SNS0_DATA_CHECK         0x08
332SNS0_OVERRUN            0x04
333SNS0_INCOMPL_DOMAIN     0x01
334======================= ====
335
336Depending on the device status, multiple of those values may be set together.
337Please refer to the device specific documentation for details.
338
339The irb->scsw.cstat field provides the (accumulated) subchannel status :
340
341========================= ============================
342SCHN_STAT_PCI             program controlled interrupt
343SCHN_STAT_INCORR_LEN      incorrect length
344SCHN_STAT_PROG_CHECK      program check
345SCHN_STAT_PROT_CHECK      protection check
346SCHN_STAT_CHN_DATA_CHK    channel data check
347SCHN_STAT_CHN_CTRL_CHK    channel control check
348SCHN_STAT_INTF_CTRL_CHK   interface control check
349SCHN_STAT_CHAIN_CHECK     chaining check
350========================= ============================
351
352The irb->scsw.dstat field provides the (accumulated) device status :
353
354===================== =================
355DEV_STAT_ATTENTION    attention
356DEV_STAT_STAT_MOD     status modifier
357DEV_STAT_CU_END       control unit end
358DEV_STAT_BUSY         busy
359DEV_STAT_CHN_END      channel end
360DEV_STAT_DEV_END      device end
361DEV_STAT_UNIT_CHECK   unit check
362DEV_STAT_UNIT_EXCEP   unit exception
363===================== =================
364
365Please see the ESA/390 Principles of Operation manual for details on the
366individual flag meanings.
367
368Usage Notes:
369
370ccw_device_start() must be called disabled and with the ccw device lock held.
371
372The device driver is allowed to issue the next ccw_device_start() call from
373within its interrupt handler already. It is not required to schedule a
374bottom-half, unless a non deterministically long running error recovery procedure
375or similar needs to be scheduled. During I/O processing the Linux/390 generic
376I/O device driver support has already obtained the IRQ lock, i.e. the handler
377must not try to obtain it again when calling ccw_device_start() or we end in a
378deadlock situation!
379
380If a device driver relies on an I/O request to be completed prior to start the
381next it can reduce I/O processing overhead by chaining a NoOp I/O command
382CCW_CMD_NOOP to the end of the submitted CCW chain. This will force Channel-End
383and Device-End status to be presented together, with a single interrupt.
384However, this should be used with care as it implies the channel will remain
385busy, not being able to process I/O requests for other devices on the same
386channel. Therefore e.g. read commands should never use this technique, as the
387result will be presented by a single interrupt anyway.
388
389In order to minimize I/O overhead, a device driver should use the
390DOIO_REPORT_ALL  only if the device can report intermediate interrupt
391information prior to device-end the device driver urgently relies on. In this
392case all I/O interruptions are presented to the device driver until final
393status is recognized.
394
395If a device is able to recover from asynchronously presented I/O errors, it can
396perform overlapping I/O using the DOIO_EARLY_NOTIFICATION flag. While some
397devices always report channel-end and device-end together, with a single
398interrupt, others present primary status (channel-end) when the channel is
399ready for the next I/O request and secondary status (device-end) when the data
400transmission has been completed at the device.
401
402Above flag allows to exploit this feature, e.g. for communication devices that
403can handle lost data on the network to allow for enhanced I/O processing.
404
405Unless the channel subsystem at any time presents a secondary status interrupt,
406exploiting this feature will cause only primary status interrupts to be
407presented to the device driver while overlapping I/O is performed. When a
408secondary status without error (alert status) is presented, this indicates
409successful completion for all overlapping ccw_device_start() requests that have
410been issued since the last secondary (final) status.
411
412Channel programs that intend to set the suspend flag on a channel command word
413(CCW)  must start the I/O operation with the DOIO_ALLOW_SUSPEND option or the
414suspend flag will cause a channel program check. At the time the channel program
415becomes suspended an intermediate interrupt will be generated by the channel
416subsystem.
417
418ccw_device_resume() - Resume Channel Program Execution
419
420If a device driver chooses to suspend the current channel program execution by
421setting the CCW suspend flag on a particular CCW, the channel program execution
422is suspended. In order to resume channel program execution the CIO layer
423provides the ccw_device_resume() routine.
424
425::
426
427  int ccw_device_resume(struct ccw_device *cdev);
428
429====  ================================================
430cdev  ccw_device the resume operation is requested for
431====  ================================================
432
433The ccw_device_resume() function returns:
434
435=========   ==============================================
436	0   suspended channel program is resumed
437   -EBUSY   status pending
438  -ENODEV   cdev invalid or not-operational subchannel
439  -EINVAL   resume function not applicable
440-ENOTCONN   there is no I/O request pending for completion
441=========   ==============================================
442
443Usage Notes:
444
445Please have a look at the ccw_device_start() usage notes for more details on
446suspended channel programs.
447
448ccw_device_halt() - Halt I/O Request Processing
449
450Sometimes a device driver might need a possibility to stop the processing of
451a long-running channel program or the device might require to initially issue
452a halt subchannel (HSCH) I/O command. For those purposes the ccw_device_halt()
453command is provided.
454
455ccw_device_halt() must be called disabled and with the ccw device lock held.
456
457::
458
459  int ccw_device_halt(struct ccw_device *cdev,
460		      unsigned long intparm);
461
462=======  =====================================================
463cdev     ccw_device the halt operation is requested for
464intparm  interruption parameter; value is only used if no I/O
465	 is outstanding, otherwise the intparm associated with
466	 the I/O request is returned
467=======  =====================================================
468
469The ccw_device_halt() function returns:
470
471=======  ==============================================================
472      0  request successfully initiated
473-EBUSY   the device is currently busy, or status pending.
474-ENODEV  cdev invalid.
475-EINVAL  The device is not operational or the ccw device is not online.
476=======  ==============================================================
477
478Usage Notes:
479
480A device driver may write a never-ending channel program by writing a channel
481program that at its end loops back to its beginning by means of a transfer in
482channel (TIC)   command (CCW_CMD_TIC). Usually this is performed by network
483device drivers by setting the PCI CCW flag (CCW_FLAG_PCI). Once this CCW is
484executed a program controlled interrupt (PCI) is generated. The device driver
485can then perform an appropriate action. Prior to interrupt of an outstanding
486read to a network device (with or without PCI flag) a ccw_device_halt()
487is required to end the pending operation.
488
489::
490
491  ccw_device_clear() - Terminage I/O Request Processing
492
493In order to terminate all I/O processing at the subchannel, the clear subchannel
494(CSCH) command is used. It can be issued via ccw_device_clear().
495
496ccw_device_clear() must be called disabled and with the ccw device lock held.
497
498::
499
500  int ccw_device_clear(struct ccw_device *cdev, unsigned long intparm);
501
502======= ===============================================
503cdev    ccw_device the clear operation is requested for
504intparm interruption parameter (see ccw_device_halt())
505======= ===============================================
506
507The ccw_device_clear() function returns:
508
509=======  ==============================================================
510      0  request successfully initiated
511-ENODEV  cdev invalid
512-EINVAL  The device is not operational or the ccw device is not online.
513=======  ==============================================================
514
515Miscellaneous Support Routines
516------------------------------
517
518This chapter describes various routines to be used in a Linux/390 device
519driver programming environment.
520
521get_ccwdev_lock()
522
523Get the address of the device specific lock. This is then used in
524spin_lock() / spin_unlock() calls.
525
526::
527
528  __u8 ccw_device_get_path_mask(struct ccw_device *cdev);
529
530Get the mask of the path currently available for cdev.
531