sfc: Make struct efx_special_buffer less special

On EF10, the firmware is in charge of allocating buffer table entries.
Change struct efx_special_buffer to use a struct efx_buffer member,
so that it

sfc: Make struct efx_special_buffer less special

On EF10, the firmware is in charge of allocating buffer table entries.
Change struct efx_special_buffer to use a struct efx_buffer member,
so that it can be used with efx_nic_{alloc,free}_buffer() in that
case.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Add GFP flags to efx_nic_alloc_buffer() and make most callers allow blocking

Most call sites for efx_nic_alloc_buffer() are part of the probe or
reconfiguration paths and can allocate with GFP_

sfc: Add GFP flags to efx_nic_alloc_buffer() and make most callers allow blocking

Most call sites for efx_nic_alloc_buffer() are part of the probe or
reconfiguration paths and can allocate with GFP_KERNEL. A few others
should use GFP_NOIO (I think). Only one is in atomic context and
must use the current GFP_ATOMIC.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Fix IRQ cleanup in case of a probe failure

The lifetime of an irq_cpu_rmap is odd: we have to allocate it before
installing IRQ handlers and free it before removing the IRQ handlers.
As a resul

sfc: Fix IRQ cleanup in case of a probe failure

The lifetime of an irq_cpu_rmap is odd: we have to allocate it before
installing IRQ handlers and free it before removing the IRQ handlers.
As a result of this asymmetry, it was omitted from some failure paths.

On another failure path, we could try to remove IRQ handlers we
had not yet installed.

Move the irq_cpu_rmap allocation and freeing alongside IRQ handler
installation and removal, in efx_nic_{init,fini}_interrupts().
Count the number of IRQ handlers successfully installed and only
remove those on the failure path.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Define and set RX buffer flag for packets parsed as TCP

This will be useful for shortcutting some software packet parsing.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

sfc: Fix EEH with legacy interrupts.

PCI legacy interrupts are level-triggered, and we cannot mask them up
on an isolated device. Instead, disable the IRQ at the controller
until we have recovered.

sfc: Fix EEH with legacy interrupts.

PCI legacy interrupts are level-triggered, and we cannot mask them up
on an isolated device. Instead, disable the IRQ at the controller
until we have recovered.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

drivers:net: dma_alloc_coherent: use __GFP_ZERO instead of memset(, 0)

Reduce the number of calls required to alloc
a zeroed block of memory.

Trivially reduces overall object size.

Other changes a

drivers:net: dma_alloc_coherent: use __GFP_ZERO instead of memset(, 0)

Reduce the number of calls required to alloc
a zeroed block of memory.

Trivially reduces overall object size.

Other changes around these removals
o Neaten call argument alignment
o Remove an unnecessary OOM message after dma_alloc_coherent failure
o Remove unnecessary gfp_t stack variable

Signed-off-by: Joe Perches <joe@perches.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

sfc: Only use TX push if a single descriptor is to be written

Using TX push when notifying the NIC of multiple new descriptors in
the ring will very occasionally cause the TX DMA engine to re-use an

sfc: Only use TX push if a single descriptor is to be written

Using TX push when notifying the NIC of multiple new descriptors in
the ring will very occasionally cause the TX DMA engine to re-use an
old descriptor. This can result in a duplicated or partly duplicated
packet (new headers with old data), or an IOMMU page fault. This does
not happen when the pushed descriptor is the only one written.

TX push also provides little latency benefit when a packet requires
more than one descriptor.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Enable RX DMA scattering where possible

Enable RX DMA scattering iff an RX buffer large enough for the current
MTU will not fit into a single page and the NIC supports DMA
scattering for kernel

sfc: Enable RX DMA scattering where possible

Enable RX DMA scattering iff an RX buffer large enough for the current
MTU will not fit into a single page and the NIC supports DMA
scattering for kernel-mode RX queues.

On Falcon and Siena, the RX_USR_BUF_SIZE field is used as the DMA
limit for both all RX queues with scatter enabled. Set it to 1824,
matching what Onload uses now.

Maintain a statistic for frames truncated due to lack of descriptors
(rx_nodesc_trunc). This is distinct from rx_frm_trunc which may be
incremented when scattering is disabled and implies an over-length
frame.

Whenever an MTU change causes scattering to be turned on or off,
update filters that point to the PF queues, but leave others
unchanged, as VF drivers assume scattering is off.

Add n_frags parameters to various functions, and make them iterate:
- efx_rx_packet()
- efx_recycle_rx_buffers()
- efx_rx_mk_skb()
- efx_rx_deliver()

Make efx_handle_rx_event() responsible for updating
efx_rx_queue::removed_count.

Change the RX pipeline state to a starting ring index and number of
fragments, and make __efx_rx_packet() responsible for clearing it.

Based on earlier versions by David Riddoch and Jon Cooper.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Work-around flush timeout when flushes have completed

We sometimes hit a "failed to flush" timeout on some TX queues, but the
flushes have completed and the flush completion events seem to go m

sfc: Work-around flush timeout when flushes have completed

We sometimes hit a "failed to flush" timeout on some TX queues, but the
flushes have completed and the flush completion events seem to go missing.
In this case, we can check the TX_DESC_PTR_TBL register and drain the
queues if the flushes had finished.

[bwh: Minor fixes to coding style]
Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Do not initialise buffer in efx_alloc_special_buffer()

Currently we initialise the newly allocated buffer to all-1s, which is
important for event queues but not for descriptor queues. And sinc

sfc: Do not initialise buffer in efx_alloc_special_buffer()

Currently we initialise the newly allocated buffer to all-1s, which is
important for event queues but not for descriptor queues. And since
we also do that in efx_nic_init_eventq(), it is completely pointless
to do it here.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Remove confusing MMIO functions

efx_writed_table() uses a step of 16 bytes but efx_readd_table() uses
a step of 4 bytes. Why are they different?

Firstly, register access is asymmetric:

- The

sfc: Remove confusing MMIO functions

efx_writed_table() uses a step of 16 bytes but efx_readd_table() uses
a step of 4 bytes. Why are they different?

Firstly, register access is asymmetric:

- The EVQ_RPTR table and RX_INDIRECTION_TBL can (or must?) be written
as dwords even though they have a step size of 16 bytes, unlike
most other CSRs.
- In general, a read of any width is valid for registers, so long as
it does not cross register boundaries. There is also no latching
behaviour in the BIU, contrary to rumour.

We write to the EVQ_RPTR table with efx_writed_table() but never read
it back as it's write-only. We write to the RX_INDIRECTION_TBL with
efx_writed_table(), but only read it back for the register dump, where
we use efx_reado_table() as for any other table with step size of 16.

We read MC_TREG_SMEM with efx_readd_table() for the register dump, but
normally read and write it with efx_readd() and efx_writed() using
offsets calculated in bytes.

Since these functions are trivial and have few callers, it's clearer
to open-code them at the call sites. While we're at it, update the
comments on the BIU behaviour again.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Really disable flow control while flushing

Receiving pause frames can block TX queue flushes. Earlier changes
work around this by reconfiguring the MAC during flushes for VFs, but
during flush

sfc: Really disable flow control while flushing

Receiving pause frames can block TX queue flushes. Earlier changes
work around this by reconfiguring the MAC during flushes for VFs, but
during flushes for the PF we would only change the fc_disable counter.
Unless the MAC is reconfigured for some other reason during the flush
(which I would not expect to happen) this had no effect at all.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

drivers/net/ethernet/sfc: use standard __{clear,set}_bit_le() functions

There are now standard functions for dealing with little-endian bit
arrays, so use them instead of our own implementations.

S

drivers/net/ethernet/sfc: use standard __{clear,set}_bit_le() functions

There are now standard functions for dealing with little-endian bit
arrays, so use them instead of our own implementations.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>
Signed-off-by: Takuya Yoshikawa <yoshikawa.takuya@oss.ntt.co.jp>
Cc: David Miller <davem@davemloft.net>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

show more ...

sfc: Simplify TSO header buffer allocation

TSO header buffers contain a control structure immediately followed by
the packet headers, and are kept on a free list when not in use. This
complicates b

sfc: Simplify TSO header buffer allocation

TSO header buffers contain a control structure immediately followed by
the packet headers, and are kept on a free list when not in use. This
complicates buffer management and tends to result in cache read misses
when we recycle such buffers (particularly if DMA-coherent memory
requires caches to be disabled).

Replace the free list with a simple mapping by descriptor index. We
know that there is always a payload descriptor between any two
descriptors with TSO header buffers, so we can allocate only one
such buffer for each two descriptors.

While we're at it, use a standard error code for allocation failure,
not -1.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Refactor struct efx_tx_buffer to use a flags field

Add a flags field to struct efx_tx_buffer, replacing the
continuation and map_single booleans.

Since a single descriptor cannot be both a TSO

sfc: Refactor struct efx_tx_buffer to use a flags field

Add a flags field to struct efx_tx_buffer, replacing the
continuation and map_single booleans.

Since a single descriptor cannot be both a TSO header and the last
descriptor for an skb, unionise efx_tx_buffer::{skb,tsoh} and add
flags for validity of these fields.

Clear all flags in free buffers (whereas previously the continuation
flag would be set).

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Disable VF queues during register self-test

Currently VF queues and drivers may remain active during this test.
This could cause memory corruption or spurious test failures.
Therefore we reset

sfc: Disable VF queues during register self-test

Currently VF queues and drivers may remain active during this test.
This could cause memory corruption or spurious test failures.
Therefore we reset the port/function before running these tests on
Siena.

On Falcon this doesn't work: we have to do some additional
initialisation before some blocks will work again. So refactor the
reset/register-test sequence into an efx_nic_type method so
efx_selftest() doesn't have to consider such quirks.

In the process, fix another minor bug: Siena does not have an
'invisible' reset and the self-test currently fails to push the PHY
configuration after resetting. Passing RESET_TYPE_ALL to
efx_reset_{down,up}() fixes this.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Use generic DMA API, not PCI-DMA API

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

sfc: Run event/IRQ self-test asynchronously when interface is brought up

Generate a test event on each event queue whenever the interface is
brought up, then after 1 second check that we have either

sfc: Run event/IRQ self-test asynchronously when interface is brought up

Generate a test event on each event queue whenever the interface is
brought up, then after 1 second check that we have either handled a
test event or handled another IRQ for each event queue.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Encapsulate access to efx_{channel,nic}::last_irq_cpu in self-test

Cleanup in preparation for doing an event test on ifup.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

sfc: Remove TX completions from adaptive IRQ scoring

RX and TX completions on the same event queue are generally not associated
with the same flows. The inclusion of TX completions in the adaptive

sfc: Remove TX completions from adaptive IRQ scoring

RX and TX completions on the same event queue are generally not associated
with the same flows. The inclusion of TX completions in the adaptive IRQ
score is more of a source of noise rather than useful feedback. Therefore,
do not include them in the score, and adjust the default threshold scores
down.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Add SR-IOV back-end support for SFC9000 family

On the SFC9000 family, each port has 1024 Virtual Interfaces (VIs),
each with an RX queue, a TX queue, an event queue and a mailbox
register. The

sfc: Add SR-IOV back-end support for SFC9000 family

On the SFC9000 family, each port has 1024 Virtual Interfaces (VIs),
each with an RX queue, a TX queue, an event queue and a mailbox
register. These may be assigned to up to 127 SR-IOV virtual functions
per port, with up to 64 VIs per VF.

We allocate an extra channel (IRQ and event queue only) to receive
requests from VF drivers.

There is a per-port limit of 4 concurrent RX queue flushes, and queue
flushes may be initiated by the MC in response to a Function Level
Reset (FLR) of a VF. Therefore, when SR-IOV is in use, we submit all
flush requests via the MC.

The RSS indirection table is shared with VFs, so the number of RX
queues used in the PF is limited to the number of VIs per VF.

This is almost entirely the work of Steve Hodgson, formerly
shodgson@solarflare.com.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Allocate SRAM between buffer table and descriptor caches at init time

Each port has a block of 64-bit SRAM that is divided between buffer
table and descriptor cache regions at initialisation ti

sfc: Allocate SRAM between buffer table and descriptor caches at init time

Each port has a block of 64-bit SRAM that is divided between buffer
table and descriptor cache regions at initialisation time. Currently
we use a fixed allocation, but it needs to be changed to support
larger numbers of queues.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Make buffer table indices and counts consistently unsigned

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

sfc: Disable flow control during flushes

The TX DMA engine issues upstream read requests when there is room in
the TX FIFO for the completion. However, the fetches for the rest of
the packet might b

sfc: Disable flow control during flushes

The TX DMA engine issues upstream read requests when there is room in
the TX FIFO for the completion. However, the fetches for the rest of
the packet might be delayed by any back pressure. Since a flush must
wait for an EOP, the entire flush may be delayed by back pressure.

Mitigate this by disabling flow control before the flushes are
started. Since PF and VF flushes run in parallel introduce
fc_disable, a reference count of the number of flushes outstanding.

The same principle could be applied to Falcon, but that
would bring with it its own testing.

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...

sfc: Generalise event generation to cover VF-owned event queues

For SR-IOV we will need to send events to event queues that belong to
VFs serviced by other drivers. Change the parameters of
efx_gen

sfc: Generalise event generation to cover VF-owned event queues

For SR-IOV we will need to send events to event queues that belong to
VFs serviced by other drivers. Change the parameters of
efx_generate_event() to allow this and declare it extern.

While we're at it, remove the existing declaration under the wrong
name efx_nic_generate_event().

Signed-off-by: Ben Hutchings <bhutchings@solarflare.com>

show more ...