| d3b75029 | 21-May-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/mem: Convert partition-info to resources
To date the per-device-partition DPA range information has only been
used for enumeration purposes. In preparation for allocating regions
from available
cxl/mem: Convert partition-info to resources
To date the per-device-partition DPA range information has only been
used for enumeration purposes. In preparation for allocating regions
from available DPA capacity, convert those ranges into DPA-type resource
trees.
With resources and the new add_dpa_res() helper some open coded end
address calculations and debug prints can be cleaned.
The 'cxlds->pmem_res' and 'cxlds->ram_res' resources are child resources
of the total-device DPA space and they in turn will host DPA allocations
from cxl_endpoint_decoder instances (tracked by cxled->dpa_res).
Cc: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/165603878921.551046.8127845916514734142.stgit@dwillia2-xfh
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| e8b7ea58 | 18-May-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/core: Rename ->decoder_range ->hpa_range
In preparation for growing a ->dpa_range attribute for endpoint
decoders, rename the current ->decoder_range to the more descriptive
->hpa_range.
Review
cxl/core: Rename ->decoder_range ->hpa_range
In preparation for growing a ->dpa_range attribute for endpoint
decoders, rename the current ->decoder_range to the more descriptive
->hpa_range.
Reviewed-by: Alison Schofield <alison.schofield@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Adam Manzanares <a.manzanares@samsung.com>
Link: https://lore.kernel.org/r/165603872867.551046.2170426227407458814.stgit@dwillia2-xfh
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| a12562bb | 18-May-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/mem: Merge cxl_dvsec_ranges() and cxl_hdm_decode_init()
In preparation for changing how the driver handles 'mem_enable' in the CXL
DVSEC control register. Merge the contents of cxl_hdm_decode_in
cxl/mem: Merge cxl_dvsec_ranges() and cxl_hdm_decode_init()
In preparation for changing how the driver handles 'mem_enable' in the CXL
DVSEC control register. Merge the contents of cxl_hdm_decode_init() into
cxl_dvsec_ranges() and rename the combined function cxl_hdm_decode_init().
The possible cleanups and fixes that result from this merge are saved for a
follow-on change.
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Link: https://lore.kernel.org/r/165291690027.1426646.10249756632415633752.stgit@dwillia2-xfh
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 14d78874 | 18-May-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/mem: Consolidate CXL DVSEC Range enumeration in the core
In preparation for fixing the setting of the 'mem_enabled' bit in CXL
DVSEC Control register, move all CXL DVSEC range enumeration into t
cxl/mem: Consolidate CXL DVSEC Range enumeration in the core
In preparation for fixing the setting of the 'mem_enabled' bit in CXL
DVSEC Control register, move all CXL DVSEC range enumeration into the
same source file.
Reviewed-by: Ira Weiny <ira.weiny@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/165291688886.1426646.15046138604010482084.stgit@dwillia2-xfh
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 64cda3ae | 23-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
tools/testing/cxl: Add a physical_node link
Emulate what ACPI does to link a host bridge platform firmware device to
device node on the PCI bus. In this case it's just self referencing
link, but it
tools/testing/cxl: Add a physical_node link
Emulate what ACPI does to link a host bridge platform firmware device to
device node on the PCI bus. In this case it's just self referencing
link, but it otherwise lets the tooling test out its lookup code.
Link: https://lore.kernel.org/r/164298433209.3018233.18101085948127163720.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 7c7d68db | 23-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
tools/testing/cxl: Enumerate mock decoders
Enumerate 2-decoders per switch port and endpoint in the cxl_test
topology.
Link: https://lore.kernel.org/r/164298432699.3018233.12131068635065601541.stgi
tools/testing/cxl: Enumerate mock decoders
Enumerate 2-decoders per switch port and endpoint in the cxl_test
topology.
Link: https://lore.kernel.org/r/164298432699.3018233.12131068635065601541.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| c1915142 | 23-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
tools/testing/cxl: Mock one level of switches
The CXL port enumeration process adds intermediate CXL ports that are
discovered between "root" CXL ports enumerated by 'cxl_acpi' and
endpoints enumera
tools/testing/cxl: Mock one level of switches
The CXL port enumeration process adds intermediate CXL ports that are
discovered between "root" CXL ports enumerated by 'cxl_acpi' and
endpoints enumerated by 'cxl_pci + cxl_mem'. Test the dynamic discovery
of intermediate switch ports in a CXL topology.
Link: https://lore.kernel.org/r/164298432189.3018233.13142151550113000967.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| a4a0ce24 | 23-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
tools/testing/cxl: Fix root port to host bridge assignment
Mocked root-ports are meant to be round-robin assigned to host-bridges.
Fixes: 67dcdd4d3b83 ("tools/testing/cxl: Introduce a mocked-up CXL
tools/testing/cxl: Fix root port to host bridge assignment
Mocked root-ports are meant to be round-robin assigned to host-bridges.
Fixes: 67dcdd4d3b83 ("tools/testing/cxl: Introduce a mocked-up CXL port hierarchy")
Link: https://lore.kernel.org/r/164298431629.3018233.14004377108116384485.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| f246abd6 | 23-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
tools/testing/cxl: Mock dvsec_ranges()
For test purposes, pretend that that CXL DVSEC ranges are not in active
use and the device is ready CXL.mem operation.
Link: https://lore.kernel.org/r/1642984
tools/testing/cxl: Mock dvsec_ranges()
For test purposes, pretend that that CXL DVSEC ranges are not in active
use and the device is ready CXL.mem operation.
Link: https://lore.kernel.org/r/164298431119.3018233.17175518196764977542.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 8dd2bc0f | 04-Feb-2022 |
Ben Widawsky <ben.widawsky@intel.com> |
cxl/mem: Add the cxl_mem driver
At this point the subsystem can enumerate all CXL ports (CXL.mem decode
resources in upstream switch ports and host bridges) in a system. The
last mile is connecting
cxl/mem: Add the cxl_mem driver
At this point the subsystem can enumerate all CXL ports (CXL.mem decode
resources in upstream switch ports and host bridges) in a system. The
last mile is connecting those ports to endpoints.
The cxl_mem driver connects an endpoint device to the platform CXL.mem
protoctol decode-topology. At ->probe() time it walks its
device-topology-ancestry and adds a CXL Port object at every Upstream
Port hop until it gets to CXL root. The CXL root object is only present
after a platform firmware driver registers platform CXL resources. For
ACPI based platform this is managed by the ACPI0017 device and the
cxl_acpi driver.
The ports are registered such that disabling a given port automatically
unregisters all descendant ports, and the chain can only be registered
after the root is established.
Given ACPI device scanning may run asynchronously compared to PCI device
scanning the root driver is tasked with rescanning the bus after the
root successfully probes.
Conversely if any ports in a chain between the root and an endpoint
becomes disconnected it subsequently triggers the endpoint to
unregister. Given lock depenedencies the endpoint unregistration happens
in a workqueue asynchronously. If userspace cares about synchronizing
delayed work after port events the /sys/bus/cxl/flush attribute is
available for that purpose.
Reported-by: Randy Dunlap <rdunlap@infradead.org>
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
[djbw: clarify changelog, rework hotplug support]
Link: https://lore.kernel.org/r/164398782997.903003.9725273241627693186.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| cf1f6877 | 23-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/memdev: Add numa_node attribute
While CXL memory targets will have their own memory target node,
individual memory devices may be affinitized like other PCI devices.
Emit that attribute for memd
cxl/memdev: Add numa_node attribute
While CXL memory targets will have their own memory target node,
individual memory devices may be affinitized like other PCI devices.
Emit that attribute for memdevs.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/164298428430.3018233.16409089892707993289.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| bcc79ea3 | 31-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/pci: Emit device serial number
Per the CXL specification (8.1.12.2 Memory Device PCIe Capabilities and
Extended Capabilities) the Device Serial Number capability is mandatory.
Emit it for user t
cxl/pci: Emit device serial number
Per the CXL specification (8.1.12.2 Memory Device PCIe Capabilities and
Extended Capabilities) the Device Serial Number capability is mandatory.
Emit it for user tooling to identify devices.
It is reasonable to ask whether the attribute should be added to the
list of PCI sysfs device attributes. The PCI layer can optionally emit
it too, but the CXL subsystem is aiming to preserve its independence and
the possibility of CXL topologies with non-PCI devices in it. To date
that has only proven useful for the 'cxl_test' model, but as can be seen
with seen with ACPI0016 devices, sometimes all that is needed is a
platform firmware table to point to CXL Component Registers in MMIO
space to define a "CXL" device.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/164366608838.196598.16856227191534267098.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 523e594d | 23-Jan-2022 |
Ben Widawsky <ben.widawsky@intel.com> |
cxl/pci: Implement wait for media active
CXL 2.0 8.1.3.8.2 states:
Memory_Active: When set, indicates that the CXL Range 1 memory is
fully initialized and available for software use. Must be se
cxl/pci: Implement wait for media active
CXL 2.0 8.1.3.8.2 states:
Memory_Active: When set, indicates that the CXL Range 1 memory is
fully initialized and available for software use. Must be set within
Range 1. Memory_Active_Timeout of deassertion of reset to CXL device
if CXL.mem HwInit Mode=1
Unfortunately, Memory_Active can take quite a long time depending on
media size (up to 256s per 2.0 spec). Provide a callback for the
eventual establishment of CXL.mem operations via the 'cxl_mem' driver
the 'struct cxl_memdev'. The implementation waits for 60s by default for
now and can be overridden by the mbox_ready_time module parameter.
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
[djbw: switch to sleeping wait]
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/164298427373.3018233.9309741847039301834.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 664bf115 | 01-Feb-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/core/port: Remove @host argument for dport + decoder enumeration
Now that dport and decoder enumeration is centralized in the port
driver, the @host argument for these helpers can be made implic
cxl/core/port: Remove @host argument for dport + decoder enumeration
Now that dport and decoder enumeration is centralized in the port
driver, the @host argument for these helpers can be made implicit. For
the root port the host is the port's uport device (ACPI0017 for
cxl_acpi), and for all other descendant ports the devm context is the
parent of @port.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Reviewed-by: Ben Widawsky <ben.widawsky@intel.com>
Link: https://lore.kernel.org/r/164375043390.484143.17617734732003230076.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 54cdbf84 | 01-Feb-2022 |
Ben Widawsky <ben.widawsky@intel.com> |
cxl/port: Add a driver for 'struct cxl_port' objects
The need for a CXL port driver and a dedicated cxl_bus_type is driven by
a need to simultaneously support 2 independent physical memory decode
do
cxl/port: Add a driver for 'struct cxl_port' objects
The need for a CXL port driver and a dedicated cxl_bus_type is driven by
a need to simultaneously support 2 independent physical memory decode
domains (cache coherent CXL.mem and uncached PCI.mmio) that also
intersect at a single PCIe device node. A CXL Port is a device that
advertises a CXL Component Register block with an "HDM Decoder
Capability Structure".
>From Documentation/driver-api/cxl/memory-devices.rst:
Similar to how a RAID driver takes disk objects and assembles them into
a new logical device, the CXL subsystem is tasked to take PCIe and ACPI
objects and assemble them into a CXL.mem decode topology. The need for
runtime configuration of the CXL.mem topology is also similar to RAID in
that different environments with the same hardware configuration may
decide to assemble the topology in contrasting ways. One may choose
performance (RAID0) striping memory across multiple Host Bridges and
endpoints while another may opt for fault tolerance and disable any
striping in the CXL.mem topology.
The port driver identifies whether an endpoint Memory Expander is
connected to a CXL topology. If an active (bound to the 'cxl_port'
driver) CXL Port is not found at every PCIe Switch Upstream port and an
active "root" CXL Port then the device is just a plain PCIe endpoint
only capable of participating in PCI.mmio and DMA cycles, not CXL.mem
coherent interleave sets.
The 'cxl_port' driver lets the CXL subsystem leverage driver-core
infrastructure for setup and teardown of register resources and
communicating device activation status to userspace. The cxl_bus_type
can rendezvous the async arrival of platform level CXL resources (via
the 'cxl_acpi' driver) with the asynchronous enumeration of Memory
Expander endpoints, while also implementing a hierarchical locking model
independent of the associated 'struct pci_dev' locking model. The
locking for dport and decoder enumeration is now handled in the core
rather than callers.
For now the port driver only enumerates and registers CXL resources
(downstream port metadata and decoder resources) later it will be used
to take action on its decoders in response to CXL.mem region
provisioning requests.
Note1: cxlpci.h has long depended on pci.h, but port.c was the first to
not include pci.h. Carry that dependency in cxlpci.h.
Note2: cxl port enumeration and probing complicates CXL subsystem init
to the point that it helps to have centralized debug logging of probe
events in cxl_bus_probe().
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Co-developed-by: Dan Williams <dan.j.williams@intel.com>
Link: https://lore.kernel.org/r/164374948116.464348.1772618057599155408.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| d17d0540 | 01-Feb-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/core/hdm: Add CXL standard decoder enumeration to the core
Unlike the decoder enumeration for "root decoders" described by platform
firmware, standard decoders can be enumerated from the compone
cxl/core/hdm: Add CXL standard decoder enumeration to the core
Unlike the decoder enumeration for "root decoders" described by platform
firmware, standard decoders can be enumerated from the component
registers space once the base address has been identified (via PCI,
ACPI, or another mechanism).
Add common infrastructure for HDM (Host-managed-Device-Memory) Decoder
enumeration and share it between host-bridge, upstream switch port, and
cxl_test defined decoders.
The locking model for switch level decoders is to hold the port lock
over the enumeration. This facilitates moving the dport and decoder
enumeration to a 'port' driver. For now, the only enumerator of decoder
resources is the cxl_acpi root driver.
Co-developed-by: Ben Widawsky <ben.widawsky@intel.com>
Signed-off-by: Ben Widawsky <ben.widawsky@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/164374688404.395335.9239248252443123526.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| 98d2d3a2 | 31-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/core: Generalize dport enumeration in the core
The core houses infrastructure for decoder resources. A CXL port's
dports are more closely related to decoder infrastructure than topology
enumerat
cxl/core: Generalize dport enumeration in the core
The core houses infrastructure for decoder resources. A CXL port's
dports are more closely related to decoder infrastructure than topology
enumeration. Implement generic PCI based dport enumeration in the core,
i.e. arrange for existing root port enumeration from cxl_acpi to share
code with switch port enumeration which just amounts to a small
difference in a pci_walk_bus() invocation once the appropriate 'struct
pci_bus' has been retrieved.
Set the convention that decoder objects are registered after all dports
are enumerated. This enables userspace to know when the CXL core is
finished establishing 'dportX' links underneath the 'portX' object.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/164368114191.354031.5270501846455462665.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| c978f1b1 | 31-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/port: Up-level cxl_add_dport() locking requirements to the caller
In preparation for moving dport enumeration into the core, require the
port device lock to be acquired by the caller.
Reviewed-
cxl/port: Up-level cxl_add_dport() locking requirements to the caller
In preparation for moving dport enumeration into the core, require the
port device lock to be acquired by the caller.
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/164367759016.324231.105551648350470000.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|
| a46cfc0f | 31-Jan-2022 |
Dan Williams <dan.j.williams@intel.com> |
cxl/pmem: Introduce a find_cxl_root() helper
In preparation for switch port enumeration while also preserving the
potential for multi-domain / multi-root CXL topologies. Introduce a
'struct device'
cxl/pmem: Introduce a find_cxl_root() helper
In preparation for switch port enumeration while also preserving the
potential for multi-domain / multi-root CXL topologies. Introduce a
'struct device' generic mechanism for retrieving a root CXL port, if one
is registered. Note that the only known multi-domain CXL configurations
are running the cxl_test unit test on a system that also publishes an
ACPI0017 device.
With this in hand the nvdimm-bridge lookup can be with
device_find_child() instead of bus_find_device() + custom mocked lookup
infrastructure in cxl_test.
The mechanism looks for a 2nd level port since the root level topology
is platform-firmware specific and the 2nd level down follows standard
PCIe topology expectations. The cxl_acpi 2nd level is associated with a
PCIe Root Port.
Reported-by: Ben Widawsky <ben.widawsky@intel.com>
Reviewed-by: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Link: https://lore.kernel.org/r/164367562182.225521.9488555616768096049.stgit@dwillia2-desk3.amr.corp.intel.com
Signed-off-by: Dan Williams <dan.j.williams@intel.com>
show more ...
|