media: rkisp1: Fix IRQ handling due to shared interrupts

[ Upstream commit ffb635bb398fc07cb38f8a7b4a82cbe5f412f08e ]

The driver requests the interrupts as IRQF_SHARED, so the interrupt
handlers ca

media: rkisp1: Fix IRQ handling due to shared interrupts

[ Upstream commit ffb635bb398fc07cb38f8a7b4a82cbe5f412f08e ]

The driver requests the interrupts as IRQF_SHARED, so the interrupt
handlers can be called at any time. If such a call happens while the ISP
is powered down, the SoC will hang as the driver tries to access the
ISP registers.

This can be reproduced even without the platform sharing the IRQ line:
Enable CONFIG_DEBUG_SHIRQ and unload the driver, and the board will
hang.

Fix this by adding a new field, 'irqs_enabled', which is used to bail
out from the interrupt handler when the ISP is not operational.

Link: https://lore.kernel.org/r/20231218-rkisp-shirq-fix-v1-2-173007628248@ideasonboard.com

Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

media: rkisp1: Store IRQ lines

[ Upstream commit 0753874617de883c6d4da903142f334f76a75d70 ]

Store the IRQ lines used by the driver for easy access. These are needed
in future patches which fix IRQ

media: rkisp1: Store IRQ lines

[ Upstream commit 0753874617de883c6d4da903142f334f76a75d70 ]

Store the IRQ lines used by the driver for easy access. These are needed
in future patches which fix IRQ race issues.

Link: https://lore.kernel.org/r/20231207-rkisp-irq-fix-v3-3-358a2c871a3c@ideasonboard.com

Tested-by: Adam Ford <aford173@gmail.com> #imx8mp-beacon
Signed-off-by: Tomi Valkeinen <tomi.valkeinen@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>
Signed-off-by: Sasha Levin <sashal@kernel.org>

show more ...

media: v4l: async: Rename v4l2_async_subdev as v4l2_async_connection

Rename v4l2_async_subdev as v4l2_async_connection, in order to
differentiate between the sub-devices and their connections: one
s

media: v4l: async: Rename v4l2_async_subdev as v4l2_async_connection

Rename v4l2_async_subdev as v4l2_async_connection, in order to
differentiate between the sub-devices and their connections: one
sub-device can have many connections but the V4L2 async framework has so
far allowed just a single one. Connections in this context will later
translate into either MC ancillary or data links.

This patch prepares changing that relation by changing existing users of
v4l2_async_subdev to switch to v4l2_async_connection. Async sub-devices
themselves will not be needed anymore

Additionally, __v4l2_async_nf_add_subdev() has been renamed
__v4l2_async_nf_add_connection().

Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com>
Tested-by: Philipp Zabel <p.zabel@pengutronix.de> # imx6qp
Tested-by: Niklas Söderlund <niklas.soderlund@ragnatech.se> # rcar + adv746x
Tested-by: Aishwarya Kothari <aishwarya.kothari@toradex.com> # Apalis i.MX6Q with TC358743
Tested-by: Lad Prabhakar <prabhakar.mahadev-lad.rj@bp.renesas.com> # Renesas RZ/G2L SMARC
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Configure LSC after enabling the ISP

The ISP8000Nano v18.02 (found in the i.MX8MP) requires the ISP to be
enabled (as indicated by the ISP_CTRL.ISP_ENABLE bit) to configure the
lens s

media: rkisp1: Configure LSC after enabling the ISP

The ISP8000Nano v18.02 (found in the i.MX8MP) requires the ISP to be
enabled (as indicated by the ISP_CTRL.ISP_ENABLE bit) to configure the
lens shading table in internal RAM. The driver currently configures all
ISP initial parameters before enabling the ISP, which causes the LSC RAM
to not be initialized properly.

To fix this, split the rkisp1_params_configure() function into a
rkisp1_params_pre_configure() and a rkisp1_params_post_configure(). The
former configures all ISP parameters but LSC, while the latter
configures LSC. To implement this, the rkisp1_params_apply_params_cfg()
function is deconstructed, with two small helpers created to deal with
the parameters buffers, which are then used in rkisp1_params_isr(),
rkisp1_params_pre_configure() and rkisp1_params_post_configure().

While this initialization ordering is only needed for the ISP8000Nano
v18.02, it doesn't affect other ISP versions negatively, and can thus be
followed unconditionally.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Configure CSM based on YCbCr encoding

The driver currently only implements the Rec. 601 YCbCr encoding, extend
it with support for the other encodings defined by V4L2 (Rec. 709, Rec.

media: rkisp1: Configure CSM based on YCbCr encoding

The driver currently only implements the Rec. 601 YCbCr encoding, extend
it with support for the other encodings defined by V4L2 (Rec. 709, Rec.
2020 and SMPTE240m). The coefficients have been calculated by rounding
the floating point values to the nearest Q1.7 fixed-point value,
adjusting the rounding to ensure that the sum of each line in the matrix
is preserved to avoid overflows.

At the hardware level, the RGB to YUV conversion matrix is fully
configurable, custom encoding could be supported by extending the ISP
parameters if desired.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Add infrastructure to support ISP features

Different ISP versions implement different sets of features. The driver
already takes the version into account in several places, but this
a

media: rkisp1: Add infrastructure to support ISP features

Different ISP versions implement different sets of features. The driver
already takes the version into account in several places, but this
approach won't scale well for features that are found in different
versions. Introduce a new mechanism using a features bitmask in the
rkisp1_info structure to indicate which features the ISP support.

The first feature bit tells if the ISP has an internal CSI-2 receiver,
which is not available in all ISP versions.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Support the ISP parallel input

The ISP has a parallel input, exposed through port 1 in the device tree
node. While the driver supports configuring the ISP for the parallel and
BT.656

media: rkisp1: Support the ISP parallel input

The ISP has a parallel input, exposed through port 1 in the device tree
node. While the driver supports configuring the ISP for the parallel and
BT.656 input modes, the DT parsing code, the subdev bound handler and
the ISP stream start handler only support the CSI input. Extend them to
support the parallel input.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: csi: Implement a V4L2 subdev for the CSI receiver

The CSI receiver is a component separate from the ISP or the resizers.
It is actually optional, not all device model include a CSI re

media: rkisp1: csi: Implement a V4L2 subdev for the CSI receiver

The CSI receiver is a component separate from the ISP or the resizers.
It is actually optional, not all device model include a CSI receiver. On
some SoCs CSI-2 support can be provided through an external CSI-2
receiver, connected to the ISP's parallel input.

To support those use cases, create a V4L2 subdev to model the CSI
receiver. It will allow the driver to handle both internal and external
CSI receivers the same way.

The next commit will plumb the CSI subdev to the rest of the driver,
replacing direct function calls.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: isp: Add rkisp1_device backpointer to rkisp1_isp

The rkisp1_isp structure documentation mentions a backpointer field to
rkisp1_device, but the field is missing. Add it, and use it to

media: rkisp1: isp: Add rkisp1_device backpointer to rkisp1_isp

The rkisp1_isp structure documentation mentions a backpointer field to
rkisp1_device, but the field is missing. Add it, and use it to replace
more complicated constructs using container_of() on the v4l2_device.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: isp: Rename rkisp1_device.active_sensor to source

The active_sensor field of the rkisp1_device structure points to the ASD
data for the active source. The source may however not be a

media: rkisp1: isp: Rename rkisp1_device.active_sensor to source

The active_sensor field of the rkisp1_device structure points to the ASD
data for the active source. The source may however not be a sensor, so
the naming is a bit confusing. Furthermore, the driver doesn't need to
access the full ASD from the active_sensor field, only the subdev
pointer is needed, when stopping streaming.

Rename the field to source, and turn it into a v4l2_subdev pointer.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: csi: Rename CSI functions with a common rkisp1_csi prefix

The CSI-related functions are not named consistently. Fix it by using a
common rkisp1_csi prefix.

Signed-off-by: Laurent Pin

media: rkisp1: csi: Rename CSI functions with a common rkisp1_csi prefix

The CSI-related functions are not named consistently. Fix it by using a
common rkisp1_csi prefix.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Split CSI handling to separate file

Not all ISP instances include a MIPI CSI-2 receiver. To prepare for
making it optional, move code related to the CSI-2 receiver to a
separate file.

media: rkisp1: Split CSI handling to separate file

Not all ISP instances include a MIPI CSI-2 receiver. To prepare for
making it optional, move code related to the CSI-2 receiver to a
separate file.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Fix sensor source pad retrieval at bound time

When a sensor is bound, its source pad is retrieved in the .bound()
operation with a call to media_entity_get_fwnode_pad(). The function

media: rkisp1: Fix sensor source pad retrieval at bound time

When a sensor is bound, its source pad is retrieved in the .bound()
operation with a call to media_entity_get_fwnode_pad(). The function
should be called with the source endpoint fwnode of the sensor, but is
instead called with the sensor's device fwnode.

Fix this, which involves storing a reference to the source endpoint
fwnode in the rkisp1_sensor_async structure, and thus implementing the
subdev notifier .destroy() operation to release the reference.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Create link from sensor to ISP at notifier bound time

Links from sensors to the ISP can be created as sensors are bound. Move
the link creation from rkisp1_create_links() to the bound

media: rkisp1: Create link from sensor to ISP at notifier bound time

Links from sensors to the ISP can be created as sensors are bound. Move
the link creation from rkisp1_create_links() to the bound notifier, and
clean up the rkisp1_create_links() function while at it.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Make rkisp1_isp_mbus_info common

The upcoming CSI receiver split from the ISP to a separate source file
will need to be able to access the list of formats supported by the
driver. Mov

media: rkisp1: Make rkisp1_isp_mbus_info common

The upcoming CSI receiver split from the ISP to a separate source file
will need to be able to access the list of formats supported by the
driver. Move it out of the ISP's header and into the common header, and
add helper functions for accessing it so that the format list doesn't
need to be stored in the header.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Save info pointer in rkisp1_device

To make it possible to use the rkisp1_info after probe time (for
instance to make code conditional on the ISP version), save it in the
main rkisp1_d

media: rkisp1: Save info pointer in rkisp1_device

To make it possible to use the rkisp1_info after probe time (for
instance to make code conditional on the ISP version), save it in the
main rkisp1_device structure. To achieve this, also move the info
structure into the common header, and document it.

While at it, drop a NULL check in rkisp1_probe() for the match data as
it can't happen.

Signed-off-by: Paul Elder <paul.elder@ideasonboard.com>
Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Align macro definitions

Make the code more readable by using the same alignment for all macros
in rkisp1-common.h.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>

media: rkisp1: Align macro definitions

Make the code more readable by using the same alignment for all macros
in rkisp1-common.h.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Compile debugfs support conditionally

When CONFIG_DEBUGFS is disabled, there's no need to compile the debugfs
support in. Make it conditional.

Signed-off-by: Laurent Pinchart <lauren

media: rkisp1: Compile debugfs support conditionally

When CONFIG_DEBUGFS is disabled, there's no need to compile the debugfs
support in. Make it conditional.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Move debugfs code to a separate file

To avoid cluttering the main rkisp1-dev.c driver file, move debugfs code
to a separate source file. This prepares for extensions to the debugfs
in

media: rkisp1: Move debugfs code to a separate file

To avoid cluttering the main rkisp1-dev.c driver file, move debugfs code
to a separate source file. This prepares for extensions to the debugfs
infrastructure.

While at it, add a missing forward declaration for struct dentry in
rkisp1-common.h to avoid depending on indirect includes.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: resizer: Simplify register access

The registers for the mainpath and selfpath resizers are located at the
same offset from the instance-specific base. Use this to simplify
register ac

media: rkisp1: resizer: Simplify register access

The registers for the mainpath and selfpath resizers are located at the
same offset from the instance-specific base. Use this to simplify
register access, removing the need to store per-register offsets in the
rkisp1_rsz_config structure.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rkisp1: Swap value and address arguments to rkisp1_write()

While writel() takes the value and address arguments in that order, most
write functions (including in the regmap API) use the oppos

media: rkisp1: Swap value and address arguments to rkisp1_write()

While writel() takes the value and address arguments in that order, most
write functions (including in the regmap API) use the opposite
convention. Having the value first is considered confusing, and often
leads to more difficult to read code compared to the opposite convention
where the write call and the register name often fit on a single line:

rkisp1_write(rkisp1, RKISP1_CIF_THE_REG_NAME,
complicate_calculation + for / the_register
value + goes | here);

Swap the arguments of the rkisp1_write() function, and use the following
semantic patch to update the callers:

@@
expression rkisp1, value, address;
@@

- rkisp1_write(rkisp1, value, address)
+ rkisp1_write(rkisp1, address, value)

This commit also includes a few additional line break cleanups in the
rkisp1_write() calls, but no other manual change.

Signed-off-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Reviewed-by: Dafna Hirschfeld <dafna@fastmail.com>
Reviewed-by: Ricardo Ribalda <ribalda@chromium.org>
Signed-off-by: Mauro Carvalho Chehab <mchehab@kernel.org>

show more ...

media: rockchip: rkisp1: make some isp-stats functions variable

The isp block evolved in subsequent socs, so some functions
will behave differently on newer variants.

Therefore make it possible to

media: rockchip: rkisp1: make some isp-stats functions variable

The isp block evolved in subsequent socs, so some functions
will behave differently on newer variants.

Therefore make it possible to override the needed stats functions.

Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com>
Reviewed-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

show more ...

media: rockchip: rkisp1: make some isp-param functions variable

The isp block evolved in subsequent socs, so some functions
will behave differently on newer variants.

Therefore make it possible to

media: rockchip: rkisp1: make some isp-param functions variable

The isp block evolved in subsequent socs, so some functions
will behave differently on newer variants.

Therefore make it possible to override the needed params functions.

Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com>
Reviewed-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Reviewed-by: Laurent Pinchart <laurent.pinchart@ideasonboard.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

show more ...

media: rockchip: rkisp1: allow separate interrupts

Depending on the per-soc implementation there can be one interrupt
for isp, mipi and capture events or separate per-event interrupts.

So simply ch

media: rockchip: rkisp1: allow separate interrupts

Depending on the per-soc implementation there can be one interrupt
for isp, mipi and capture events or separate per-event interrupts.

So simply check for the presence of a named "mi" interrupt
to differentiate between the two cases.

Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com>
Reviewed-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

show more ...

media: rockchip: rkisp1: remove unused irq variable

The irq variable in struct rkisp1 is unused as it is only used
to request the irq during probe, so remove it.

Signed-off-by: Heiko Stuebner <heik

media: rockchip: rkisp1: remove unused irq variable

The irq variable in struct rkisp1 is unused as it is only used
to request the irq during probe, so remove it.

Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com>
Reviewed-by: Dafna Hirschfeld <dafna.hirschfeld@collabora.com>
Signed-off-by: Hans Verkuil <hverkuil-cisco@xs4all.nl>
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org>

show more ...