net/mlx5: DR, Fix potential race in dr_rule_create_rule_nic

Selecting builder should be protected by the lock to prevent the case
where a new rule sets a builder in the nic_matcher while the previou

net/mlx5: DR, Fix potential race in dr_rule_create_rule_nic

Selecting builder should be protected by the lock to prevent the case
where a new rule sets a builder in the nic_matcher while the previous
rule is still using the nic_matcher.

Fixing this issue and cleaning the error flow.

Fixes: b9b81e1e9382 ("net/mlx5: DR, For short chains of STEs, avoid allocating ste_arr dynamically")
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Fix 'stack frame size exceeds limit' error in dr_rule

If the kernel configuration asks the compiler to check frame limit of 1K,
dr_rule_create_rule_nic exceed this limit:
"stack fr

net/mlx5: DR, Fix 'stack frame size exceeds limit' error in dr_rule

If the kernel configuration asks the compiler to check frame limit of 1K,
dr_rule_create_rule_nic exceed this limit:
"stack frame size (1184) exceeds limit (1024)"

Fixing this issue by checking configured frame limit and using the
optimization STE array only for cases with the usual 2K (or larger)
stack size warning.

Fixes: b9b81e1e9382 ("net/mlx5: DR, For short chains of STEs, avoid allocating ste_arr dynamically")
Reported-by: kernel test robot <lkp@intel.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>

show more ...

net/mlx5: DR, Add function that tells if STE miss addr has been initialized

Up until now miss address in all the STEs was used to connect miss lists
and to link the last STE in the list to end ancho

net/mlx5: DR, Add function that tells if STE miss addr has been initialized

Up until now miss address in all the STEs was used to connect miss lists
and to link the last STE in the list to end anchor.
Match range STE will require special handling because its miss address is
part of the 'action'. That is, range action has hit and miss addresses.
Since the range action is always the last action, need to make sure that
its miss address isn't overwritten by the end anchor.

Adding new function mlx5dr_ste_is_miss_addr_set() to answer the question
whether the STE's miss address has already been set as part of STE
initialization. Use a callback that always returns false right now. Once
match range is added, a different callback will be used for that STE type.

Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Erez Shitrit <erezsh@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Some refactoring of miss address handling

In preparation for MATCH RANGE STE support, create a function
to set the miss address of an STE.

Signed-off-by: Yevgeny Kliteynik <kliteyn@nv

net/mlx5: DR, Some refactoring of miss address handling

In preparation for MATCH RANGE STE support, create a function
to set the miss address of an STE.

Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Erez Shitrit <erezsh@nvidia.com>
Reviewed-by: Mark Bloch <mbloch@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Fix matcher disconnect error flow

When 2nd flow rules arrives, it will merge together with the
1st one if matcher criteria is the same.

If merge fails, driver will rollback the merge

net/mlx5: DR, Fix matcher disconnect error flow

When 2nd flow rules arrives, it will merge together with the
1st one if matcher criteria is the same.

If merge fails, driver will rollback the merge contents, and
reject the 2nd rule. At rollback stage, matcher can't be
disconnected unconditionally, otherise the 1st rule can't be
hit anymore.

Add logic to check if the matcher should be disconnected or not.

Fixes: cc2295cd54e4 ("net/mlx5: DR, Improve steering for empty or RX/TX-only matchers")
Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
Link: https://lore.kernel.org/r/20221026135153.154807-4-saeed@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>

show more ...

net/mlx5: DR, Manage STE send info objects in pool

Instead of allocating/freeing send info objects dynamically, manage them
in pool. The number of send info objects doesn't depend on rules, so after

net/mlx5: DR, Manage STE send info objects in pool

Instead of allocating/freeing send info objects dynamically, manage them
in pool. The number of send info objects doesn't depend on rules, so after
pre-populating the pool with an initial batch of send info objects, the
pool is not expected to grow.
This way we save alloc/free during writing STEs to ICM, which can
sometimes take up to 40msec.

Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, In rehash write the line in the entry immediately

Don't wait for the whole table to be ready - write each row immediately.
This way we save allocations of the ste_send_info structure a

net/mlx5: DR, In rehash write the line in the entry immediately

Don't wait for the whole table to be ready - write each row immediately.
This way we save allocations of the ste_send_info structure and improve
performance.

Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, For short chains of STEs, avoid allocating ste_arr dynamically

While creating rule, ste_arr is an array that is allocated at the start
of the function and freed at the end.
This memory

net/mlx5: DR, For short chains of STEs, avoid allocating ste_arr dynamically

While creating rule, ste_arr is an array that is allocated at the start
of the function and freed at the end.
This memory allocation can sometimes lead to "hiccups" of up to 10ms.
However, the common use case is short chains of STEs. For such cases,
we can use a local buffer on stack instead.

Changes in v2:
Use small local array for short rules, allocate dynamically for long rules

Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Remove hw_ste from mlx5dr_ste to reduce memory

It can be calculated via function mlx5dr_ste_get_hw_ste().
Very simple and lightweight, no need to use a dedicated member.

Reduce 8 byte

net/mlx5: DR, Remove hw_ste from mlx5dr_ste to reduce memory

It can be calculated via function mlx5dr_ste_get_hw_ste().
Very simple and lightweight, no need to use a dedicated member.

Reduce 8 bytes from struct mlx5dr_ste and its size is 48 bytes now.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Reviewed-by: Shun Hao <shunh@nvidia.com>
Reviewed-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Remove 4 members from mlx5dr_ste_htbl to reduce memory

Remove chunk_size in struct mlx5dr_icm_chunk and use
chunk->size instead.

Remove ste_arr/hw_ste_arr/miss_list since they can be

net/mlx5: DR, Remove 4 members from mlx5dr_ste_htbl to reduce memory

Remove chunk_size in struct mlx5dr_icm_chunk and use
chunk->size instead.

Remove ste_arr/hw_ste_arr/miss_list since they can be accessed
from htbl->chunk pointer, no need to keep a copy.

This commit reduces 28 bytes from struct mlx5dr_ste_htbl and its
size is 32 bytes now.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Reviewed-by: Shun Hao <shunh@nvidia.com>
Reviewed-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Remove num_of_entries byte_size from struct mlx5_dr_icm_chunk

Target to reduce the memory consumption in large scale of flow rules.

They can be calculated quickly from buddy memory po

net/mlx5: DR, Remove num_of_entries byte_size from struct mlx5_dr_icm_chunk

Target to reduce the memory consumption in large scale of flow rules.

They can be calculated quickly from buddy memory pool.
1. num_of_entries calls dr_icm_pool_get_chunk_num_of_entries().
2. byte_size calls dr_icm_pool_get_chunk_byte_size().

Use chunk size in dr_icm_chunk to speed up and the one in dr_ste_htbl
will be removed in the upcoming commit.

This commit reduce 8 bytes from struct mlx5_dr_icm_chunk and its
current size is 56 bytes.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Reviewed-by: Shun Hao <shunh@nvidia.com>
Reviewed-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Remove icm_addr from mlx5dr_icm_chunk to reduce memory

It can be calculated quickly from buddy memory pool by
function mlx5dr_icm_pool_get_chunk_icm_addr().
This function is very light

net/mlx5: DR, Remove icm_addr from mlx5dr_icm_chunk to reduce memory

It can be calculated quickly from buddy memory pool by
function mlx5dr_icm_pool_get_chunk_icm_addr().
This function is very lightweight and straightforward.

Reduce 8 bytes and current size of struct mlx5_dr_icm_chunk
is 64 bytes.

Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Reviewed-by: Shun Hao <shunh@nvidia.com>
Reviewed-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Improve steering for empty or RX/TX-only matchers

Every matcher has RX and TX paths. When a new matcher is created, its RX
and TX start/end anchors are connected to the respective RX a

net/mlx5: DR, Improve steering for empty or RX/TX-only matchers

Every matcher has RX and TX paths. When a new matcher is created, its RX
and TX start/end anchors are connected to the respective RX and TX anchors
of the previous and next matchers.
This creates a potential performance issue: when a certain rule is added
to a matcher, in many cases it is RX or TX only rule, which may create a
long chain of RX/TX-only paths w/o the actual rules.

This patch aims to handle this issue.

RX and TX matchers are now handled separately: matcher connection in the
matchers chain is split into two separate lists: RX only and TX only.
when a new matcher is created, it is initially created 'detached' - its
RX/TX members are not inserted into the table's matcher list.
When an actual rule is added, only its appropriate RX or TX nic matchers
are then added to the table's nic matchers list and inserted into its
place in the chain of matchers.
I.e., if the rule that is being added is an RX-only rule, only the RX
part of the matcher will be connected to the chain, while TX part of the
matcher remains detached and doesn't prolong the TX chain of the matchers.

Same goes for rule deletion: when the last RX/TX rule of the nic matcher
is destroyed, the nic matcher is removed from its list.

Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>

show more ...

net/mlx5: DR, Add misc5 to match_param structs

Add misc5 match params to enable matching tunnel headers.

Signed-off-by: Muhammad Sammar <muhammads@nvidia.com>

net/mlx5: DR, Add support for dumping steering info

Extend mlx5 debugfs support to present Software Steering resources:
dr_domain including it's tables, matchers and rules.
The interface is read-onl

net/mlx5: DR, Add support for dumping steering info

Extend mlx5 debugfs support to present Software Steering resources:
dr_domain including it's tables, matchers and rules.
The interface is read-only. While dump is being presented, new steering
rules cannot be inserted/deleted.

The steering information is dumped in the CSV form with the following
format:

<object_type>,<object_ID>, <object_info>,...,<object_info>

This data can be read at the following path:

/sys/kernel/debug/mlx5/<BDF>/steering/fdb/<domain_handle>

Example:

# cat /sys/kernel/debug/mlx5/0000:82:00.0/steering/fdb/dmn_000018644
3100,0x55caa4621c50,0xee802,4,65533
3101,0x55caa4621c50,0xe0100008

Changes in V2:
- Reduce temp hex buffer size and avoid unnecessary memset
- Use bin2hex() instead of DIY loop
- Don't check debugfs functions return values

Signed-off-by: Muhammad Sammar <muhammads@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>

show more ...

net/mlx5: DR, Add check for unsupported fields in match param

When a matcher is being built, we "consume" (clear) mask fields one by one,
and to verify that we do support all the required fields we

net/mlx5: DR, Add check for unsupported fields in match param

When a matcher is being built, we "consume" (clear) mask fields one by one,
and to verify that we do support all the required fields we check if the
whole mask was consumed, else the matching request includes unsupported
fields.

Signed-off-by: Muhammad Sammar <muhammads@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>
Reviewed-by: Yevgeny Kliteynik <kliteyn@nvidia.com>

show more ...

net/mlx5: DR, Replace local WIRE_PORT macro with the existing MLX5_VPORT_UPLINK

SW steering defines its own macro for uplink vport number.
Replace this macro with an already existing mlx5 macro.

Si

net/mlx5: DR, Replace local WIRE_PORT macro with the existing MLX5_VPORT_UPLINK

SW steering defines its own macro for uplink vport number.
Replace this macro with an already existing mlx5 macro.

Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Improve rule tracking memory consumption

To track each STE of the rule a rule member was allocated, each
member would point to one STE. This means that we would allocate
40B (rule memb

net/mlx5: DR, Improve rule tracking memory consumption

To track each STE of the rule a rule member was allocated, each
member would point to one STE. This means that we would allocate
40B (rule member) * number of STEs per rule.

To reduce this per rule allocation we use the STE tree pointers
for next_htbl and pointing STE to navigate the tree, this allows
us to keep only the pointer to the last STE of rule (always unique).
From the last rule STE we are able to traverse and rebuild all of
the STEs that construct the rule.

In our testing with 8M rules, each consisting of 7 STES, we were able
to reduce 1.6GB of memory.

Signed-off-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Remove rehash ctrl struct from dr_htbl

The calculations to decide for the maximum allowed collision threshold
are simple and there is no reason to save them on the htbl struct.

Signed

net/mlx5: DR, Remove rehash ctrl struct from dr_htbl

The calculations to decide for the maximum allowed collision threshold
are simple and there is no reason to save them on the htbl struct.

Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Remove HW specific STE type from nic domain

Instead of using the HW specific STEv0 type, it is better to use
an enum to indicate if this is an RX or TX nic domain.
This means that now

net/mlx5: DR, Remove HW specific STE type from nic domain

Instead of using the HW specific STEv0 type, it is better to use
an enum to indicate if this is an RX or TX nic domain.
This means that now we will need to convert the nic domain type
to the corresponding STE type.

Signed-off-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, fix a potential use-after-free bug

In line 849 (#1), "mlx5dr_htbl_put(cur_htbl);" drops the reference to
cur_htbl and may cause cur_htbl to be freed.

However, cur_htbl is subsequently

net/mlx5: DR, fix a potential use-after-free bug

In line 849 (#1), "mlx5dr_htbl_put(cur_htbl);" drops the reference to
cur_htbl and may cause cur_htbl to be freed.

However, cur_htbl is subsequently used in the next line, which may result
in an use-after-free bug.

Fix this by calling mlx5dr_err() before the cur_htbl is put.

Signed-off-by: Wentao_Liang <Wentao_Liang_g@163.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Add support for dynamic flex parser

Flex parser is a HW parser that can support protocols that are not
natively supported by the HCA, such as Geneve (TLV options) and GTP-U.
There are

net/mlx5: DR, Add support for dynamic flex parser

Flex parser is a HW parser that can support protocols that are not
natively supported by the HCA, such as Geneve (TLV options) and GTP-U.
There are 8 such parsers, and each of them can be assigned to parse a
specific set of protocols.
This patch adds misc4 match params which allows using a correct flex parser
that was programmed to the required protocol.

Signed-off-by: Muhammad Sammar <muhammads@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Use HW specific logic API when writing STE

STEv0 format and STEv1 HW format are different, each has a
different order:
STEv0: CTRL 32B, TAG 16B, BITMASK 16B
STEv1: CTRL 32B, BITMASK 16

net/mlx5: DR, Use HW specific logic API when writing STE

STEv0 format and STEv1 HW format are different, each has a
different order:
STEv0: CTRL 32B, TAG 16B, BITMASK 16B
STEv1: CTRL 32B, BITMASK 16B, TAG 16B

To make this transparent to upper layers we introduce a
new ste_ctx function to format the STE prior to writing it.

Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Use the right size when writing partial STE into HW

In these cases we need to update only the ctrl area of the STE.
So it is better to write only the control 32B and avoid copying
the

net/mlx5: DR, Use the right size when writing partial STE into HW

In these cases we need to update only the ctrl area of the STE.
So it is better to write only the control 32B and avoid copying
the unneeded reduced 48B (control 32B + tag 16B).

Signed-off-by: Erez Shitrit <erezsh@nvidia.com>
Signed-off-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...

net/mlx5: DR, Move STEv0 setters and getters

Use the new setters and getters API for STEv0: move HW specific setter and
getters from dr_ste to STEv0 file. Since STEv0 and STEv1 format are
different

net/mlx5: DR, Move STEv0 setters and getters

Use the new setters and getters API for STEv0: move HW specific setter and
getters from dr_ste to STEv0 file. Since STEv0 and STEv1 format are
different each version should implemented different setters and getters.
Rename remaining static functions w/o mlx5 prefix.

Signed-off-by: Alex Vesker <valex@nvidia.com>
Signed-off-by: Yevgeny Kliteynik <kliteyn@nvidia.com>
Reviewed-by: Saeed Mahameed <saeedm@nvidia.com>
Signed-off-by: Saeed Mahameed <saeedm@nvidia.com>

show more ...