tc-testing: concurrency: wrap piped rule update commands

Concurrent tests use several commands to update rules in parallel: 'find'
prints names of batch files in tmp directory and pipes result to 'x

tc-testing: concurrency: wrap piped rule update commands

Concurrent tests use several commands to update rules in parallel: 'find'
prints names of batch files in tmp directory and pipes result to 'xargs'
which runs instance of tc per batch file in parallel. This breaks when used
with ns plugin that adds 'ip netns exec $NS' prefix to the command, which
causes only first command in pipe to be executed in namespace:

=====> Test e41d: Add 1M flower filters with 10 parallel tc instances
-----> prepare stage
ns/SubPlugin.adjust_command
adjust_command: stage is setup; inserting netns stuff in command [/bin/mkdir tmp] list [['/bin/mkdir', 'tmp']]
adjust_command: return command [ip netns exec tcut /bin/mkdir tmp]
command "ip netns exec tcut /bin/mkdir tmp"
ns/SubPlugin.adjust_command
adjust_command: stage is setup; inserting netns stuff in command [/sbin/tc qdisc add dev ens1f0 ingress] list [['/sbin/tc', 'qdisc', 'add', 'dev', 'ens1f0', 'ingress']]
adjust_command: return command [ip netns exec tcut /sbin/tc qdisc add dev ens1f0 ingress]
command "ip netns exec tcut /sbin/tc qdisc add dev ens1f0 ingress"
ns/SubPlugin.adjust_command
adjust_command: stage is setup; inserting netns stuff in command [./tdc_multibatch.py ens1f0 tmp 100000 10 add] list [['./tdc_multibatch.py', 'ens1f0', 'tmp', '100000', '10', 'add']]
adjust_command: return command [ip netns exec tcut ./tdc_multibatch.py ens1f0 tmp 100000 10 add]
command "ip netns exec tcut ./tdc_multibatch.py ens1f0 tmp 100000 10 add"
-----> execute stage
ns/SubPlugin.adjust_command
adjust_command: stage is execute; inserting netns stuff in command [find tmp/add* -print | xargs -n 1 -P 10 /sbin/tc -b] list [['find', 'tmp/add*', '-print', '|', 'xargs', '-n', '1', '-P', '10', '/sbin/tc', '-b']
]
adjust_command: return command [ip netns exec tcut find tmp/add* -print | xargs -n 1 -P 10 /sbin/tc -b]
command "ip netns exec tcut find tmp/add* -print | xargs -n 1 -P 10 /sbin/tc -b"
exit: 123
exit: 0
Cannot find device "ens1f0"
Cannot find device "ens1f0"
Command failed tmp/add_0:1
Command failed tmp/add_1:1
Cannot find device "ens1f0"
Command failed tmp/add_2:1
Cannot find device "ens1f0"
Command failed tmp/add_4:1
Cannot find device "ens1f0"
Command failed tmp/add_3:1
Cannot find device "ens1f0"
Command failed tmp/add_5:1
Cannot find device "ens1f0"
Command failed tmp/add_6:1
Cannot find device "ens1f0"
Command failed tmp/add_8:1
Cannot find device "ens1f0"
Command failed tmp/add_7:1
Cannot find device "ens1f0"
Command failed tmp/add_9:1

Fix the issue by executing whole compound command in namespace by wrapping
it in 'bash -c' invocation.

Fixes: 489ce2f42514 ("tc-testing: Restore original behaviour for namespaces in tdc")
Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel replace/delete

Implement test that runs 5 instances of tc replace filter in parallel with
5 instances of tc del filter from same tp instance. Each

selftests: concurrency: add test to verify parallel replace/delete

Implement test that runs 5 instances of tc replace filter in parallel with
5 instances of tc del filter from same tp instance. Each instance uses its
own filter handle and key range.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel add/delete

Implement test that runs 5 instances of tc add filter in parallel with 5
instances of tc del filter from same tp instance. Each instanc

selftests: concurrency: add test to verify parallel add/delete

Implement test that runs 5 instances of tc add filter in parallel with 5
instances of tc del filter from same tp instance. Each instance uses its
own filter handle and key range.

Extend tdc_multibatch.py with additional options required to implement the
test: common prefix for all generated batch files, first value of filter
handle range, MAC address prefix modifier. These are necessary to allow
creating batch files with unique keys and handle ranges with multiple
invocation of tdc_multibatch.py helper script.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify concurrent delete

Implement test that verifies concurrent deletion of rules by executing 10
tc instances that delete flower filters in same handle range. I

selftests: concurrency: add test to verify concurrent delete

Implement test that verifies concurrent deletion of rules by executing 10
tc instances that delete flower filters in same handle range. In this case
only one tc instance succeeds in deleting a filter with particular handle.
To mitigate expected failures of all other instances, run tc with 'force'
option to continue processing batch file in case of errors and expect xargs
to return code '123' that indicates that invocation of command(s) exited
with error in range 1-125.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify concurrent replace

Implement test that verifies concurrent replacement of rules by executing
10 tc instances that replace flower filters in same handle ran

selftests: concurrency: add test to verify concurrent replace

Implement test that verifies concurrent replacement of rules by executing
10 tc instances that replace flower filters in same handle range.

Extend tdc_multibatch.py script with new optional CLI argument that is used
to generate all batch files with same filter handle range.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel rules replace

Implement test that verifies parallel rules replacement by adding 1 million
flower filters and then replacing them with 10 concurren

selftests: concurrency: add test to verify parallel rules replace

Implement test that verifies parallel rules replacement by adding 1 million
flower filters and then replacing them with 10 concurrent tc instances.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel rules deletion

Implement test that verifies parallel rules deletion by adding 1 million
flower filters and then deleting them with 10 concurrent t

selftests: concurrency: add test to verify parallel rules deletion

Implement test that verifies parallel rules deletion by adding 1 million
flower filters and then deleting them with 10 concurrent tc instances.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel rules insertion

Implement test that verifies parallel rules insertion by adding 1 million
flower filters with 10 concurrent tc instances. Put it t

selftests: concurrency: add test to verify parallel rules insertion

Implement test that verifies parallel rules insertion by adding 1 million
flower filters with 10 concurrent tc instances. Put it to standalone
'concurrency' category.

Implement tdc_multibatch.py helper script that is used to generate multiple
batch files for concurrent tc execution. Extend config with new 'BATCH_DIR'
variable to specify temporary output directory that is used to store batch
files generated by tdc_multibatch.py.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

tc-testing: concurrency: wrap piped rule update commands

Concurrent tests use several commands to update rules in parallel: 'find'
prints names of batch files in tmp directory and pipes

tc-testing: concurrency: wrap piped rule update commands

Concurrent tests use several commands to update rules in parallel: 'find'
prints names of batch files in tmp directory and pipes result to 'xargs'
which runs instance of tc per batch file in parallel. This breaks when used
with ns plugin that adds 'ip netns exec $NS' prefix to the command, which
causes only first command in pipe to be executed in namespace:

=====> Test e41d: Add 1M flower filters with 10 parallel tc instances
-----> prepare stage
ns/SubPlugin.adjust_command
adjust_command: stage is setup; inserting netns stuff in command [/bin/mkdir tmp] list [['/bin/mkdir', 'tmp']]
adjust_command: return command [ip netns exec tcut /bin/mkdir tmp]
command "ip netns exec tcut /bin/mkdir tmp"
ns/SubPlugin.adjust_command
adjust_command: stage is setup; inserting netns stuff in command [/sbin/tc qdisc add dev ens1f0 ingress] list [['/sbin/tc', 'qdisc', 'add', 'dev', 'ens1f0', 'ingress']]
adjust_command: return command [ip netns exec tcut /sbin/tc qdisc add dev ens1f0 ingress]
command "ip netns exec tcut /sbin/tc qdisc add dev ens1f0 ingress"
ns/SubPlugin.adjust_command
adjust_command: stage is setup; inserting netns stuff in command [./tdc_multibatch.py ens1f0 tmp 100000 10 add] list [['./tdc_multibatch.py', 'ens1f0', 'tmp', '100000', '10', 'add']]
adjust_command: return command [ip netns exec tcut ./tdc_multibatch.py ens1f0 tmp 100000 10 add]
command "ip netns exec tcut ./tdc_multibatch.py ens1f0 tmp 100000 10 add"
-----> execute stage
ns/SubPlugin.adjust_command
adjust_command: stage is execute; inserting netns stuff in command [find tmp/add* -print | xargs -n 1 -P 10 /sbin/tc -b] list [['find', 'tmp/add*', '-print', '|', 'xargs', '-n', '1', '-P', '10', '/sbin/tc', '-b']
]
adjust_command: return command [ip netns exec tcut find tmp/add* -print | xargs -n 1 -P 10 /sbin/tc -b]
command "ip netns exec tcut find tmp/add* -print | xargs -n 1 -P 10 /sbin/tc -b"
exit: 123
exit: 0
Cannot find device "ens1f0"
Cannot find device "ens1f0"
Command failed tmp/add_0:1
Command failed tmp/add_1:1
Cannot find device "ens1f0"
Command failed tmp/add_2:1
Cannot find device "ens1f0"
Command failed tmp/add_4:1
Cannot find device "ens1f0"
Command failed tmp/add_3:1
Cannot find device "ens1f0"
Command failed tmp/add_5:1
Cannot find device "ens1f0"
Command failed tmp/add_6:1
Cannot find device "ens1f0"
Command failed tmp/add_8:1
Cannot find device "ens1f0"
Command failed tmp/add_7:1
Cannot find device "ens1f0"
Command failed tmp/add_9:1

Fix the issue by executing whole compound command in namespace by wrapping
it in 'bash -c' invocation.

Fixes: 489ce2f42514 ("tc-testing: Restore original behaviour for namespaces in tdc")
Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel replace/delete

Implement test that runs 5 instances of tc replace filter in parallel with
5 instances of tc del filter from same tp in

selftests: concurrency: add test to verify parallel replace/delete

Implement test that runs 5 instances of tc replace filter in parallel with
5 instances of tc del filter from same tp instance. Each instance uses its
own filter handle and key range.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel add/delete

Implement test that runs 5 instances of tc add filter in parallel with 5
instances of tc del filter from same tp instance.

selftests: concurrency: add test to verify parallel add/delete

Implement test that runs 5 instances of tc add filter in parallel with 5
instances of tc del filter from same tp instance. Each instance uses its
own filter handle and key range.

Extend tdc_multibatch.py with additional options required to implement the
test: common prefix for all generated batch files, first value of filter
handle range, MAC address prefix modifier. These are necessary to allow
creating batch files with unique keys and handle ranges with multiple
invocation of tdc_multibatch.py helper script.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify concurrent delete

Implement test that verifies concurrent deletion of rules by executing 10
tc instances that delete flower filters in same han

selftests: concurrency: add test to verify concurrent delete

Implement test that verifies concurrent deletion of rules by executing 10
tc instances that delete flower filters in same handle range. In this case
only one tc instance succeeds in deleting a filter with particular handle.
To mitigate expected failures of all other instances, run tc with 'force'
option to continue processing batch file in case of errors and expect xargs
to return code '123' that indicates that invocation of command(s) exited
with error in range 1-125.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify concurrent replace

Implement test that verifies concurrent replacement of rules by executing
10 tc instances that replace flower filters in sam

selftests: concurrency: add test to verify concurrent replace

Implement test that verifies concurrent replacement of rules by executing
10 tc instances that replace flower filters in same handle range.

Extend tdc_multibatch.py script with new optional CLI argument that is used
to generate all batch files with same filter handle range.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel rules replace

Implement test that verifies parallel rules replacement by adding 1 million
flower filters and then replacing them with

selftests: concurrency: add test to verify parallel rules replace

Implement test that verifies parallel rules replacement by adding 1 million
flower filters and then replacing them with 10 concurrent tc instances.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel rules deletion

Implement test that verifies parallel rules deletion by adding 1 million
flower filters and then deleting them with 10

selftests: concurrency: add test to verify parallel rules deletion

Implement test that verifies parallel rules deletion by adding 1 million
flower filters and then deleting them with 10 concurrent tc instances.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...

selftests: concurrency: add test to verify parallel rules insertion

Implement test that verifies parallel rules insertion by adding 1 million
flower filters with 10 concurrent tc instanc

selftests: concurrency: add test to verify parallel rules insertion

Implement test that verifies parallel rules insertion by adding 1 million
flower filters with 10 concurrent tc instances. Put it to standalone
'concurrency' category.

Implement tdc_multibatch.py helper script that is used to generate multiple
batch files for concurrent tc execution. Extend config with new 'BATCH_DIR'
variable to specify temporary output directory that is used to store batch
files generated by tdc_multibatch.py.

Signed-off-by: Vlad Buslov <vladbu@mellanox.com>
Signed-off-by: David S. Miller <davem@davemloft.net>

show more ...