feat: add support for com.docker.network.bridge.enable_icc network option #4311

swagatbora90 · 2025-06-10T23:32:15Z

This PR adds support for docker compatible com.docker.network.bridge.enable_icc or --icc network create option. This option is used to enable/disable intercontainer connectivity.

By default, any bridge network allows connectivity between containers attached to the same network, while containers in different networks are isolated. The enable_icc feature can be further used to enable/disable intra bridge container connectivity.

Requires firewall plugin >= v1.7.1

Testing

Create a network with com.docker.network.bridge.enable_icc set to false, say test-net
Run a container and attach it to test-net
Run a second container and attach it to test-net
Ping from container 1 to container 2 should not succeed.

sudo nerdctl network create  -o "com.docker.network.bridge.enable_icc=false" test-net          
254ddbe0ec1e5f3ed4b492876e2b6b9a051436b44c40d00ebd5e72bf7aa88435

sudo nerdctl run -d  --network test-net --name C1 ubuntu  sleep infinity  
6f22e651b9a4f3988e1ebce35f13806f65d0be942bfe9a2325c668a53cafcc9b

 sudo nerdctl run --network test-net alpine ping -c 1 -W 1 C1
PING C1 (10.4.4.4): 56 data bytes

--- C1 ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss

With icc=true, it will fallback to default behavior

sudo nerdctl network create  -o "com.docker.network.bridge.enable_icc=true" test-net1   
cf021853e2fd31f5ef29264e5fd834d45bf1ea364c3fef8ff60edec326899b77
 
sudo nerdctl run -d  --network test-net1 --name C2 ubuntu  sleep infinity  
2c95ae6ad9da8a07de97c609d9e65acdc95f3b343ea6e4c46ccd4c9b3df79b90

sudo nerdctl run --network test-net1 alpine ping -c 1 -W 1 C2
PING C2 (10.4.5.2): 56 data bytes
64 bytes from 10.4.5.2: seq=0 ttl=127 time=0.115 ms

--- C2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.115/0.115/0.115 ms

apostasie · 2025-06-11T01:19:30Z

cmd/nerdctl/network/network_create_linux_test.go

+				return helpers.Command("run", "--rm", "--net", data.Identifier(),
+					testutil.CommonImage, "ping", "-c", "1", "-W", "1", data.Identifier("c1"))
+			},
+			Expected: test.Expects(1, nil, nil), // Expect ping to fail with exit code 1


Wondering if ping portably returns that same exit code.
Maybe use expect.ExitCodeGenericFail to check for failure without being specific on exit code?

eg: on macOS, it seems to exit 2

I will change this to just check for any non-zero exit code

apostasie · 2025-06-11T01:21:51Z

pkg/netutil/netutil_unix.go

+}
+
+// firewallPluginGEQ110 checks if the firewall plugin is greater than or equal to v1.1.0
+func firewallPluginGEQ110(firewallPath string) (bool, error) {


Maybe we can just remove this helper and call directly firewallPluginGEQVersion(firewallPath, "v1.1.0") where needs be?

apostasie · 2025-06-11T01:25:24Z

cmd/nerdctl/network/network_create_linux_test.go

+					testutil.CommonImage, "ping", "-c", "1", "-W", "1", data.Identifier("c1"))
+			},
+			Expected: test.Expects(0, nil, nil), // Expect ping to succeed with exit code 0
+		},


Maybe add a third case where enable_icc is not specified, to verify the default behavior?

Sure, will add

pkg/netutil/netutil_unix.go

AkihiroSuda · 2025-06-11T02:47:18Z

pkg/netutil/netutil_unix.go

@@ -111,6 +112,11 @@ func (e *CNIEnv) generateCNIPlugins(driver string, name string, ipam map[string]
 				if err != nil {
 					return nil, err
 				}
+			case "icc", "com.docker.network.bridge.enable_icc":


Needs documentation

pkg/testutil/nerdtest/requirements.go

AkihiroSuda · 2025-06-12T04:54:56Z

cmd/nerdctl/network/network_create_linux_test.go

+
+	testCase.Require = require.All(
+		require.Linux,
+		nerdtest.Rootful,


AkihiroSuda

Thanks

swagatbora90 · 2025-06-16T16:36:29Z

Looks like the test disabling ICC is failing on a few platform. Taking a look.

AkihiroSuda · 2025-06-30T01:59:26Z

Moving to the v2.1.4 milestone

AkihiroSuda · 2025-08-19T05:51:29Z

Needs rebase, in addition to fixing the CI

…tion Signed-off-by: Swagat Bora <[email protected]>

swagatbora90 · 2025-08-21T21:52:45Z

Needs rebase, in addition to fixing the CI

@AkihiroSuda I had a hard time debugging the CI failures since this was working on my local test host. Finally, figured out that the the test are failing because the ubuntu runners do not have br-netfilter module loaded, which is required for this feature to work. We need the bridged traffic to go through netfilter.

We will need to load the module in our ubuntu runners first for this test to work. Given we have other features that also rely on cni firewall rules, I am thinking of enabling it across all the runners. let me know what you think?

AkihiroSuda · 2025-08-22T03:28:26Z

We will need to load the module in our ubuntu runners first for this test to work. Given we have other features that also rely on cni firewall rules, I am thinking of enabling it across all the runners. let me know what you think?

👍, but maybe you do not need to modprobe it explicitly if you mount -v /lib/modules:/lib/modules:ro here:

nerdctl/.github/workflows/job-test-in-container.yml

Line 166 in 832c455

    
                       docker run -t --rm --privileged -e GITHUB_STEP_SUMMARY="$GITHUB_STEP_SUMMARY" -v "$GITHUB_STEP_SUMMARY":"$GITHUB_STEP_SUMMARY" -e WORKAROUND_ISSUE_622=${WORKAROUND_ISSUE_622:-} "${args[@]}" -test.only-flaky=false -test.target=${{ inputs.binary }}

(and in the similar places in other test-in-*.yaml)

swagatbora90 · 2025-08-22T21:22:45Z

but maybe you do not need to modprobe it explicitly if you mount -v /lib/modules:/lib/modules:ro here

I tried this option, but ran into the same errors swagatbora90@06d49d0. Looks like I need to load the br_netfilter module explicitly as they are not loaded by default.

swagatbora90 · 2025-08-22T21:32:45Z

Opened [https://github.com//pull/4481] to load the br_netfilter module inn the runners.

apostasie reviewed Jun 11, 2025

View reviewed changes

pkg/netutil/netutil_unix.go Show resolved Hide resolved

AkihiroSuda reviewed Jun 11, 2025

View reviewed changes

swagatbora90 force-pushed the feat-icc-nerdctl branch 3 times, most recently from c85e72d to bea516f Compare June 12, 2025 00:32

AkihiroSuda reviewed Jun 12, 2025

View reviewed changes

pkg/testutil/nerdtest/requirements.go Outdated Show resolved Hide resolved

AkihiroSuda reviewed Jun 12, 2025

View reviewed changes

cmd/nerdctl/network/network_create_linux_test.go Outdated

testCase.Require = require.All(

require.Linux,

nerdtest.Rootful,

Copy link

Member

AkihiroSuda Jun 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why?

AkihiroSuda approved these changes Jun 12, 2025

View reviewed changes

AkihiroSuda added this to the v2.1.3 milestone Jun 12, 2025

swagatbora90 force-pushed the feat-icc-nerdctl branch from bea516f to e0f7d38 Compare June 12, 2025 07:25

swagatbora90 closed this Jun 18, 2025

swagatbora90 reopened this Jun 18, 2025

AkihiroSuda mentioned this pull request Jun 20, 2025

v2.1.3 planning #4331

Closed

AkihiroSuda modified the milestones: v2.1.3, v2.1.4 Jun 30, 2025

swagatbora90 force-pushed the feat-icc-nerdctl branch from e0f7d38 to a518963 Compare July 15, 2025 16:33

swagatbora90 force-pushed the feat-icc-nerdctl branch 4 times, most recently from 546602c to 9ce8625 Compare July 24, 2025 23:43

swagatbora90 force-pushed the feat-icc-nerdctl branch from 9ce8625 to c83c41a Compare August 21, 2025 21:11

feat: add support for com.docker.network.bridge.enable_icc network op…

8d9aa54

…tion Signed-off-by: Swagat Bora <[email protected]>

swagatbora90 force-pushed the feat-icc-nerdctl branch from c83c41a to 8d9aa54 Compare August 21, 2025 21:13

swagatbora90 mentioned this pull request Aug 22, 2025

ci: Load br_netfilter module in linux runners #4481

Open

feat: add support for com.docker.network.bridge.enable_icc network option #4311

Are you sure you want to change the base?

feat: add support for com.docker.network.bridge.enable_icc network option #4311

Conversation

swagatbora90 commented Jun 10, 2025

Testing

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AkihiroSuda left a comment

Choose a reason for hiding this comment

Uh oh!

swagatbora90 commented Jun 16, 2025

Uh oh!

AkihiroSuda commented Jun 30, 2025

Uh oh!

AkihiroSuda commented Aug 19, 2025

Uh oh!

swagatbora90 commented Aug 21, 2025

Uh oh!

AkihiroSuda commented Aug 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swagatbora90 commented Aug 22, 2025

Uh oh!

swagatbora90 commented Aug 22, 2025

Uh oh!

Uh oh!

AkihiroSuda commented Aug 22, 2025 •

edited

Loading