Skip to content

Commit d33b2bc

Browse files
troglobittroglobit
committed
New post
Cross posted from the original done for Addiva Elektronik, March 11. https://addiva-elektronik.github.io/2024/03/11/advancd-container-networking/ Signed-off-by: Joachim Wiberg <[email protected]>
1 parent 32cdfdc commit d33b2bc

File tree

2 files changed

+273
-0
lines changed

2 files changed

+273
-0
lines changed

‎_posts/2024-03-11-advanced-containers.md

Lines changed: 269 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,269 @@
1+
---
2+
title: Advanced Container Networking
3+
author: troglobit
4+
date: 2024-03-11 14:11:50 +0100
5+
categories: [showcase]
6+
tags: [container, containers, docker podman]
7+
---
8+
9+
![Docker whale](/assets/img/docker.webp){: width="200" .right}
10+
11+
This is the second post about containers in Infix, the focus this time
12+
is on more advanced network setups. In particular, containers with more
13+
than one interface, and an Infix network configuration that includes
14+
both VETH pairs and bridges with LLDP forwarding.
15+
16+
See the [first post][1] for a background and networking basics.
17+
18+
> This post assumes knowledge and familiarity with the [Infix Network
19+
> Operating System](https://kernelkit.github.io/). Ensure you have
20+
> either a network connection or console access to your Infix system and
21+
> can log in to it using SSH. Recommended reading includes the
22+
> [networking documentation][0].
23+
{: .prompt-info }
24+
25+
26+
----
27+
28+
## Introduction
29+
30+
The setup below aims to show how a container can have multiple network
31+
connections. Useful for applications where critical process data should
32+
be isolated to prevent external access.
33+
34+
![](/assets/img/container-two-bridges.svg)
35+
_**Figure 1:** Container with access to two physically separated networks._
36+
37+
We use two bridges for this example, we could easily have used a single
38+
VLAN filtering bridge instead, but as a twist we want the left-most
39+
bridge to forward LLDP[^1] traffic, and in that case it is better to use
40+
two bridges instead.
41+
42+
> Depending on the underlying (switch) hardware, there are limitations
43+
> to consider. For example, a system with Marvel SOHO switch fabric,
44+
> using the Linux kernel DSA driver, is limited to one regular bridge
45+
> and multiple VLAN filtering bridges. Our example with two non-VLAN
46+
> filtering bridges means that they share the same MAC database. In
47+
> most use-case this is not a problem, but it is worth mentioning for
48+
> networks where the same MAC address may appear in both bridges. {:
49+
> .prompt-info }
50+
51+
## System Container
52+
53+
For this case study to make sense we need a more competent container
54+
image than the simple web server used in the first blog post. To that
55+
end we introduce [curiOS][2], a small container operating system built
56+
on the same stable [Buildroot][3] foundation as Infix. Specifically,
57+
we'll be using the *system container* build, available as `docker pull`
58+
from [GHCR][4] and as an [OCI tarball][5].
59+
60+
> Infix supports bundling OCI tarballs in the Infix image itself. This
61+
> can significantly simplify initial deployment of appliances running
62+
> Infix with customer containers.
63+
{: .prompt-tip }
64+
65+
66+
## Configuration
67+
68+
Let's start by setting up the first bridge, `br0`:
69+
70+
```console
71+
admin@infix:/> configure
72+
admin@infix:/config/> edit interface br0
73+
admin@infix:/config/interface/br0/> set bridge ieee-group-forward lldp
74+
admin@infix:/config/interface/br0/> end
75+
admin@infix:/config/> set interface eth1 bridge-port bridge br0
76+
admin@infix:/config/> set interface eth2 bridge-port bridge br0
77+
admin@infix:/config/> set interface eth3 bridge-port bridge br0
78+
admin@infix:/config/> leave
79+
```
80+
81+
The second bridge is similar:
82+
83+
```console
84+
admin@infix:/> configure
85+
admin@infix:/config/> edit interface br1
86+
admin@infix:/config/interface/br1/> end
87+
admin@infix:/config/> set interface eth4 bridge-port bridge br1
88+
admin@infix:/config/> set interface eth5 bridge-port bridge br1
89+
admin@infix:/config/> set interface eth6 bridge-port bridge br1
90+
admin@infix:/config/> leave
91+
```
92+
93+
Now, for the container we need to add two VETH pairs and attach one end
94+
of each to each of the bridges. We use the recommended nomenclature of
95+
Infix to skip configuration steps and save time:
96+
97+
```console
98+
admin@infix:/> configure
99+
admin@infix:/config/> edit interface veth0a
100+
admin@infix:/config/interface/veth0a/> set veth peer veth0b
101+
admin@infix:/config/interface/veth0a/> end
102+
admin@infix:/config/> edit interface veth1a
103+
admin@infix:/config/interface/veth1a/> set veth peer veth1b
104+
admin@infix:/config/interface/veth1a/> end
105+
admin@infix:/config/> set interface veth0b bridge-port bridge br0
106+
admin@infix:/config/> set interface veth1b bridge-port bridge br1
107+
admin@infix:/config/> leave
108+
```
109+
110+
Initial network setup is now complete (no IP addresses yet) and end
111+
devices connected to ports in both bridges can now communicate with
112+
each other, but not between bridges.
113+
114+
Time for IP networking!
115+
116+
> You could hold off on this until after you've created the container,
117+
> but in that case the addresses for `veth0a` and `veth1a` will not be
118+
> set until you manually restart the container. In this documented
119+
> order things will work from the first try.
120+
{: .prompt-info }
121+
122+
```console
123+
admin@infix:/> configure
124+
admin@infix:/config/> edit interface br0
125+
admin@infix:/config/interface/br0/> set ipv4 address 10.0.1.1 prefix-length 24
126+
admin@infix:/config/interface/br0/> end
127+
admin@infix:/config/> edit interface br1
128+
admin@infix:/config/interface/br1/> set ipv4 address 192.168.1.1 prefix-length 24
129+
admin@infix:/config/interface/br1/> end
130+
admin@infix:/config/> edit interface veth0a
131+
admin@infix:/config/interface/veth0a/> set ipv4 address 10.0.1.2 prefix-length 24
132+
admin@infix:/config/interface/veth0a/> end
133+
admin@infix:/config/> edit interface veth1a
134+
admin@infix:/config/interface/veth1a/> set ipv4 address 192.168.1.2 prefix-length 24
135+
admin@infix:/config/interface/veth1a/> leave
136+
```
137+
138+
You can verify the setup with the following CLI command.
139+
140+
```console
141+
admin@infix:/> show interfaces
142+
INTERFACE PROTOCOL STATE DATA
143+
lo ethernet UP 00:00:00:00:00:00
144+
ipv4 127.0.0.1/8 (static)
145+
ipv6 ::1/128 (static)
146+
br0 bridge
147+
│ ipv4 10.0.1.1/24 (static)
148+
├ eth1 bridge FORWARDING
149+
├ eth2 bridge FORWARDING
150+
├ eth3 bridge FORWARDING
151+
└ veth0b bridge FORWARDING
152+
br1 bridge
153+
│ ipv4 192.168.1.1/24 (static)
154+
├ eth4 bridge FORWARDING
155+
├ eth5 bridge FORWARDING
156+
├ eth6 bridge FORWARDING
157+
└ veth1b bridge FORWARDING
158+
eth0 ethernet UP 02:00:00:00:00:00
159+
ipv4 169.254.1.3/16 (random)
160+
ipv6 fe80::ff:fe00:0/64 (link-layer)
161+
eth7 ethernet UP 02:00:00:00:00:07
162+
ipv6 fe80::ff:fe00:7/64 (link-layer)
163+
eth8 ethernet UP 02:00:00:00:00:08
164+
ipv6 fe80::ff:fe00:8/64 (link-layer)
165+
eth9 ethernet UP 02:00:00:00:00:09
166+
ipv6 fe80::ff:fe00:9/64 (link-layer)
167+
veth0a ethernet
168+
ipv4 10.0.1.2/24 (static)
169+
veth0b ethernet UP 42:c0:a2:f1:41:fa
170+
veth1a ethernet
171+
ipv4 192.168.1.2/24 (static)
172+
veth1b ethernet UP da:d5:e8:0d:0b:55
173+
```
174+
175+
Finally, the container configuration. We start by marking the access
176+
port ends of the VETH pairs as container networks, then we can refer
177+
to them from the container.
178+
179+
```console
180+
admin@infix:/> configure
181+
admin@infix:/config/> set interface veth0a container-network
182+
admin@infix:/config/> set interface veth1a container-network
183+
admin@infix:/config> edit container system
184+
admin@infix:/config/container/system/> set image docker://ghcr.io/kernelkit/curios:24.02.0
185+
admin@infix:/config/container/system/> set hostname sys101
186+
admin@infix:/config/container/system/> edit network
187+
admin@infix:/config/container/system/network/> set interface veth0a
188+
admin@infix:/config/container/system/network/> set interface veth1a
189+
admin@infix:/config/container/system/network/> leave
190+
```
191+
192+
> We don't have to `leave` after each of the above sections, we could
193+
> just as easily kept going all through the new configuration.
194+
{: .prompt-info }
195+
196+
197+
## The Result
198+
199+
We should now have a running container that you can connect to and
200+
inspect the results:
201+
202+
```console
203+
admin@infix:/> show container
204+
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
205+
ccb2338b9f33 ghcr.io/kernelkit/curios:24.02.0 11 seconds ago Up 10 seconds system
206+
admin@infix:/> container shell system
207+
root@sys101:/# ifconfig
208+
lo Link encap:Local Loopback
209+
inet addr:127.0.0.1 Mask:255.0.0.0
210+
inet6 addr: ::1/128 Scope:Host
211+
UP LOOPBACK RUNNING MTU:65536 Metric:1
212+
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
213+
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
214+
collisions:0 txqueuelen:1000
215+
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
216+
217+
eth0 Link encap:Ethernet HWaddr D2:A3:70:0D:50:00
218+
inet addr:10.0.1.2 Bcast:10.0.1.255 Mask:255.255.255.0
219+
inet6 addr: fe80::d0a3:70ff:fe0d:5000/64 Scope:Link
220+
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
221+
RX packets:63 errors:0 dropped:9 overruns:0 frame:0
222+
TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
223+
collisions:0 txqueuelen:1000
224+
RX bytes:12867 (12.5 KiB) TX bytes:3064 (2.9 KiB)
225+
226+
eth1 Link encap:Ethernet HWaddr E6:E8:40:2F:19:F0
227+
inet addr:192.168.1.2 Bcast:192.168.1.255 Mask:255.255.255.0
228+
inet6 addr: fe80::e4e8:40ff:fe2f:19f0/64 Scope:Link
229+
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
230+
RX packets:41 errors:0 dropped:3 overruns:0 frame:0
231+
TX packets:19 errors:0 dropped:0 overruns:0 carrier:0
232+
collisions:0 txqueuelen:1000
233+
RX bytes:5845 (5.7 KiB) TX bytes:3064 (2.9 KiB)
234+
235+
root@sys101:~$ exit
236+
admin@infix:/>
237+
```
238+
239+
The two interfaces, `eth0` and `eth1`, correspond to the access end of
240+
the VETH pairs, `veth0a` and `veth1a`, respectively. The order in which
241+
they are added to the container configuration determines their names
242+
inside the container. This can of course be controlled from the
243+
[network specific options][6] in the container configurations.
244+
245+
246+
## Fin
247+
248+
That's the end of the second post about containers in Infix. As usual,
249+
remember to
250+
251+
```console
252+
admin@infix:/> copy running-config startup-config
253+
```
254+
255+
Take care! <3
256+
257+
----
258+
259+
[^1]: A standards compliant bridge by default does not forward IEEE
260+
reserved link-local multicast groups, in the 01:80:C2:00:00:0X
261+
range. The Infix bridge supports forwarding select protocols.
262+
263+
[0]: https://github.com/kernelkit/infix/blob/main/doc/networking.md
264+
[1]: /posts/containers/
265+
[2]: https://github.com/kernelkit/curiOS/
266+
[3]: https://buildroot.org/
267+
[4]: https://github.com/kernelkit/curiOS/pkgs/container/curios/185490653?tag=24.02.0
268+
[5]: https://github.com/kernelkit/curiOS/releases/tag/v24.02.0
269+
[6]: https://github.com/kernelkit/infix/blob/main/doc/container.md#networking-and-containers

‎assets/img/container-two-bridges.svg

Lines changed: 4 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)