Skip to content

Commit f6dc2be

Browse files
ehsjoarJonas Arndt
ehsjoar
and
Jonas Arndt
authored
Add MetalLB BGP documentation (#819)
* First draft * Addressed review comments * Addressed review comments - II * Addressed review comments - III * Addressed review comments - IV --------- Co-authored-by: Jonas Arndt <[email protected]>
1 parent 3aa4bb5 commit f6dc2be

File tree

4 files changed

+254
-3
lines changed

4 files changed

+254
-3
lines changed

‎asciidoc/components/metallb.adoc

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,18 +28,21 @@ ____
2828

2929
== How does SUSE Edge use MetalLB?
3030

31-
SUSE Edge uses MetalLB in two key ways:
31+
SUSE Edge uses MetalLB in three key ways:
3232

3333
* As a Load Balancer Solution: MetalLB serves as the Load Balancer solution for bare-metal machines.
3434
* For an HA K3s/RKE2 Setup: MetalLB allows for load balancing the Kubernetes API using a Virtual IP address.
35+
* As an L3 BGP solution where MetalLB advertises routes to the service IPs to
36+
nearby routers.
3537

3638
[NOTE]
3739
====
3840
In order to be able to expose the API, the <<components-eco, Endpoint Copier Operator>> is used to keep in sync the K8s API endpoints from the `kubernetes` service to a `kubernetes-vip` LoadBalancer service.
3941
====
4042

4143
== Best practices
42-
Installation of MetalLB in L2 mode is described in <<guides-metallb-k3s>>.
44+
Installation of MetalLB in L2 mode is described in <<guides-metallb-k3s>> and
45+
for L3 mode in <<guides-metallb-k3s-l3>>.
4346

4447
A guide on installing MetalLB in front of the `kube-api-server` to achieve high-availability topology can be found in <<guides-metallb-kubernetes>>.
4548

‎asciidoc/edge-book/edge.adoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,8 @@ How-to guides and best practices
115115
116116
include::../guides/metallb-k3s.adoc[leveloffset=+1]
117117
118+
include::../guides/metallb-k3s_l3.adoc[leveloffset=+1]
119+
118120
include::../guides/metallb-kube-api.adoc[leveloffset=+1]
119121
120122
include::../guides/air-gapped-eib-deployments.adoc[leveloffset=+1]

‎asciidoc/guides/metallb-k3s.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ MetalLB is a load-balancer implementation for bare-metal Kubernetes clusters, us
1717

1818
In this guide, we demonstrate how to deploy MetalLB in layer 2 (L2) mode.
1919

20-
== Why use this method
20+
== Why use MetalLB
2121

2222
MetalLB is a compelling choice for load balancing in bare-metal Kubernetes clusters for several reasons:
2323

‎asciidoc/guides/metallb-k3s_l3.adoc

Lines changed: 246 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,246 @@
1+
[#guides-metallb-k3s-l3]
2+
= MetalLB on K3s (using Layer 3 Mode)
3+
:revdate: 2025-09-08
4+
:page-revdate: {revdate}
5+
:experimental:
6+
7+
ifdef::env-github[]
8+
:imagesdir: ../images/
9+
:tip-caption: :bulb:
10+
:note-caption: :information_source:
11+
:important-caption: :heavy_exclamation_mark:
12+
:caution-caption: :fire:
13+
:warning-caption: :warning:
14+
endif::[]
15+
16+
MetalLB is a load-balancer implementation for bare-metal Kubernetes clusters, using standard routing protocols.
17+
18+
In this guide, we demonstrate how to deploy MetalLB in layer 3 (L3) BGP mode.
19+
20+
== Why use MetalLB
21+
22+
MetalLB is a compelling choice for load balancing in bare-metal Kubernetes clusters for several reasons:
23+
24+
. Native Integration with Kubernetes: MetalLB seamlessly integrates with Kubernetes, making it easy to deploy and manage using familiar Kubernetes tools and practices.
25+
. Bare-Metal Compatibility: Unlike cloud-based load balancers, MetalLB is designed specifically for on-premises deployments where traditional load balancers might not be available or feasible.
26+
. Supports Multiple Protocols: MetalLB supports both Layer 2 and Layer 3 BGP (Border Gateway Protocol) modes, providing flexibility for different network architectures and requirements.
27+
. High Availability: By distributing load-balancing responsibilities across multiple nodes, MetalLB ensures high availability and reliability for your services.
28+
. Scalability: MetalLB can handle large-scale deployments, scaling alongside your Kubernetes cluster to meet increasing demand.
29+
30+
In layer 2 mode, one node assumes the responsibility of advertising a service to the local network. From the network’s perspective, it simply looks like that machine has multiple IP addresses assigned to its network interface.
31+
32+
The major advantage of the layer 2 mode is its universality: it works on any Ethernet network, with no special hardware required, not even fancy routers.
33+
34+
== MetalLB on K3s (using L3)
35+
36+
In this quick start, L3 mode is used.
37+
This means that we need to have neighboring router(s) with BGP capabilities within
38+
the network range.
39+
40+
== Prerequisites
41+
42+
* A K3s cluster where MetalLB is going to be deployed.
43+
* Router(s) on the network that support the BGP protocol.
44+
* A free IP address within the network range for the service. In this example
45+
`192.168.10.100`
46+
47+
[IMPORTANT]
48+
====
49+
You must make sure this IP address is unassigned.
50+
In a DHCP environment this address must not be part of the DHCP pool to avoid dual assignments.
51+
====
52+
53+
== Configuration to Advertise Service IP Addresses
54+
Out of the box BGP advertises a Service IP address to all the peers that are
55+
configured. These peers, which are usually routers, will receive a route for
56+
each Service IP address with a 32 bit network mask. In this examlple we will use
57+
an FRR based router and is on the same network as our cluster. We will then use
58+
MetalLB's BGP capability to advertise a service to that FRR based router.
59+
60+
== Deployment
61+
62+
We will be using the MetalLB Helm chart published as part of the SUSE Edge solution:
63+
64+
[,bash,subs="attributes"]
65+
----
66+
helm install \
67+
metallb oci://registry.suse.com/edge/charts/metallb \
68+
--namespace metallb-system \
69+
--create-namespace
70+
71+
while ! kubectl wait --for condition=ready -n metallb-system $(kubectl get\
72+
pods -n metallb-system -l app.kubernetes.io/component=controller -o name)\
73+
--timeout=10s; do
74+
sleep 2
75+
done
76+
----
77+
78+
== Configuration
79+
80+
. At this point, the installation is complete. Create an `IPAddressPool`:
81+
82+
[,bash]
83+
----
84+
cat <<-EOF | kubectl apply -f -
85+
apiVersion: metallb.io/v1beta1
86+
kind: IPAddressPool
87+
metadata:
88+
name: bgp-pool
89+
namespace: metallb-system
90+
labels:
91+
app: httpd
92+
spec:
93+
addresses:
94+
- 192.168.10.100/32
95+
autoAssign: true
96+
avoidBuggyIPs: false
97+
serviceAllocation:
98+
namespaces:
99+
- metallb-system
100+
priority: 100
101+
serviceSelectors:
102+
- matchExpressions:
103+
- key: serviceType
104+
operator: In
105+
values:
106+
- httpd
107+
EOF
108+
----
109+
110+
[start=2]
111+
. Configure a `BGPPeer`.
112+
113+
[NOTE]
114+
====
115+
The FRR router has ASN 1000 while our `BGPPeer` will have 1001. We can also see that the FRR Router has an IP
116+
address that is 192.168.3.140.
117+
====
118+
119+
[,bash]
120+
----
121+
cat <<-EOF | kubectl apply -f -
122+
apiVersion: metallb.io/v1beta2
123+
kind: BGPPeer
124+
metadata:
125+
namespace: metallb-system
126+
name: mypeertest
127+
spec:
128+
peerAddress: 192.168.3.140
129+
peerASN: 1000
130+
myASN: 1001
131+
routerID: 4.4.4.4
132+
EOF
133+
----
134+
135+
[start=3]
136+
. Create the BGPAdvertisement (L3):
137+
138+
[,bash]
139+
----
140+
cat <<-EOF | kubectl apply -f -
141+
apiVersion: metallb.io/v1beta1
142+
kind: BGPAdvertisement
143+
metadata:
144+
name: bgpadvertisement-test
145+
namespace: metallb-system
146+
spec:
147+
ipAddressPools:
148+
- bgp-pool
149+
EOF
150+
----
151+
152+
== Usage
153+
154+
. Create an example application with a service. In this case, IP address from the `IPAddressPool` is `192.168.10.100` for that service.
155+
156+
[,bash]
157+
----
158+
cat <<- EOF | kubectl apply -f -
159+
apiVersion: apps/v1
160+
kind: Deployment
161+
metadata:
162+
name: httpd-deployment
163+
namespace: metallb-system
164+
labels:
165+
app: httpd
166+
spec:
167+
replicas: 3
168+
selector:
169+
matchLabels:
170+
pod-label: httpd
171+
template:
172+
metadata:
173+
labels:
174+
pod-label: httpd
175+
spec:
176+
containers:
177+
- name: httpdcontainer
178+
image: image: docker.io/library/httpd:2.4
179+
ports:
180+
- containerPort: 80
181+
protocol: TCP
182+
restartPolicy: Always
183+
184+
---
185+
apiVersion: v1
186+
kind: Service
187+
metadata:
188+
name: http-service
189+
namespace: metallb-system
190+
labels:
191+
serviceType: httpd
192+
spec:
193+
selector:
194+
pod-label: httpd
195+
type: LoadBalancer
196+
ports:
197+
- protocol: TCP
198+
port: 8080
199+
name: 8080-tcp
200+
targetPort: 80
201+
EOF
202+
----
203+
204+
205+
[start=2]
206+
. To verify, log onto the FRR Router to can see the routes created from the BGP advertisement.
207+
208+
[,console]
209+
----
210+
42178089cba5# show ip bgp all
211+
212+
For address family: IPv4 Unicast
213+
BGP table version is 3, local router ID is 2.2.2.2, vrf id 0
214+
Default local pref 100, local AS 1000
215+
Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,
216+
i internal, r RIB-failure, S Stale, R Removed
217+
Nexthop codes: @NNN nexthop's vrf id, < announce-nh-self
218+
Origin codes: i - IGP, e - EGP, ? - incomplete
219+
RPKI validation codes: V valid, I invalid, N Not found
220+
221+
Network Next Hop Metric LocPrf Weight Path
222+
* i172.16.0.0/24 1.1.1.1 0 100 0 i
223+
*> 0.0.0.0 0 32768 i
224+
* i172.17.0.0/24 3.3.3.3 0 100 0 i
225+
*> 0.0.0.0 0 32768 i
226+
*= 192.168.10.100/32
227+
192.168.3.162 0 1001 i
228+
*= 192.168.3.163 0 1001 i
229+
*> 192.168.3.161 0 1001 i
230+
231+
Displayed 3 routes and 7 total paths
232+
kubectl get svc -n hello-kubernetes
233+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
234+
hello-kubernetes LoadBalancer 10.43.127.75 192.168.122.11 80:31461/TCP 8s
235+
----
236+
237+
[start=3]
238+
. If this router is the default gateway for your network, you can run the `curl` command from a box on that network to verify that they can reach the httpd sample app
239+
240+
[,console]
241+
----
242+
# curl http://192.168.10.100:8080
243+
<html><body><h1>It works!</h1></body></html>
244+
#
245+
----
246+

0 commit comments

Comments
 (0)