|
| 1 | +# BGP EVPN Multihoming with IPv4/IPv6 VTEPs and EBGP Numbered Peering |
| 2 | + |
| 3 | +## Overview |
| 4 | + |
| 5 | +This test validates BGP EVPN Multi-Homing functionality using: |
| 6 | +- **IPv6 VTEP addresses** for VXLAN tunnel endpoints (IPv4 support planned) |
| 7 | +- **EBGP numbered peering** with explicit IPv6 neighbor addresses |
| 8 | +- **Unified FRR configuration** using `service integrated-vtysh-config` |
| 9 | +- **PIM6** for IPv6 multicast BUM traffic handling |
| 10 | + |
| 11 | +## Topology |
| 12 | + |
| 13 | +``` |
| 14 | + +--------+ +--------+ |
| 15 | + | spine1 | | spine2 | |
| 16 | + | AS6500 | | AS6500 | |
| 17 | + +---+----+ +----+---+ |
| 18 | + | \/ | |
| 19 | + | /\ | |
| 20 | + +---+----+ +----+---+ |
| 21 | + | leaf1 | | leaf2 | |
| 22 | + | AS6510 | | AS6510 | |
| 23 | + +---+----+ +----+---+ |
| 24 | + | \/ | |
| 25 | + | /\ | |
| 26 | + +--------------+--++--------+---+--------------+ |
| 27 | + | | | | |
| 28 | + +--+---+ +----+---+ +----+---+ +----+---+ |
| 29 | + |torm11| |torm12 | |torm21 | |torm22 | |
| 30 | + |AS6500| |AS6500 | |AS6500 | |AS6500 | |
| 31 | + | 2 | | 3 | | 4 | | 5 | |
| 32 | + +--+---+ +----+---+ +----+---+ +----+---+ |
| 33 | + | \ / | | \ / | |
| 34 | + | \ / | | \ / | |
| 35 | + | \ / | | \ / | |
| 36 | + +--+---+ +----+---+ | +--+---+ +------+---+ | |
| 37 | + |hostd11 |hostd12 | |hostd21| |hostd22 | |
| 38 | + +--------+----------+ +--------+-----------+ |
| 39 | + (Rack 1) (Rack 2) |
| 40 | +``` |
| 41 | + |
| 42 | +## Key Features |
| 43 | + |
| 44 | +### 1. IPv6 VTEP Addressing |
| 45 | + |
| 46 | +All VXLAN tunnel endpoints currently use IPv6 addresses: |
| 47 | +- **torm11**: `2001:db8:100::15/128` |
| 48 | +- **torm12**: `2001:db8:100::16/128` |
| 49 | +- **torm21**: `2001:db8:100::17/128` |
| 50 | +- **torm22**: `2001:db8:100::18/128` |
| 51 | + |
| 52 | +### 2. IPv6 Point-to-Point Links (/127) |
| 53 | + |
| 54 | +All inter-router links use `/127` addressing for efficient IPv6 P2P: |
| 55 | +- **spine1 ↔ leaf1**: `2001:db8:50::0/127` and `::1/127` |
| 56 | +- **spine1 ↔ leaf2**: `2001:db8:51::0/127` and `::1/127` |
| 57 | +- **spine2 ↔ leaf1**: `2001:db8:60::0/127` and `::1/127` |
| 58 | +- **spine2 ↔ leaf2**: `2001:db8:61::0/127` and `::1/127` |
| 59 | +- **leaf1 ↔ torm11**: `2001:db8:1::0/127` and `::1/127` |
| 60 | +- **leaf1 ↔ torm12**: `2001:db8:2::0/127` and `::1/127` |
| 61 | +- And so on... |
| 62 | + |
| 63 | +### 3. AS Number Scheme (EBGP Numbered) |
| 64 | + |
| 65 | +- **Spine layer**: AS 65001 |
| 66 | +- **Leaf layer**: AS 65101 |
| 67 | +- **TOR Rack 1**: AS 65002 (torm11), AS 65003 (torm12) |
| 68 | +- **TOR Rack 2**: AS 65004 (torm21), AS 65005 (torm22) |
| 69 | + |
| 70 | +### 4. BGP Configuration |
| 71 | + |
| 72 | +All BGP sessions use: |
| 73 | +- IPv6 neighbor addresses (EBGP numbered) |
| 74 | +- `capability extended-nexthop` for IPv4 over IPv6 BGP sessions |
| 75 | +- `no bgp default ipv4-unicast` (IPv6-only underlay for now) |
| 76 | +- L2VPN EVPN address family for EVPN routes |
| 77 | + |
| 78 | +### 5. PIM6 for IPv6 Multicast |
| 79 | + |
| 80 | +TOR routers use PIM6 for handling BUM (Broadcast, Unknown Unicast, Multicast) traffic: |
| 81 | +- IPv6 multicast group: `ff0e::100` |
| 82 | +- PIM6 enabled on uplink interfaces |
| 83 | +- Multicast termination device: `ipmr-lo` |
| 84 | + |
| 85 | +### 6. EVPN Multihoming |
| 86 | + |
| 87 | +Each rack has: |
| 88 | +- Two TORs acting as PEs (Provider Edge) |
| 89 | +- Dual-attached hosts with LACP bonds |
| 90 | +- ES-ID (Ethernet Segment Identifier) per host |
| 91 | +- ES system MAC for MH coordination |
| 92 | + |
| 93 | +**Rack 1 ES Configuration:** |
| 94 | +- ES system MAC: `44:38:39:ff:ff:01` |
| 95 | +- hostd11 ES-ID: 1 |
| 96 | +- hostd12 ES-ID: 2 |
| 97 | + |
| 98 | +**Rack 2 ES Configuration:** |
| 99 | +- ES system MAC: `44:38:39:ff:ff:02` |
| 100 | +- hostd21 ES-ID: 1 |
| 101 | +- hostd22 ES-ID: 2 |
| 102 | + |
| 103 | +## Configuration Structure |
| 104 | + |
| 105 | +### Unified FRR Configuration |
| 106 | + |
| 107 | +Each router uses a single `frr.conf` file with: |
| 108 | +``` |
| 109 | +frr defaults datacenter |
| 110 | +service integrated-vtysh-config |
| 111 | +hostname <router-name> |
| 112 | +! |
| 113 | +# Interface configurations |
| 114 | +# Routing protocol configurations |
| 115 | +! |
| 116 | +end |
| 117 | +``` |
| 118 | + |
| 119 | +This approach: |
| 120 | +- Simplifies configuration management |
| 121 | +- Matches production deployments |
| 122 | +- Uses modern FRR best practices |
| 123 | + |
| 124 | +## IPv6 Addressing Plan |
| 125 | + |
| 126 | +### Loopback Addresses (VTEP) |
| 127 | +- `2001:db8:100::/64` - VTEP loopback addresses |
| 128 | + - ::13 - spine1 |
| 129 | + - ::14 - spine2 |
| 130 | + - ::15 - torm11 (VTEP) |
| 131 | + - ::16 - torm12 (VTEP) |
| 132 | + - ::17 - torm21 (VTEP) |
| 133 | + - ::18 - torm22 (VTEP) |
| 134 | + |
| 135 | +### Spine Loopbacks |
| 136 | +- `2001:db8:200::/64` - Leaf/spine additional loopbacks |
| 137 | + - ::13 - leaf1 |
| 138 | + - ::14 - leaf2 |
| 139 | + |
| 140 | +### Point-to-Point Links |
| 141 | +- `2001:db8:1::/127` through `2001:db8:8::/127` - TOR to Leaf links |
| 142 | +- `2001:db8:50::/127`, `2001:db8:51::/127` - Spine1 to Leaf links |
| 143 | +- `2001:db8:60::/127`, `2001:db8:61::/127` - Spine2 to Leaf links |
| 144 | + |
| 145 | +### SVI (Switched Virtual Interface) |
| 146 | +- `2001:db8:45::/64` - VLAN 1000 SVI subnet |
| 147 | + - ::1 - Anycast gateway |
| 148 | + - ::2-::5 - TOR SVI addresses |
| 149 | + - ::11, ::12, ::21, ::22 - Host addresses |
| 150 | + |
| 151 | +## Test Coverage |
| 152 | + |
| 153 | +The test suite validates: |
| 154 | + |
| 155 | +1. **ES Peering** (`test_evpn_es`) |
| 156 | + - Local ES peer discovery via Type-1 EAD routes |
| 157 | + - Remote ES PE list correctness |
| 158 | + |
| 159 | +2. **EAD Route Updates** (`test_evpn_ead_update`) |
| 160 | + - Link flap handling |
| 161 | + - EAD route withdrawal and re-advertisement |
| 162 | + |
| 163 | +3. **MAC Learning** (`test_evpn_mac`) |
| 164 | + - Local MAC sync between PEs |
| 165 | + - Remote MAC installation |
| 166 | + |
| 167 | +4. **DF Election** (`test_evpn_df`) |
| 168 | + - Designated Forwarder role assignment |
| 169 | + - DF preference changes |
| 170 | + |
| 171 | +5. **Uplink Tracking** (`test_evpn_uplink_tracking`) |
| 172 | + - Access port protodown on uplink failure |
| 173 | + - Recovery on uplink restoration |
| 174 | + |
| 175 | +## Running the Tests |
| 176 | + |
| 177 | +```bash |
| 178 | +cd /work/penta-01/chirag/docker-home/tree/up/fr/frr/tests/topotests/bgp_evpn_mh_v4_v6_num |
| 179 | + |
| 180 | +# Run all tests (fixture will run IPv6; IPv4 is currently skipped) |
| 181 | +pytest test_evpn_mh_v4_v6_numbered.py -v |
| 182 | + |
| 183 | +# Run specific test |
| 184 | +pytest test_evpn_mh_v4_v6_numbered.py::test_evpn_es -v |
| 185 | + |
| 186 | +# Run with debug output |
| 187 | +pytest test_evpn_mh_v4_v6_numbered.py -v -s |
| 188 | +``` |
| 189 | + |
| 190 | +## Requirements |
| 191 | + |
| 192 | +- **FRR**: 8.1 or later (with mgmtd support) |
| 193 | +- **Kernel**: 4.19 or later (for MH support) |
| 194 | +- **IPv6**: Fully enabled with forwarding |
| 195 | +- **Modules**: VXLAN, bridge, bonding, PIM6 |
| 196 | + |
| 197 | +## Key Differences from Original test_evpn_mh |
| 198 | + |
| 199 | +1. **IPv6 VTEP first**: Uses IPv6 for VXLAN tunnel endpoints (IPv4 VTEPs can be added later) |
| 200 | +2. **EBGP Numbered**: Explicit IPv6 neighbor addresses instead of unnumbered |
| 201 | +3. **Unified Config**: Single `frr.conf` per router instead of separate daemon configs |
| 202 | +4. **PIM6**: Uses IPv6 PIM instead of IPv4 PIM |
| 203 | +5. **IPv6 Multicast**: Uses `ff0e::100` instead of `239.1.1.100` |
| 204 | +6. **Pytest Fixture**: `tgen_and_ip_version` parametrizes the test for `"ipv4"` / `"ipv6"` underlay (IPv4 currently skipped) |
| 205 | +7. **Critical Fixes**: Includes IPv6 forwarding and DAD configuration before daemon startup |
| 206 | + |
| 207 | +## Known Issues and Workarounds |
| 208 | + |
| 209 | +### IPv6 DAD (Duplicate Address Detection) |
| 210 | + |
| 211 | +DAD is disabled globally and per-interface to prevent address configuration delays: |
| 212 | +```python |
| 213 | +router.run("sysctl -w net.ipv6.conf.all.accept_dad=0") |
| 214 | +router.run("sysctl -w net.ipv6.conf.all.dad_transmits=0") |
| 215 | +``` |
| 216 | + |
| 217 | +### mgmtd Requirement |
| 218 | + |
| 219 | +Modern FRR requires mgmtd to be loaded before other daemons: |
| 220 | +```python |
| 221 | +router.load_config(TopoRouter.RD_MGMTD, "") |
| 222 | +``` |
| 223 | + |
| 224 | +### IPv6 Forwarding |
| 225 | + |
| 226 | +Must be enabled BEFORE starting FRR daemons: |
| 227 | +```python |
| 228 | +router.run("sysctl -w net.ipv6.conf.all.forwarding=1") |
| 229 | +``` |
| 230 | + |
| 231 | +## Debugging |
| 232 | + |
| 233 | +### View BGP EVPN ES Information |
| 234 | +```bash |
| 235 | +vtysh -c "show bgp l2vpn evpn es" |
| 236 | +vtysh -c "show bgp l2vpn evpn es detail" |
| 237 | +``` |
| 238 | + |
| 239 | +### View MAC Addresses |
| 240 | +```bash |
| 241 | +vtysh -c "show evpn mac vni 1000" |
| 242 | +``` |
| 243 | + |
| 244 | +### View BGP IPv6 Neighbors |
| 245 | +```bash |
| 246 | +vtysh -c "show bgp ipv6 unicast summary" |
| 247 | +``` |
| 248 | + |
| 249 | +### View EVPN Routes |
| 250 | +```bash |
| 251 | +vtysh -c "show bgp l2vpn evpn route" |
| 252 | +``` |
| 253 | + |
| 254 | +### View PIM6 Status |
| 255 | +```bash |
| 256 | +vtysh -c "show ipv6 pim interface" |
| 257 | +vtysh -c "show ipv6 mroute" |
| 258 | +``` |
| 259 | + |
| 260 | +## References |
| 261 | + |
| 262 | +- Original test: `tests/topotests/bgp_evpn_mh/` |
| 263 | +- IPv6 VTEP reference: `tests/topotests/bgp_evpn_three_tier_clos_topo1/` |
| 264 | +- FRR EVPN MH documentation: https://docs.frrouting.org/en/latest/evpn.html |
| 265 | + |
| 266 | +## Authors |
| 267 | + |
| 268 | +- Original test: Anuradha Karuppiah (Cumulus Networks) |
| 269 | +- IPv4/IPv6 VTEP and EBGP numbered adaptation: Chirag Shah (Nvidia), 2025 |
| 270 | + |
| 271 | +## License |
| 272 | + |
| 273 | +SPDX-License-Identifier: ISC |
0 commit comments