Integration lab: building an ISP core
Turn an address plan into a working IPv6 provider core with an IGP, iBGP route reflection and eBGP policy. Apply the operational rules from this level: discard routes, next-hop-self, RFC 7454 filters, RPKI, roles and MTU. Write and run an acceptance test that proves the core works and fails safely.
Take the quiz · pass mark 70% · 9 questions
Explanation
This lesson brings Level 4 together. You build the core of ISP-A from the course lab plan, then prove it works with an acceptance test: a written list of checks with expected results, run before the network carries customers and after every change. The Level 4 exam practical is a smaller version of the same task.
The design
Addressing (lab plan, ISP-A, AS64496, 2001:db8:1000::/36):
| Block | Use |
|---|---|
2001:db8:1000::/48 | Infrastructure |
2001:db8:1000::/64 | Loopbacks, 2001:db8:1000::n/128 for router n |
2001:db8:1000:1LLL::/64 | Link LLL (hex): one /64 reserved per link, ::a/127 and ::b/127 used |
2001:db8:1100::/40 | Prefix-delegation pool for residential customers (/56 each) |
2001:db8:1200::/40 onwards | Business customers (/48 each) |
2001:db8:f000::/48 | Simulated Internet and IXP (shared, not ISP-A's) |
Routing design:
- IGP (OSPFv3, RFC 5340, or IS-IS): loopbacks and links only. No customer routes, no BGP routes. Small, fast to converge.
- iBGP: every router peers from its loopback with two route reflectors (RFC 4456) (
a-rr1,a-rr2, same cluster ID). Border routers set next-hop-self. Customer and Internet routes live here. - Origination: the /36 is originated on both border routers with a discard route. BNG and PoP routers originate their customer blocks (the PD pool as a /40 with its own discard route) into iBGP.
- eBGP policy on every external session: RFC 7454 filters (bogons with the lab exception, longer than /48, own space, default), drop RPKI Invalid (RFC 6811), maximum-prefix, an explicit outbound prefix-list, and BGP roles (RFC 9234,
local-role) matching the relationship. - Communities on ingress record where a route came from (for example
64496:1000transit,64496:2000peer,64496:3000customer), and outbound policy uses them, so only customer routes and ISP-A's own space go to peers and transit. - MTU: core links at 9000, customer-facing at 1500, PTB never filtered.
Writing an acceptance test
A good acceptance test is specific, repeatable and includes failure cases. Each line states a command, the expected result and why it matters:
| # | Check | Expected |
|---|---|---|
| 1 | IGP neighbours on every router | FULL / Up on every core link |
| 2 | Loopback reachability (ping every loopback from every router) | 100% |
| 3 | iBGP sessions | Every client Established with both RRs |
| 4 | eBGP sessions | Established, policy applied (no (Policy)), roles agreed |
| 5 | Routes from ISP-B | ISP-B /36 present on every router with next hop = a border loopback |
| 6 | Route filters | Test bogon, too-long, own-space and RPKI Invalid routes are absent |
| 7 | Outbound announcements | ISP-B receives only the /36 and customer routes, nothing learned from other peers |
| 8 | Unused-space test | Traffic to an unused address in the /36 is dropped at the border, not looped |
| 9 | RR failure | Shut a-rr1: all routes remain via a-rr2 |
| 10 | Link failure | Shut a core link: IGP reconverges, BGP sessions stay up (loopback peering) |
| 11 | MTU | 1500-byte packets from a customer to the Internet pass; PTB reaches the source when a 1400 link is inserted |
Automate what you can. FRR's JSON output makes checks scriptable:
#!/bin/sh
# check 3 and 4: every BGP peer Established (field names can differ slightly between FRR versions)
for r in a-br1 a-br2 a-rr1 a-rr2 a-pe1 a-pe2; do
docker exec clab-isp-$r vtysh -c 'show bgp ipv6 unicast summary json' |
jq -r --arg r "$r" '.peers | to_entries[] | "\($r) \(.key) \(.value.state)"'
done | awk '$3 != "Established" {bad=1; print "FAIL", $0} END {exit bad}'
Record the outputs with a date: they are the baseline you compare against when something breaks later.
ictNetworks scenario
The same design runs ictNetworks' production core in AS65550: OSPFv3 area 0 for infrastructure (3fff:a00::/48), the core routers as route reflectors, 3fff:a00::/32 originated with discard routes on both border routers, residential BNG /40s from 3fff:a00:8000::/33 and business /48s from 3fff:a00:2000::/36 in iBGP, and the RFC 7454, RPKI and roles policy from L4.05 on every external session. Its change process requires the acceptance test to pass before and after every maintenance window.
Drawing diagram…
Worked examples
Example 1: The PD pool
The residential pool 2001:db8:1100::/40 holds 2^(56 − 40) = 65,536 customer /56s. The 16-bit /56 index sits in bits 40 to 55: its high byte fills the low byte of the third group and its low byte fills the high byte of the fourth group. Customer 300 has index 299 = 0x012b, so the third group is 1101 and the fourth 2b00: 2001:db8:1101:2b00::/56.
Example 2: The smallest aggregate
The infrastructure /48 (2001:db8:1000::/48) and the PD pool (2001:db8:1100::/40) share only the first 39 bits, so the smallest covering prefix is 2001:db8:1000::/39. ISP-A still announces its whole allocation, the /36, with one discard route, and never the internal pieces.
Example 3: Session count
Six ISP-A routers with two RRs: 4 clients × 2 + 1 = 9 iBGP sessions, against 15 for a full mesh.
Lab
Goal: Build ISP-A's core from the design and pass the acceptance test.
You need: the Level 4 lab pack (containerlab with 8–10 FRR nodes: a-br1, a-br2, a-rr1, a-rr2, a-pe1, a-pe2, b-br1 for ISP-B, inet for the simulated Internet, an enterprise edge e-br1, and a Routinator container). Proxmox VMs with VyOS or MikroTik CHR can replace any FRR node.
- Write the address plan for every loopback and link (a table), then configure interfaces and the IGP. Run checks 1 and 2.
- Configure iBGP with the two RRs and next-hop-self, originate the /36 with discard routes, and the PD pool on
a-pe1. Run check 3. - Configure eBGP to
b-br1,inetande-br1with full policy: RFC 7454 filters, RPKI drop-invalid (SLURM test ROAs as in L4.05), maximum-prefix, explicit outbound lists, ingress communities and roles (provider towardse-br1, peer towardsb-br1, customer towardsinet).inetoriginates test routes: a bogon (the course ULAfdf1:a861:26b6::/48), a /56, ISP-A's own2001:db8:1000::/48, an RPKI Invalid route and normal routes. - Run checks 4 to 8 and record the outputs.
- Run the failure tests 9 to 11 and record convergence times.
- Automate at least checks 2 to 4 with a script and include it.
What to submit: the address table, all configurations, the completed acceptance test (command, expected, actual, pass/fail for each check), the script, and a short report of anything that failed first and how you fixed it.
Summary
- Keep the IGP for infrastructure only; carry customer and Internet routes in iBGP with route reflectors and next-hop-self.
- Originate aggregates with discard routes; never announce internal pieces.
- Apply the full external policy on every eBGP session: RFC 7454 filters, RPKI, maximum-prefix, outbound lists, communities and roles.
- An acceptance test is specific, repeatable and includes failure cases; automate it and keep the outputs as a baseline.
- Level 4 is complete. Level 5 covers IPv6 security.
Quiz imported · 9 questions · pass mark 70%. Lab questions are not auto-marked.