I’m using my IOU topology version 2 for this set up. It’s configured like so:
I’ve configured everything for a fairly simple L3VPN setup. As I stated above, I’m using OSPF as the Provider IGP over the frame network (with a broadcast network type to start). I’m also running OSPF for the PE-CE protocol on both PE routers. You can find the full starting configs for all the routers here so that you can peruse them at your leisure.
So, what is our problem here exactly? Well the problem is on the broadcast type network when the hub and spokes are all in the same subnet. OSPF assumes here that the next hop is the advertising router. This is all well and fine for regular IP routing over frame relay (as long as your frame maps are configured correctly). Allow me to show you what I’m referring to. We’ll look at this from the VRF_D perspective, trying to reach the CE in VRF_C. Here’s the routing information on R6 for this:
R6#sh bgp vpnv4 uni all 5.5.5.5
BGP routing table entry for 4:4:5.5.5.5/32, version 50
Paths: (1 available, best #1, table VRF_D)
Not advertised to any peer
Local
3.3.3.3 (metric 65) from 2.2.2.2 (2.2.2.2)
Origin incomplete, metric 65, localpref 100, valid, internal, best
Extended Community: RT:4:4 OSPF DOMAIN ID:0x0005:0x000000030200
OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:33.33.33.33:27126
Originator: 3.3.3.3, Cluster list: 2.2.2.2
mpls labels in/out nolabel/18
Nothing out of the ordinary here. To reach 5.5.5.5 our next hop is 3.3.3.3 (with a VPN label of 18) which is the PE router on the other side of the MPLS network. Let’s check our routing information for 3.3.3.3:
R6#sh ip ro 3.3.3.3
Routing entry for 3.3.3.3/32
Known via "ospf 1", distance 110, metric 65, type intra area
Last update from 10.23.67.3 on Serial1/0, 00:05:57 ago
Routing Descriptor Blocks:
* 10.23.67.3, from 3.3.3.3, 00:05:57 ago, via Serial1/0
Route metric is 65, traffic share count is 1
Again, nothing odd here. But do notice, that whereas in the BGP route info we could see our advertising router was 2.2.2.2 (because it is a route reflector) here our advertising router is 3.3.3.3. This is due to using the OSPF broadcast network type. With this network type OSPF assumes that both R6 and R3 are part of the same layer 3 broadcast domain. We told it to assume this when we changed the default network type from non-broadcast on the physical interfaces used for Frame Relay. This doesn’t break Frame Relay because we configured frame maps to compensate for it:
R6#sh frame map
Serial1/0 (up): ip 10.23.67.2 dlci 602(0x25A,0x94A0), static,
broadcast,
CISCO, status defined, active
Serial1/0 (up): ip 10.23.67.3 dlci 602(0x25A,0x94A0), static,
CISCO, status defined, active
Serial1/0 (up): ip 10.23.67.7 dlci 602(0x25A,0x94A0), static,
CISCO, status defined, active
Serial1/0 (up): ip 10.23.67.6 dlci 602(0x25A,0x94A0), static,
CISCO, status defined, active
You can see we have manually set up a frame map for 10.23.67.3 via DLCI 602. Because of this we “bounce” traffic for R3 off of R2. Life is good.
R6#ping 10.23.67.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.23.67.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/33/36 ms
R6#ping 3.3.3.3
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 3.3.3.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 28/32/36 ms
Now our problem comes in when add MPLS into the mix. As I hope you know (if not head on back to the MPLS books) each router will take all the entries in its FIB, assign labels to them, and then advertise those labels to its neighbours. What happens here is that R6 knows it needs to send the traffic to 3.3.3.3, so let’s look at our LFIB for 3.3.3.3.
R6#sh mpls forwarding-table 3.3.3.3 detail
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
17 No Label 3.3.3.3/32 0 Se1/0 10.23.67.3
MAC/Encaps=4/4, MRU=1504, Label Stack{}
94A10800
No output feature configured
We have an outgoing label of No Label. Why would we have an outgoing label of No Label? Well, we don’ have an outgoing label because of the way the router populates it’s LFIB. The router takes its routing table entry for 3.3.3.3 and compares the next hop found (10.23.67.3) to the list of interface IP addresses on each LDP neighbour to determine which LDP neighbour is the next hop.
If we look at R6’s LDP neighbours we see:
R6#sh mpls ldp neighbor
Peer LDP Ident: 2.2.2.2:0; Local LDP Ident 6.6.6.6:0
TCP connection: 2.2.2.2.646 - 6.6.6.6.59628
State: Oper; Msgs sent/rcvd: 1353/1344; Downstream
Up time: 19:21:52
LDP discovery sources:
Serial1/0, Src IP addr: 10.23.67.2
Addresses bound to peer LDP Ident:
10.23.67.2 2.2.2.2
We only have one neighbour, and that neighbour doesn’t have 10.23.67.3 as one of its interfaces. If our only LDP neighbour doesn’t have the next hop as one of its interfaces then we can’t possibly assign an outgoing label for 3.3.3.3 can we? Of course not!
So we have No Label.
What happens to our illustrious packets then? Well with an outgoing label of No Label ALL labels are stripped off, including the inner VPN label, and the router tries to process the packet normally. In this case this means the router will look up 5.5.5.5 in the global routing table, fail, and drop the packet. Things are broken.
Luckily the fix to this is fairly simple for labbing. If you ever had a real network like this you should shoot your designer…
To fix this we’re simply going to change the network type on our frame interfaces to point-to-multipoint.
R2(config)#int s1/0
R2(config-if)#ip ospf net point-to-mul
R2(config-if)#
*Aug 30 03:22:47.587: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Serial1/0 from 2WAY to DOWN, Neighbor Down: Interface down or detached
*Aug 30 03:22:47.587: %OSPF-5-ADJCHG: Process 1, Nbr 6.6.6.6 on Serial1/0 from 2WAY to DOWN, Neighbor Down: Interface down or detached
R2(config-if)#
*Aug 30 03:23:44.491: %OSPF-5-ADJCHG: Process 1, Nbr 6.6.6.6 on Serial1/0 from LOADING to FULL, Loading Done
*Aug 30 03:24:13.019: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Serial1/0 from LOADING to FULL, Loading Done
R2(config-if)#
I’ll let you use your imagination for what this looks like on R3 and R6.
Let’s circle back and check all our routing information for 5.5.5.5 from VRF_D.
BGP:
R6#sh bgp vpnv4 uni all 5.5.5.5
BGP routing table entry for 4:4:5.5.5.5/32, version 62
Paths: (1 available, best #1, table VRF_D)
Not advertised to any peer
Local
3.3.3.3 (metric 129) from 2.2.2.2 (2.2.2.2)
Origin incomplete, metric 65, localpref 100, valid, internal, best
Extended Community: RT:4:4 OSPF DOMAIN ID:0x0005:0x000000030200
OSPF RT:0.0.0.0:2:0 OSPF ROUTER ID:33.33.33.33:27126
Originator: 3.3.3.3, Cluster list: 2.2.2.2
mpls labels in/out nolabel/18
No change there. None was needed. Now our IP route:
R6#sh ip route 3.3.3.3
Routing entry for 3.3.3.3/32
Known via "ospf 1", distance 110, metric 129, type intra area
Last update from 10.23.67.2 on Serial1/0, 00:03:02 ago
Routing Descriptor Blocks:
* 10.23.67.2, from 3.3.3.3, 00:03:02 ago, via Serial1/0
Route metric is 129, traffic share count is 1
This has changed. Our next hop is now R2 and not R3. This should solve our LFIB problem…
R6#sh mpls forwarding-table 3.3.3.3 detail
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
17 17 3.3.3.3/32 0 Se1/0 10.23.67.2
MAC/Encaps=4/8, MRU=1500, Label Stack{17}
94A18847 00011000
No output feature configured
And we have an outgoing label! Excellent.
Just for fun, we ping it.
R8#ping 5.5.5.5 sou lo0To wrap up I owe a thanks to Brian McGahan for taking the time in IRC to help me understand parts of this.
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 5.5.5.5, timeout is 2 seconds:
Packet sent with a source address of 8.8.8.8
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/41/44 ms
What's IRC? Can you give a URL?
ReplyDeletehttp://en.wikipedia.org/wiki/Internet_Relay_Chat
DeleteGreat information, thanks for sharing this good article about mpls and bgp network. Keep sharing such good stuff.
ReplyDelete