Traffic Engineering for L2VPN Services. Part 2
In the previous post, I’ve described how I was seeking a working solution for a tactical traffic engineering with LDP L2VPNs. Eventually, such a solution was found, but it had several significant constraints. The software release on routers of interest was updated to a fresher state. Thus, I decided to improve the situation.
The most painful limitation described at the end of the last post was the lack of CSPF support. This is a point that I target now.
Intent
The industry has long had an idea to bring the support of intent into networking. This idea is described in RFC 9315. From its terminology, we can see that the intent is:
A set of operational goals (that a network should meet) and outcomes (that a network is supposed to deliver) defined in a declarative manner without specifying how to achieve or implement them.
Later in Section 3.1, the intent is clarified much deeper.
RFC9315 is an informational standard that gives us reasons to think in an intent-ful way, but it does not provide (and does not have to) any tools.
It is not surprising that big vendors started elaborating on these tools. There are two competitors right now. One belongs to Cisco. The second is yet another Juniper’s try to show they are also capable. BGP-CAR versus BGP-CT, correspondingly.
At the moment of writing the IDR WG has a rough consensus to move both solutions further as experimental and see what would happen in real deployments. The reason why I bring this topic up is Juniper already ships its solution, and starting from the JunOS 21.2 (if I’m not mistaken) everybody is free to taste it.
As the software on routers of interest were updated there are no more obstacles for me too.
Colors
Let’s briefly remember our intent: avoid high latency paths for a required set of services. This is a declarative form without any implementation specifics.
In practice, an intent is defined as a specific color, which can take the form of a special extended BGP community. This community comes from the other work, RFC 9012, but many others adopted it for their own needs, especially since the TEA work does not strictly define the color value field.
The solution provided by BGP-CT is complicated and covers lots of cases, specifically targeting the Inter-Area and Inter-AS options. It obviously fully leans on BGP to disseminate all the necessary stuff. As it may be clear from the previous post, almost all of it I do not need in my simple case (even BGP).
All that I need is to mark RSVP LSPs and specific services with my color value to tie them together. This use-case seems to be at least as useful as its big brother to me. Some other vendors (I know at least about Nokia) had this option long before. Others provided the same for SR-based transport and only now <irony>legacy</irony> stuff is finally covered.
So, I don’t consider designing a full-blown CT solution but use some of its conceptions.
Solution Description
Finally! I told you a boring story you hadn’t asked about, now we can proceed with the solution.
I started researching the 21.4 release in a virtual environment. It is not the same as in the production, but it is close to the lowest possible (across the updated ones of course).
In the previous post, I described a concern where having additional LSPs toward a destination poses a problem with an unsolicited sharing of these LSPs for all services on an ingress LER. This was nothing surprising because these LSPs reside in a common auxiliary table, inet.3. Now, we can set the traffic-class option for an LSP to tell the system this LSP is special.
root@ls:ingress> show configuration routing-options
route-distinguisher-id 192.0.2.2;
router-id 192.0.2.2;
autonomous-system 64512;
transport-class {
name low-latency {
color 10;
}
}
forwarding-table {
export ps-all-lb;
}
root@ls:ingress>
Note the route-distinguisher-id knob which is needed for CT routes. We don’t use them but without this knob, we can’t commit any traffic classes. Also, note the color knob. This is a network-wide value that we will use later.
As I said before, the traffic-class is applied for an LSP. Let’s configure it for an additional LSP that excludes a specific path constraint.
root@ls:ingress> show configuration protocols mpls
admin-groups {
ag-high-latency 0;
}
label-switched-path lsp-pf-to-egress-1 {
to 192.0.2.3;
admin-group exclude ag-high-latency;
transport-class low-latency;
}
label-switched-path lsp-be-to-egress-1 {
to 192.0.2.3;
node-link-protection;
}
interface lt-0/0/0.1 {
admin-group ag-high-latency;
}
interface lt-0/0/0.6;
root@ls:ingress>
So far, so good. We have two working LSPs one of which avoids the lt-0/0/0.1 link.
root@ls:ingress> show mpls lsp ingress
Ingress LSP: 2 sessions
To From State Rt P ActivePath LSPname
192.0.2.3 192.0.2.2 Up 0 * lsp-be-to-egress-1
192.0.2.3 192.0.2.2 Up 0 * lsp-pf-to-egress-1
Total 2 displayed, Up 2, Down 0
root@ls:ingress> show rsvp session name lsp-be-to-egress-1 detail | match sentto
PATH sentto: 10.0.0.0 (lt-0/0/0.1) 2 pkts
root@ls:ingress> show rsvp session name lsp-pf-to-egress-1 detail | match sentto
PATH sentto: 10.0.0.7 (lt-0/0/0.6) 1 pkts
root@ls:ingress>
But what about my concern? Will any services use the second LSP? No, JunOS has a special auxiliary table for every defined color in the system. This table is not used by any service by default. Note 10 in its name.
root@ls:ingress> show route table junos-rti-tc-10.inet.3
junos-rti-tc-10.inet.3: 1 destinations, 1 routes (1 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.0.2.3/32 *[RSVP/7/1] 06:01:16, metric 200
> to 10.0.0.7 via lt-0/0/0.6, label-switched-path lsp-pf-to-egress-1
root@ls:ingress>
Service Mapping
Remember we have a service of interest. Now we need to map this service into the table above. Thus can be achieved by setting up a color community for the service.
root@ls:ingress> show configuration policy-options
policy-statement ps-all-lb {
then {
load-balance per-flow;
}
}
community cm-l2-low-latency-svc-1 members color:0:10;
root@ls:ingress> show configuration protocols l2circuit
neighbor 192.0.2.3 {
interface ge-0/0/0.10 {
virtual-circuit-id 10;
control-word;
community cm-l2-low-latency-svc-1;
pseudowire-status-tlv;
}
}
root@ls:ingress>
Here we can see 10 again. This is our glue!
root@ls:ingress> show route table l2circuit.0 protocol l2circuit
l2circuit.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.0.2.3:CtrlWord:4:10:Local/96
*[L2CKT/7] 06:10:35, metric2 200
> to 10.0.0.7 via lt-0/0/0.6, label-switched-path lsp-pf-to-egress-1
root@ls:ingress> show route forwarding-table table default family mpls | find CCC
ge-0/0/0.10 (CCC) user 0 indr 1048574 2
10.0.0.7 Push 299776, Push 299776(top) 860 2 lt-0/0/0.6
...
root@ls:ingress>
The service is mapped into the table with a common value for the color. That’s all.
Mapping Approach
An important thing to consider is where I mapped a service to the intent. This is done at ingress. Frankly speaking, the colorful architecture expects you to do so at egress. For this, you need to attach the community to a BGP route. Having this route from any peer, an ingress PE will resolve the route’s next-hop via the corresponding color table. In my case, an LDP service does not provide the same option and it is only possible to bind it at ingress. Nevertheless, it is some kind of hack.
Fallback
This scheme supports a fallback on a regular LSP in case the designated one falls. If we set the disable knob for the lsp-pf-to-egress-1 we will see the services jumps to the lsp-be-to-egress-1 immediately.
You can disable the fallback in the configuration of the traffic-class.
Bypass
A sharp eye noticed I didn’t provide the extra LSP with any protection. That was by design. The reason for that is I had an issue in my lab. Before I provide further details, I must tell you that we didn’t have the same on real hardware. Maybe this is a vMX bug or a bug of the 21.4 release (in production we deployed the feature on 23.4). But if you stumble upon it by chance, it may be useful to know the roots.
The ugliest part here is that RIBs tell you there is something wrong but you can easily overlook it. The LSP name is correct but its interface is not. I hope no one ever faces it!
root@ls:ingress> show route table l2circuit.0 protocol l2circuit
l2circuit.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
192.0.2.3:CtrlWord:4:10:Local/96
*[L2CKT/7] 00:03:24, metric2 200
> to 10.0.0.0 via lt-0/0/0.1, label-switched-path lsp-pf-to-egress-1
root@ls:ingress>
Only FIB (lol, a good name for a web service where nasty dirty routers show you their inwards for money!) will give you the sore truth alone.
root@ls:ingress> show route forwarding-table table default family mpls | find CCC
ge-0/0/0.10 (CCC) user 0 indr 1048574 2
10.0.0.0 Push 299776, Push 299840(top) 915 2 lt-0/0/0.1
...
root@ls:ingress>
The label 299840 belongs to a bypass. Let’s check it.
root@ls:ingress> show rsvp session name lsp-pf-to-egress-1 extensive | match Bypass
Type: Node/Link protected LSP, using Bypass->10.0.0.7->10.0.0.8
4 Mar 24 11:44:11 Node protection up, using Bypass->10.0.0.7->10.0.0.8
2 Mar 24 11:44:10 Link protection up, using Bypass->10.0.0.7
1 Mar 24 11:44:09 New bypass Bypass->10.0.0.7->10.0.0.8
root@ls:ingress>
Now I switch to an intermediate LSR (right behind the lt-0/0/0.1 link) and ask it politely what this label is for.
root@ls:lsr> show route label 299840
mpls.0: 13 destinations, 13 routes (13 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
299840 *[RSVP/7/1] 00:05:02, metric 1
> to 10.0.0.3 via lt-0/0/0.2, label-switched-path Bypass->10.0.0.7->10.0.0.8
299840(S=0) *[RSVP/7/1] 00:05:02, metric 1
> to 10.0.0.3 via lt-0/0/0.2, label-switched-path Bypass->10.0.0.7->10.0.0.8
root@ls:lsr>
Boom! Yes, a bypass LSP is used just as a plain one to transport service traffic.
Summary
This time the solution gives us a fully fledged CSPF RSVP LSP. Moreover, it can be used by any other service (L3VPN, VPLS, GRT routing) if they also require the same treatment. One can think that this solution is proprietary but this is not true. For the rest of the network, this LSP is completely similar to any other RSVP LSP. There are no fancy objects in control messages. Finally, as I stated, other vendors go in the same direction providing similar technologies.