This article is taken from http://www.subnetzero.info/2014/04/10/rib-group-confusion/
Continuing on the subject of confusing Junos features, I’d like to talk about RIB groups. When I started here at Juniper, I remember being utterly baffled by this feature and its use. RIB groups are confusing both because the official documentation is confusing, and because many people, trying to be helpful, say things that are entirely wrong. I do think there would have been an easier way to design this feature, but RIB groups are what we have, so that’s what I’ll talk about.
Routing Tables
First, for those who do not have a lot of experience with Juniper routers or service provider routing, the router seems like a simple enough device. A packet comes in, its destination is evaluated against the routing table, and the packet exits on the interface that table points to.
Note: I am not considering the details of the forwarding table in this post. I am assuming the router routes the way we were taught in Networking 101.
This works fine, but there are some situations which require a router to run multiple routing tables. For example, when MPLS Layer 3 VPNs came along there was a need for multiple customers to attach to the same router, and potentially share overlapping routes. For example, customer A attaches to the telco provider edge router (PE) and sends route 172.16.0.0/16. Customer B attaches and sends 172.16.1.0/24. The router needs to separate the routing tables to allow the overlapping routes.
How does the router know which routing table to consult when forwarding a packet? If the packet comes in on an interface that is configured to be a part of the VRF, then the router knows (by configuration) to consult the table for that VRF. If the packet comes in on an interface that is not a part of the VRF, then the router looks at the MPLS label on the packet to determine which routing table to consult. The point here isn’t an MPLS refresher, but to show how a router can have multiple routing tables which exist independently.
Enter RIB Groups
Junos makes it quite easy to create multiple routing tables, even outside of an MPLS VPN context. We can create a routing table and call it whatever we want by using the rib-group command under the routing-options hierarchy:
root@r1> show route table inet.51
error: No routing tables matching specification.
root@r1> configure
Entering configuration mode
[edit]
root@r1# set routing-options rib-groups test import-rib [inet.0 inet.51]
root@r1# commit and-quit
commit complete
Exiting configuration mode
root@r1> show route table inet.51
root@r1>
More on the “import-rib” syntax in a minute. First, note that no routing table “inet.51” exists at first, and Junos tells me this when I try to examine it. After I create the rib group with inet.51 in it, Junos stops giving me the error. In other words, the table now exists, but it is empty. Important point: creating the RIB with the import-rib command sets up a relationship between two routing tables, but it does not actually import routes. We need additional commands for that.
Note: While I created inet.51 out of thin air, Junos reserves some routing table names, like inet.2 and inet.3, for specific purposes.
Let’s bring some routes into our table. We can import them from any protocol, but we’ll start with the routes for interfaces on the router itself. First, note that the inet.0 table has routes for the directly attached interfaces:
root@r1> show route
inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 06:45:58
> to 10.92.31.254 via ge-0/0/0.0
10.92.16.0/20 *[Direct/0] 06:45:58
> via ge-0/0/0.0
10.92.24.129/32 *[Local/0] 06:46:00
Local via ge-0/0/0.0
172.16.123.0/24 *[Direct/0] 06:45:30
> via ge-0/0/1.0
172.16.123.1/32 *[Local/0] 06:45:30
Local via ge-0/0/1.0
172.255.255.1/32 *[Direct/0] 06:45:30
> via lo0.0
Remember, inet.51 is empty at this point. We can tell Junos to use RIB group “test” to propagate interface routes. This is done under the routing-options hierarchy:
[edit]
root@r1# set routing-options interface-routes rib-group test
root@r1# commit
commit complete
root@r1# run show route table inet.51
inet.51: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.92.16.0/20 *[Direct/0] 00:00:14
> via ge-0/0/0.0
10.92.24.129/32 *[Local/0] 00:00:14
Local via ge-0/0/0.0
172.16.123.0/24 *[Direct/0] 00:00:14
> via ge-0/0/1.0
172.16.123.1/32 *[Local/0] 00:00:14
Local via ge-0/0/1.0
172.255.255.1/32 *[Direct/0] 00:00:14
> via lo0.0
We can see here that the routes associated with interfaces, which are in inet.0 by default, are now in inet.51.
Import-RIB: Source and Destination?
A common misunderstanding about the import-rib statement is that the first table listed is the source and the second is the destination. This is not quite true. Remember, the tables are put into a group. There is indeed a significance to the first table listed in the command, as we shall see later, but it is not a source/destination relationship. This can be seen by reversing the order of the tables listed in the command. I did this on another router:
root@r4> show route
inet.0: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 06:53:37
> to 10.92.31.254 via ge-0/0/0.0
10.92.16.0/20 *[Direct/0] 06:53:37
> via ge-0/0/0.0
10.92.25.119/32 *[Local/0] 06:53:38
Local via ge-0/0/0.0
172.19.45.0/24 *[Direct/0] 06:52:55
> via ge-0/0/1.0
172.19.45.4/32 *[Local/0] 06:52:55
Local via ge-0/0/1.0
172.255.255.4/32 *[Direct/0] 06:52:55
> via lo0.0
root@r4> configure
Entering configuration mode
[edit]
root@r4# set routing-options rib-groups test import-rib [inet.51 inet.0]
root@r4# set routing-options interface-routes rib-group test
root@r4# commit
commit complete
root@r4# run show route table inet.51
inet.51: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.92.16.0/20 *[Direct/0] 00:00:05
> via ge-0/0/0.0
10.92.25.119/32 *[Local/0] 00:00:05
Local via ge-0/0/0.0
172.19.45.0/24 *[Direct/0] 00:00:05
> via ge-0/0/1.0
172.19.45.4/32 *[Local/0] 00:00:05
Local via ge-0/0/1.0
172.255.255.4/32 *[Direct/0] 00:00:05
> via lo0.0
Here we can see that, even though we have reversed the order of the tables in the command, the behavior is exactly the same as when inet.0 was first. inet.51 still gets the interface routes.
Another thing to note is that the default route in inet.0, which is statically configured, did not make it into inet.51. This is what we would expect, since we only told Junos to share interface routes. It will only share routes for the protocols that are explicitly configured to do so (we are considering local/direct to be protocols here). What if we want static routes? We simply need to apply a RIB group command under the routing-options hierarchy for static routes:
[edit]
root@r4# set routing-options static rib-group test
root@r4# commit
[edit routing-options]
'static'
rib group test not legal for rib (null),first rib in ribgroup has to be same as current rib
error: configuration check-out failed
The error here is interesting and gets back to the question of primary versus non-primary tables. Remember that we said the first table listed in the import-rib command has a significance, but that it is not simply the source table. We proved this by reversing the order. So what is the significance? For imports of any routes other than interface routes, the primary table (i.e., the one listed first) must be the table where those routes would normally be placed. In other words, static routes normally go into inet.0. Therefore, inet.0 has to be listed first in the command. Why are interface routes exempted from this rule? I have no idea at all. However, this is what leads to this idea of import-rib being source/destination based.
Note:If you find this confusing, this means you are a normal, thinking individual. Remember that CLIs and user interfaces are written by humans who make mistakes.
To make the static routes import into inet.51, I reverse the order of the rib-group statements. Just for fun, I added another table:
[edit]
root@r4# show routing-options rib-groups
test {
import-rib [ inet.51 inet.0 ];
}
[edit]
root@r4# delete routing-options rib-groups test import-rib inet.51
root@r4# set routing-options rib-groups test import-rib inet.51
root@r4# show routing-options rib-groups
test {
import-rib [ inet.0 inet.51 ];
}
[edit]
root@r4# set routing-options rib-groups test import-rib inet.52
root@r4# show routing-options rib-groups
test {
import-rib [ inet.0 inet.51 inet.52 ];
}
After committing, our static route shows up in both tables:
[edit]
root@r4# run show route table inet.51 protocol static
inet.51: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 00:01:42
> to 10.92.31.254 via ge-0/0/0.0
[edit]
root@r4# run show route table inet.52 protocol static
inet.52: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
0.0.0.0/0 *[Static/5] 00:01:49
> to 10.92.31.254 via ge-0/0/0.0
[edit]
root@r4#
While static routes are imported with a configuration statement under the routing-options, specific protocols are enabled under the protocol hierarchy. I’ve enabled OSPF on my router and sent a route into it. Of course, it does not appear in our new tables until I add the command:
root@r4# run show route protocol ospf
inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.1.57.0/24 *[OSPF/10] 00:00:06, metric 2
> to 172.19.45.5 via ge-0/0/1.0
10.255.255.5/32 *[OSPF/10] 00:00:06, metric 1
> to 172.19.45.5 via ge-0/0/1.0
224.0.0.5/32 *[OSPF/10] 00:00:51, metric 1
MultiRecv
inet.51: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
inet.52: 6 destinations, 6 routes (6 active, 0 holddown, 0 hidden)
[edit]
root@r4# set protocols ospf rib-group test
root@r4# commit
commit complete
root@r4# run show route protocol ospf
inet.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.1.57.0/24 *[OSPF/10] 00:00:05, metric 2
> to 172.19.45.5 via ge-0/0/1.0
10.255.255.5/32 *[OSPF/10] 00:00:05, metric 1
> to 172.19.45.5 via ge-0/0/1.0
224.0.0.5/32 *[OSPF/10] 00:01:16, metric 1
MultiRecv
inet.51: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.1.57.0/24 *[OSPF/10] 00:00:05, metric 2
> to 172.19.45.5 via ge-0/0/1.0
10.255.255.5/32 *[OSPF/10] 00:00:05, metric 1
> to 172.19.45.5 via ge-0/0/1.0
inet.52: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.1.57.0/24 *[OSPF/10] 00:00:05, metric 2
> to 172.19.45.5 via ge-0/0/1.0
10.255.255.5/32 *[OSPF/10] 00:00:05, metric 1
> to 172.19.45.5 via ge-0/0/1.0
After applying the command, both inet.51 and inet.52 have the OSPF routes.
Note: They are missing the 224 route, but that’s a multicast route and a different story.
Inter-VRF Route Leaking
Until now I’ve been dealing with routing tables of my own invention. They show how routes are imported into RIB groups, but they don’t actually route traffic. In other words, they don’t actually do anything. To continue the discussion we need a more practical example, with actual working routing tables. One way to do this is to use different VRFs, since each VRF has its own table. So, I’ve set up a router with two VRFs.
In this case we have three inet.0 routing tables: one for each VRF and the global table. I’m advertising routes via BGP into each VRF:
root@r12# run show route protocol bgp
inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
vrf1.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.255.255.10/32 *[BGP/170] 01:20:01, localpref 100
AS path: 1000 I, validation-state: unverified
> to 172.17.102.10 via ge-0/0/1.0
vrf2.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.255.255.11/32 *[BGP/170] 01:22:32, localpref 100
AS path: 1100 I, validation-state: unverified
> to 172.18.112.11 via ge-0/0/2.0
VRFs are designed to isolate traffic, so we normally want these tables to be totally separate. But, if we want, we can share or “leak” routes from one to the other using RIB groups. To set this up, first remember that we define the routing table relationship using the RIB group commands under routing options, with the first table listed being the primary table, i.e., the table where the route normally lives. I’ll import the vrf1.inet.0 10.255.255.10 route into the vrf2.inet.0 table:
[edit]
root@r12# set routing-options rib-groups vrf_rib_group import-rib [vrf1.inet.0 vrf2.inet.0]
Note that this is done at the global level and not in a VRF. Just as in my earlier inet.51 examples, this doesn’t actually do anything—it just sets up the relationship. Now we need to make it happen under the specific protocol, in this case BGP:
[edit]
root@r12# set routing-instances vrf1 protocols bgp family inet unicast rib-group vrf_rib_group
root@r12# commit
commit complete
root@r12# run show route protocol bgp
inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
vrf1.inet.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.255.255.10/32 *[BGP/170] 01:30:05, localpref 100
AS path: 1000 I, validation-state: unverified
> to 172.17.102.10 via ge-0/0/1.0
vrf2.inet.0: 4 destinations, 4 routes (4 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
10.255.255.10/32 *[BGP/170] 00:00:04, localpref 100
AS path: 1000 I, validation-state: unverified
> to 172.17.102.10 via ge-0/0/1.0
10.255.255.11/32 *[BGP/170] 01:32:36, localpref 100
AS path: 1100 I, validation-state: unverified
> to 172.18.112.11 via ge-0/0/2.0
We can now see the route from vrf1 inserted into vrf2. There will be some reachability problems, which I could solve here by importing the interface routes as well, but I don’t want to go that far in an already lengthy article (and we’re not finished yet!)
At this point you may be asking why I put the BGP statement under vrf1. In the first place, if you put it under vrf2 it fails with an error. In the second, we have to remember that the RIB group statement is telling the ingress routing protocol where to put the information it’s learned. vrf1 BGP is learning the route, therefore vrf1 is the place to put the command.
Why export-rib?
Hopefully you are keeping up with me so far. If you have looked at RIB groups before, you have probably encountered the export-rib command. Juniper’s documentation, even if it didn’t have excess commas, is confusing on this matter. Most people basically establish in their mind that the primary to secondary table relationship with the import-rib command is a source/destination relationship, even if it isn’t quite. But then we are confusingly told that the “export-rib” command specifies the table from which routes are taken. Well, when we’ve configured an import-rib statement and we have routes importing from one table to another, why on Earth would we need it?
I have some good news for you. I have searched high and low for an explanation of the export-rib command. I have read everything that is out there; I have examined every bug filed on it. I have contacted experts, including developers here at Juniper. And I have concluded it is useless. The two or three examples I have seen of its use, primarily in inet.2 RPF situations, worked just fine without it. So forget about it. Ignore it. Act like it doesn’t exist. I have found one case where a config won’t commit without it, but I won’t even mention it unless I get a lot of comments from people who have run into it.
In summary:
- If you think RIB groups are confusing, that’s because they are.
- I seriously think we could have implemented this feature better and more consistently.
- That said, a RIB group defines a relationship between routing tables allowing them to share routes.
- The first table listed is technically not the source, but basically is, so you can think of it that way if it helps.
- After defining the relationship between routing tables with the import-rib command, you can actually move routes into the tables by applying the RIB group under the appropriate protocol.
- RIB groups can be used to leak routes between VRFs.
- The “export-rib” command is confusing and not needed in any scenario I know of.