Mikrotik RouterBoard 450G BGP Testing
As you know, or should know, shame on you for not knowing, the 450G is the new 5 port gig router. It’s features at a glace:
- 5 gig ports, though port 1 to any port is full gig while all other ports will only get ~650Mb between each other
- METArouter support
- 256 megs of ram (full internet route table will require around 100MB)
- 680Mhz processor.
Tests running code 3.23.
***265K BGP Routes***
Using Quagga (I wrote an article on easily configuring Quagga), I advertised around 265K routes over to him…the internet route table is around 256K routes, so it is a pretty accurate test.
With No Traffic Running
When bringing up a single peer, the route table isn’t completely full for around 2 minutes on average. Also, the CPU stays at 100% for right around 3 minutes on average.
When bringing up the second peer it takes around 5 minutes for all routes to hit the 450G and CPU stays at 100% for around 7 minutes.
With Traffic Running
80Mb throughput test
While running the bandwidth tests, the CPU remains at 100% with or without BGP enabled.
From the time the first peer starts to come up my throughput averaged 40Mb. It took roughly 5 mins 40 secs on average to pull the routes. My throughput didn’t go back up until 7.4 minuts had passed, on average.
Second peer coming up takes the same amount of time as the first peer.
I noted that once you have both peers established and stable, if you drop one peer the router becomes pretty much completely unresponsive to winbox or serial. My throughput remains at 40Mb for about 15 minutes, at which time the only remaining peer flaps. After the flap the router reestablishes and begins to rectify the route table. For this entire 20 minute procedure, the route table remains the same; all routes still exist, even from the missing peer! The new peer takes the average loaded time to complete.
Here is a little screen shot of the failure:
Brick house
2Mb Throughput Test
First and second peer take roughly 2 mins to pull routes and roughly 5 mins for the CPU to return to normal.
The same peer flap happens even with a lightly loaded link. We have two full peers and drop one. After 20 minutes of the router being unresponsive to administration, the second peer flaps and then everything starts to respond again. During this 20 minute interval, all routes from both peers, even the dropped peer, remain. I need to test if this bug exists when I do a larger server based install. Not to say this is really a bug…I believe it more to be the fact that the 450G just doesn’t have enough CPU.
***10 BGP Routes***
Peers come up and down withing seconds. CPU is virtually unaffected. Everything works flawlessly. I’m pretty sure this is the realm this router was designed for. This could be connecting a small internet connection as well as peering you into a provider’s MPLS network.
Summary
| Throughput | # BGP Routes | 1 peer pull routes | 1 peer normalize | 2nd Peer pull routes | 2nd peer normalize |
| 2Mb | 10 | 5 Secs | 10 Secs | 5 Secs | 10 Secs |
| 2Mb | 265,000 | 2 Mins | 5 Mins | 2 Mins | 5 Mins |
| 80Mb | 10 | 5 Secs | 10 Secs | 5 Secs | 10 Secs |
| 80Mb * | 265,000 | 5.5 Mins | 7.5 Mins | 5.5 Mins | 7.5 Mins |
*40Mb was attainable while processing was being performed
All in all, my impressions are positive. I wouldn’t suggest pulling the full internet route table with one of these guys, though they are technically capable of it. As you could see from the testing, definately DO NOT dual peer these guys with the full internet route table! If one of your peers drops you will black hole your traffic for around 25 mins, or if you reboot the router, you can cut that down to about 5 minutes. Also, if you only have a single peer and you flap the link, you pay a 10 minute or so penalty until the peer will reestablish.
I’ve also noticed that when you flap peers that the prefix statistics gets way off, AKA the count of routes learned from each peer. This would be another bug.
On the other hand, if you use this router as it was intended, it works splendidly! If you are holding a few hundred routes via BGP, you can rely on this guy to work efficiently.
450G loaned for testing by Learnmikrotik.com
Thanks for the stats. I’d bet that Mikrotik’s successor to the 450G will have lots more horsepower to handle the large route tables.
Tom,
I’m not so sure. I think perhaps they will say “if you want to run the full table, upgrade to the RB1000.” I’ve got one sitting here for a little bench testing, so I’ll better know how well it will perform with the full table. This guy, I assume, will hold the table quite well, but one never knows 🙂 If it handles it well, I’ll do some filtering and such to further stress it.
Greg
Thank you for the nice article! We are actually working on this speed issue and we are quite confident that the performance will be better.
Normunds,
Thanks for reading! I feel like even at the current speed of the box it is quite adequate as long as you use it in the right application.
I’m also pleased to see you guys are monitoring its use out in the wild.
Greg
I test rb450G with Quagga simulated full route and consume less mem ram that real full bgp table. In test router its stable with 40Mb remains ram. But with real full bgp table, the router is freeze because bgp consumes all ram. In all other systems for calculate ram consumption is 1 route = 1Kb ram.
Its possible that difference between test and real full bgp tables resides in BGP atributes, in test all tables have same as-path, in real full bgp table every route is different with different AS-path.
@tronity
Indeed sir, good point. Not only will they have more transitive attributes, but they will also be in constant flux. The full internet route table changes on a regular basis, so this will cause recalculation on a regular basis.