Skip to content
Jun 30 / Greg

Cisco CMP – Out Of Band Built Into Your Router

Today I had a customer ask us for assistance in configuring their CMP on their Cisco Sup2T for a 6500 chassis. To which I promptly googled it hehe.

A CMP is the Connectivity Management Processor, AKA out of band manager. This is similar to IPMI in the server world. It lives on inside the supervisor, but it is completely separated out. It has its own proc, ram, interface. If the supervisor tanks, you can still access it via the CMP.

Connecting from the console is simple:

1
2
3
4
5
6
7
8
9
Action
Control Sequence
RP to CMP
Press Ctrl-c and then Shift-m 3 times consecutively:
Ctrl-c Shift-m Ctrl-c Shift-m Ctrl-c Shift-m
 
CMP to RP
Press Ctrl r and then Shift m 3 times consecutively:
Ctrl-r Shift-m Ctrl-r Shift-m Ctrl-r Shift-m

Once there you can use “root” as the user and “default” as the password.

Initial config(IP/Gateway) is equally simple:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Step 1 
switch-cmp# configure terminal
!Enters configuration mode on the CMP.
 
Step 2 
switch-cmp(config)# interface cmpmgmt
!Enters interface configuration mode for the cmp-mgmt interface on either the active or the standby supervisor engine.
 
Step 3 
switch-cmp(config-if)# ip default gateway 192.0.2.10
!Configures the default gateway for the cmp-mgmt interface.
 
Step 4 
switch-cmp(config-if)# ip address 192.0.2.1/16
!Configures the IP address/mask

Pretty cool feature…even the Cisco haters out there can appreciate it ;)

May 28 / thebrotherswisp

TheBrothersWISP 22 – 2015 MUMs

Mike, Miller, Tom and Greg went to the US MUM(well, Mike didn’t) and we have a little chat about it.

Some of the things discussed:
New MUM Hardware. Greg’s thoughts on said hardware.
Mike’s IX midwest-ix
Miller’s IX rva-ix
OpenBGPd – openbsd bgp routing.
BIRD – linux routing.
FastTrack – fastpath with connection tracking.
Greg’s MUM Presentation – using BGP to build QoS.
RoMon – MTK layer 2ish management protocol.
Lots-o-random things inbetween.

To see the video please visit the link below!!!

May 27 / Greg

Mikrotik Expect Script Slow Connection

Tomas was having issues running an expect script that SSH’s into a Mikrotik to pull information. He would load about 80% of the MOTD, then it would hang for 10 seconds, then finish loading.

Thrift had the thought about an issue with terminal emulation…which turned out to be the case.

Tomas says to run with the “ct” additions(disable auto detect and color) as seen here.

I’m putting this here because I will forget and I’ll need this somewhere in the future :P

May 15 / thebrotherswisp

TheBrothersWISP 21 – WISPAmerica 2015

Mike and Wilson went to WISPAmerica 2015 and did a quick live event talking about everything new they saw. Quick input from Scott.

Some of the things discussed:
Telrad
Mimosa
Mike is giving away SFPs lulz.
Lots-o-random things inbetween.

To see the video please visit the link below!!!

Apr 24 / Greg

Using BGP For QoS – MUM 2015

This presentation is an example of using BGP for your QoS policies.

The slides are available here: BGP For QoS Slides (491)
The configs are available here: BGP For QoS (447)
The video is here:

You can use route filters to mark incoming BGP routes. In this case we use route comments. This allows you to also manually add your own entries in with the correct route comment.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
#configure our route filters for twitch, vudu, and our open IX
/routing filter
#Here we use regex matching inside of the AS Path attribute.  We are finding out 
#where the traffic is sourced from, then adding a route comment based on that.
add bgp-as-path="^.*,46489\$" chain=bgp-qos-in comment=Twitch \
    set-route-comment=RCstreamingvideo
add bgp-as-path="^.*,40582\$" chain=bgp-qos-in comment=Vudu set-route-comment=\
    RCstreamingvideo
#Here we are using community strings to mark these routes.  On top of that we are
#also setting the BGP LP above the default of 100 so it will be the preferred route.
add bgp-communities=65101:10 chain=bgp-qos-in comment="OIX set LP 110" \
    set-bgp-local-pref=110 set-route-comment=RCoix
#This is a catch all rule you can enable.  This rule will remove all route comments
#from above.
add chain=bgp-qos-in comment="*clear all*" disabled=yes set-route-comment=""

You will then need to add these to your BGP peer incoming.

1
2
3
/routing bgp peer
add in-filter=bgp-qos-in name=peer-ISP out-filter=bgp-out remote-address=\
    x.x.x.x remote-as=x ttl=default

*Remember that when you apply a filter to a peer, it resets the peer completely.
**Remember that when you adjust these lists, all of your routes from this peer become momentarily disabled while they run through the adjusted filter.

Once this is done we have a script that searches out these specially commented routes. In this case, the route comments must start with “RC”, as in “RCstreamingvideo”. Once it finds a matching entry, it creates an address-list entry using that route’s destination address and naming it whatever the comment was. If we have a route to 10.10.10.0/24 that has a route comment of “RCoffnet”, then it will add an address-list entry named RCoffnet with address 10.10.10.0/24. Before adding the entry, the script will check that this entry doesn’t currently exist, if it does it will remove, then re-add it. All of these entries are set to expire after 24.5 hours.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# create the script
/system script
add name=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":log info\
    \_\"BGP QoS script start\";\r\
    \n#Define Local Var and load data\r\
    \n#loop variables\r\
    \n:local i 0;\r\
    \n#route ip address\r\
    \n:local ipAddress;\r\
    \n#is it marked for us\r\
    \n:local routeMark \"null\";\r\
    \n#route comment\r\
    \n:local routeComment \"null\";\r\
    \n#check the beginning of our routeComment\r\
    \n:local listName \"null\";\r\
    \n\r\
    \n#loop to check the entire routing table\r\
    \n:foreach i in=[/ip rou find] do={\r\
    \n  #grab the route's comment\r\
    \n  :set routeComment [/ip route get \$i comment]\r\
    \n  #check if to make sure the route comment isn't null\r\
    \n  :if (\$routeComment!=\"\") do={\r\
    \n    #grab the first two letters off of the route comment\r\
    \n    set listName [:pick \$routeComment 0 2]\r\
    \n    #make sure the first two letters are RC\r\
    \n    :if (\$listName=\"RC\") do={\r\
    \n      #get the IP address of the route\r\
    \n      :set ipAddress [/ip route get \$i dst-address]\r\
    \n      #log debug info to the log\r\
    \n#      :log info \"\$i - \$routeSize - \$routeMark - \$routeComment - \$li\
    stName - \$ipAddress\";\r\
    \n        #if it is the default gateway don't add it, otherwise add it to th\
    e addresslist for 24.5 hours\r\
    \n          :if (\$ipAddress!=0.0.0.0/0) do={\r\
    \n          /ip firewall address-list rem [find where list=\$routeComment ad\
    dress=\$ipAddress];\r\
    \n          /ip firewall address-list add list=\$routeComment address=\$ipAd\
    dress timeout=88200;}\r\
    \n    }\r\
    \n  }\r\
    \n}\r\
    \n:log info \"BGP QoS script complete\";"

Here’s the script in a human readable form, just to make life a little easier :)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
:log info "BGP QoS script start";
#Define Local Var and load data
#loop variables
:local i 0;
#route ip address
:local ipAddress;
#is it marked for us
:local routeMark "null";
#route comment
:local routeComment "null";
#check the beginning of our routeComment
:local listName "null";
 
#loop to check the entire routing table
:foreach i in=[/ip rou find] do={
  #grab the route's comment
  :set routeComment [/ip route get $i comment]
  #check if to make sure the route comment isn't null
  :if ($routeComment!="") do={
    #grab the first two letters off of the route comment
    set listName [:pick $routeComment 0 2]
    #make sure the first two letters are RC
    :if ($listName="RC") do={
      #get the IP address of the route
      :set ipAddress [/ip route get $i dst-address]
      #log debug info to the log
#      :log info "$i - $routeSize - $routeMark - $routeComment - $listName - $ipAddress";
        #if it is the default gateway don't add it, otherwise add it to the addresslist for 24.5 hours
          :if ($ipAddress!=0.0.0.0/0) do={
          /ip firewall address-list rem [find where list=$routeComment address=$ipAddress];
          /ip firewall address-list add list=$routeComment address=$ipAddress timeout=88200;}
    }
  }
}
:log info "BGP QoS script complete";

We then schedule the script to run every 24 hours. In this way, if new entries are advertised from our peers, we will pick them up every 24 hours. If they stop advertising a route, it will timeout and be removed after 24.5 hours. If the entry still exists, it will be refreshed for another day. This prevents us from having any gaps without our specialized routing.
**Be sure to schedule this in non-peak times; the script can be CPU intensive.

1
2
3
4
5
# schedule our script
/system scheduler
add interval=1d name=bgp-qos on-event=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    apr/22/2015 start-time=04:00:00

Now that we have address-lists created, we can use these in our traditional QoS policies.

First we create mangle rules to mark traffic based on these address-lists.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# our mangle rules to mark OIX connections, then packets
/ip firewall mangle
add action=mark-connection chain=prerouting comment="oix con mark" \
    dst-address-list=RCoix new-connection-mark=oix
add action=mark-packet chain=prerouting comment="oix-in packet mark" \
    connection-mark=oix in-interface=gre-oix new-packet-mark=oix-in \
    passthrough=no
add action=mark-packet chain=prerouting comment="oix-out packet mark" \
    connection-mark=oix new-packet-mark=oix-out passthrough=no
# our mangle rules to mark streaming video based on address lists
# ######streaming mark for twitch
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark BGP" dst-address-list=RCstreamingvideo \
    new-connection-mark=streaming-video protocol=tcp src-address-list=\
    internal-nets
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark L7" connection-mark=no-mark dst-port=80 \
    layer7-protocol=video new-connection-mark=streaming-video protocol=tcp \
    src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=\
    "streaming video in packet mark" connection-mark=streaming-video \
    in-interface=ether3 new-packet-mark=streaming-video-in passthrough=no
add action=mark-packet chain=prerouting comment=\
    "streaming video out packet mark" connection-mark=streaming-video \
    new-packet-mark=streaming-video-out passthrough=no

Next we create or utilize existing queue tree entries to act upon those marked packets.

1
2
3
4
5
6
7
8
9
10
11
# #####the open exchange queue that is outside other queues
# ##streaming video inside the 10mb queue
/queue tree
add max-limit=10M name=in parent=global
add max-limit=10M name=out parent=global
add name=oix-in packet-mark=oix-in parent=global queue=default
add name=oix-out packet-mark=oix-out parent=global queue=default
add limit-at=4M max-limit=10M name=streaming-video-in packet-mark=streaming-video-in parent=in priority=3 \
    queue=default
add limit-at=4M max-limit=10M name=streaming-video-out packet-mark=streaming-video-out parent=out \
    priority=3 queue=default

Here’s all of the CLI’s in one block:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
# #####the open exchange queue that is outside other queues
# ##streaming video inside the 10mb queue
/queue tree
add max-limit=10M name=in parent=global
add max-limit=10M name=out parent=global
add name=oix-in packet-mark=oix-in parent=global queue=default
add name=oix-out packet-mark=oix-out parent=global queue=default
add limit-at=4M max-limit=10M name=streaming-video-in packet-mark=streaming-video-in parent=in priority=3 \
    queue=default
add limit-at=4M max-limit=10M name=streaming-video-out packet-mark=streaming-video-out parent=out \
    priority=3 queue=default
 
# setting up this local router's BGP instance - don't use outside of lab
/routing bgp instance
set default as=65100
 
# our mangle rules to mark OIX connections, then packets
/ip firewall mangle
add action=mark-connection chain=prerouting comment="oix con mark" \
    dst-address-list=RCoix new-connection-mark=oix
add action=mark-packet chain=prerouting comment="oix-in packet mark" \
    connection-mark=oix in-interface=gre-oix new-packet-mark=oix-in \
    passthrough=no
add action=mark-packet chain=prerouting comment="oix-out packet mark" \
    connection-mark=oix new-packet-mark=oix-out passthrough=no
# our mangle rules to mark streaming video based on address lists
# ######streaming mark for twitch
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark BGP" dst-address-list=RCstreamingvideo \
    new-connection-mark=streaming-video protocol=tcp src-address-list=\
    internal-nets
add action=mark-connection chain=prerouting comment=\
    "streaming video connection mark L7" connection-mark=no-mark dst-port=80 \
    layer7-protocol=video new-connection-mark=streaming-video protocol=tcp \
    src-address-list=internal-nets
add action=mark-packet chain=prerouting comment=\
    "streaming video in packet mark" connection-mark=streaming-video \
    in-interface=ether3 new-packet-mark=streaming-video-in passthrough=no
add action=mark-packet chain=prerouting comment=\
    "streaming video out packet mark" connection-mark=streaming-video \
    new-packet-mark=streaming-video-out passthrough=no
 
#setting up our peers - particularly our route filters
/routing bgp peer
add in-filter=bgp-qos-in name=peer-ISP out-filter=bgp-out remote-address=\
    x.x.x.x remote-as=x ttl=default
add in-filter=bgp-qos-in name=peer-OIX out-filter=bgp-out remote-address=\
    172.17.1.2 remote-as=65101 ttl=default
 
#configure our route filters for twitch, vudu, and our open IX
/routing filter
add bgp-as-path="^.*,46489\$" chain=bgp-qos-in comment=Twitch \
    set-route-comment=RCstreamingvideo
add bgp-as-path="^.*,40582\$" chain=bgp-qos-in comment=Vudu set-route-comment=\
    RCstreamingvideo
add bgp-communities=65101:10 chain=bgp-qos-in comment="OIX set LP 110" \
    set-bgp-local-pref=110 set-route-comment=RCoix
add chain=bgp-qos-in comment="*clear all*" disabled=yes set-route-comment=""
 
# schedule our script
/system scheduler
add interval=1d name=bgp-qos on-event=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive start-date=\
    apr/22/2015 start-time=04:00:00
 
# create the script
/system script
add name=bgp-qos policy=\
    ftp,reboot,read,write,policy,test,password,sniff,sensitive source=":log info\
    \_\"BGP QoS script start\";\r\
    \n#Define Local Var and load data\r\
    \n#loop variables\r\
    \n:local i 0;\r\
    \n#route ip address\r\
    \n:local ipAddress;\r\
    \n#is it marked for us\r\
    \n:local routeMark \"null\";\r\
    \n#route comment\r\
    \n:local routeComment \"null\";\r\
    \n#check the beginning of our routeComment\r\
    \n:local listName \"null\";\r\
    \n\r\
    \n#loop to check the entire routing table\r\
    \n:foreach i in=[/ip rou find] do={\r\
    \n  #grab the route's comment\r\
    \n  :set routeComment [/ip route get \$i comment]\r\
    \n  #check if to make sure the route comment isn't null\r\
    \n  :if (\$routeComment!=\"\") do={\r\
    \n    #grab the first two letters off of the route comment\r\
    \n    set listName [:pick \$routeComment 0 2]\r\
    \n    #make sure the first two letters are RC\r\
    \n    :if (\$listName=\"RC\") do={\r\
    \n      #get the IP address of the route\r\
    \n      :set ipAddress [/ip route get \$i dst-address]\r\
    \n      #log debug info to the log\r\
    \n#      :log info \"\$i - \$routeSize - \$routeMark - \$routeComment - \$li\
    stName - \$ipAddress\";\r\
    \n        #if it is the default gateway don't add it, otherwise add it to th\
    e addresslist for 24.5 hours\r\
    \n          :if (\$ipAddress!=0.0.0.0/0) do={\r\
    \n          /ip firewall address-list rem [find where list=\$routeComment ad\
    dress=\$ipAddress];\r\
    \n          /ip firewall address-list add list=\$routeComment address=\$ipAd\
    dress timeout=88200;}\r\
    \n    }\r\
    \n  }\r\
    \n}\r\
    \n:log info \"BGP QoS script complete\";"

When I recorded the video I hadn’t eaten in 12 hours, so I might be just a tad bit loopy…hehe
Please let me know in the comments what you think about life and the pursuit of happiness. Thanks and happy routing.

Apr 3 / Greg

EU Mikrotik MUM 2015 Hardware Annoucements

You can find the announcement PDF HERE.

These aren’t all of the announcements, just the ones I have more interest in.

hAP 951 replacement with a little more kick and a price tag at only $45.

hAP
951 replacement with a little more kick and a price tag at only $45.

FINALLY, a dual freq radio!  This looks like a nice light home version since the 5ghz only has a single chain.

FINALLY, a dual freq radio! This looks like a nice light home version since the 5ghz only has a single chain.

So the hAP AC has the larger form factor with larger dual band antennas.  5ghz has 3 chains and includes a beefier proc.

So the hAP AC has the larger form factor with larger dual band antennas. 5ghz has 3 chains and includes a beefier proc.

No price estimate on this switch, but it could be useful to have the 4 SFP slots on this unit.

No price estimate on this switch, but it could be useful to have the 4 SFP slots on this unit.

Medium point to point for the AC all-in-one.

Medium point to point for the AC all-in-one.

Smaller AC SXT, though they have decent specs.

Smaller AC SXT, though they have decent specs.

The upgrade to the 2011 we've been looking for.  Dual core with all gig.  I want to say I heard that you have gig channels to the proc from either switch chip, so all ports won't be line rate routing.  I know a Kiwi that is particularly happy about the miniPCIe slot.

The upgrade to the 2011 we’ve been looking for. Dual core with all gig. I want to say I heard that you have gig channels to the proc from either switch chip, so all ports won’t be line rate routing. I know a Kiwi that is particularly happy about the miniPCIe slot.

eumum8

This would be the distance AC point to point.  It seems to have the same board as the SXT, but with a larger antenna.

This would be the distance AC point to point. It seems to have the same board as the SXT, but with a larger antenna.

What is most interesting for you guys?

Mar 27 / Greg

Mikrotik hAP Lite – RB941-2nD – Throughput Test

I previously took a look at the hAP Lite here. I finally got a chance to bandwidth test the device and was pleasantly surprised.

There is an x86 server on ether 1 and another server on ether 2. One btest session running 99 instances at full rate.

  • A bare router.
  • CPU 32%.
    Throughput 99Mb.

    hap1

  • One nat accept rule added.
  • CPU 52%
    Throughput 96-98Mb

    hap2

  • One nat rule.
  • One mangle rule connection mark.
  • One mangle packet mark based off of connection mark.
  • CPU 68%
    Throughput 96-98Mb

    hap3

  • One nat rule.
  • One mangle rule connection mark.
  • One mangle packet mark based off of connection mark.
  • One simple queue at 10Mb. *NOTE* adding a queue of 40Mb or greater results in 100% CPU.
  • CPU 24%
    hap4

  • One nat rule.
  • One mangle rule connection mark.
  • One mangle packet mark based off of connection mark.
  • Two filter accept rules.
  • CPU 71%
    Throughput 96-98Mb

    hap5

  • One nat rule.
  • One mangle rule connection mark.
  • One mangle packet mark based off of connection mark.
  • Two filter accept rules.
  • One simple queue at 30Mb on the x86 btest.
  • One wireless client bridged to ether2 running download btest to x86 through router.
  • CPU 97%
    hap6

    As you can see in the last image, we stretched it about as far as we could.

    Since the btest isn’t a real world test, these are only sample values, but you can see that this little router still packs a punch. This guy should be able to do light QoS, firewalling, wifi, and move a good bit of traffic while doing it.

    And for the $20 price-tag, I can see disabling the wireless and using them in quite a few locations to act just as routers or why not as MPLS/VPLS clients ;)